summaryrefslogtreecommitdiff
path: root/htroot/CrawlProfileEditor_p.html
diff options
context:
space:
mode:
authororbiter <orbiter@6c8d7289-2bf4-0310-a012-ef5d649a1542>2010-09-30 12:50:34 +0000
committerorbiter <orbiter@6c8d7289-2bf4-0310-a012-ef5d649a1542>2010-09-30 12:50:34 +0000
commitf6eebb6f99ef8fd0874be7bc29065c37fec90621 (patch)
treeb8555d3a9b5ecb5003986dda0e6d59ff6aebc976 /htroot/CrawlProfileEditor_p.html
parent63e387508ccacb89d43d8e83f074aa21597ffd6c (diff)
replaced auto-dom filter with easy-to-understand Site Link-List crawler option
- nobody understand the auto-dom filter without a lenghtly introduction about the function of a crawler - nobody ever used the auto-dom filter other than with a crawl depth of 1 - the auto-dom filter was buggy since the filter did not survive a restart and then a search index contained waste - the function of the auto-dom filter was in fact to just load a link list from the given start url and then start separate crawls for all these urls restricted by their domain - the new Site Link-List option shows the target urls in real-time during input of the start url (like the robots check) and gives a transparent feed-back what it does before it can be used - the new option also fits into the easy site-crawl start menu git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7213 6c8d7289-2bf4-0310-a012-ef5d649a1542
Diffstat (limited to 'htroot/CrawlProfileEditor_p.html')
-rw-r--r--htroot/CrawlProfileEditor_p.html4
1 files changed, 1 insertions, 3 deletions
diff --git a/htroot/CrawlProfileEditor_p.html b/htroot/CrawlProfileEditor_p.html
index f00c8d814..0a027828e 100644
--- a/htroot/CrawlProfileEditor_p.html
+++ b/htroot/CrawlProfileEditor_p.html
@@ -38,8 +38,7 @@
<td><strong>Must Match</strong></td>
<td><strong>Must Not Match</strong></td>
<td><strong>MaxAge</strong></td>
- <td><strong>Auto Filter Depth</strong></td>
- <td><strong>Auto Filter Content</strong></td>
+ <td><strong>Domain Counter Content</strong></td>
<td><strong>Max Page Per Domain</strong></td>
<td><strong>Accept '?' URLs</strong></td>
<td><strong>Fill Proxy Cache</strong></td>
@@ -70,7 +69,6 @@
<td>#[mustmatch]#</td>
<td>#[mustnotmatch]#</td>
<td>#[crawlingIfOlder]#</td>
- <td>#[crawlingDomFilterDepth]#</td>
<td>#{crawlingDomFilterContent}##[item]#<br />#{/crawlingDomFilterContent}#</td>
<td>#[crawlingDomMaxPages]#</td>
<td>#(withQuery)#no::yes#(/withQuery)#</td>