diff options
| author | orbiter <orbiter@6c8d7289-2bf4-0310-a012-ef5d649a1542> | 2010-09-30 12:50:34 +0000 |
|---|---|---|
| committer | orbiter <orbiter@6c8d7289-2bf4-0310-a012-ef5d649a1542> | 2010-09-30 12:50:34 +0000 |
| commit | f6eebb6f99ef8fd0874be7bc29065c37fec90621 (patch) | |
| tree | b8555d3a9b5ecb5003986dda0e6d59ff6aebc976 /htroot/CrawlProfileEditor_p.html | |
| parent | 63e387508ccacb89d43d8e83f074aa21597ffd6c (diff) | |
replaced auto-dom filter with easy-to-understand Site Link-List crawler option
- nobody understand the auto-dom filter without a lenghtly introduction about the function of a crawler
- nobody ever used the auto-dom filter other than with a crawl depth of 1
- the auto-dom filter was buggy since the filter did not survive a restart and then a search index contained waste
- the function of the auto-dom filter was in fact to just load a link list from the given start url and then start separate crawls for all these urls restricted by their domain
- the new Site Link-List option shows the target urls in real-time during input of the start url (like the robots check) and gives a transparent feed-back what it does before it can be used
- the new option also fits into the easy site-crawl start menu
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7213 6c8d7289-2bf4-0310-a012-ef5d649a1542
Diffstat (limited to 'htroot/CrawlProfileEditor_p.html')
| -rw-r--r-- | htroot/CrawlProfileEditor_p.html | 4 |
1 files changed, 1 insertions, 3 deletions
diff --git a/htroot/CrawlProfileEditor_p.html b/htroot/CrawlProfileEditor_p.html index f00c8d814..0a027828e 100644 --- a/htroot/CrawlProfileEditor_p.html +++ b/htroot/CrawlProfileEditor_p.html @@ -38,8 +38,7 @@ <td><strong>Must Match</strong></td> <td><strong>Must Not Match</strong></td> <td><strong>MaxAge</strong></td> - <td><strong>Auto Filter Depth</strong></td> - <td><strong>Auto Filter Content</strong></td> + <td><strong>Domain Counter Content</strong></td> <td><strong>Max Page Per Domain</strong></td> <td><strong>Accept '?' URLs</strong></td> <td><strong>Fill Proxy Cache</strong></td> @@ -70,7 +69,6 @@ <td>#[mustmatch]#</td> <td>#[mustnotmatch]#</td> <td>#[crawlingIfOlder]#</td> - <td>#[crawlingDomFilterDepth]#</td> <td>#{crawlingDomFilterContent}##[item]#<br />#{/crawlingDomFilterContent}#</td> <td>#[crawlingDomMaxPages]#</td> <td>#(withQuery)#no::yes#(/withQuery)#</td> |
