yacy/htroot/CrawlStartSite.html, branch masterYaCy search server with some extra patches
https://git.fennell.dev/yacy/atom?h=master2026-02-09T22:54:34ZRenamed Table_API_p.html -> Automation_p.html2026-02-09T22:54:34ZMichael Peter Christenmc@yacy.net2026-02-09T22:54:34Zurn:sha1:8687ae0e8e7f6cf8028d542ea833d7bb691ad604new limitation documentation2020-12-22T15:33:12ZMichael Peter Christenmc@yacy.net2020-12-22T15:33:12Zurn:sha1:256fa3d9859bcd1922165dba8218361bc3f7aa69
just replaced two by fourenhanced crawl start url check experience2020-01-16T19:59:02ZMichael Peter Christenmc@yacy.net2020-01-16T19:59:02Zurn:sha1:f03e16d3dfce9bf3d8d5b3535954f1c0327aa553
urls are now urlencoded and a check is also performed
in case that an url is copied into the url field using
copy-pasteIssue #156 : new option to clean up (or not) search cache on crawl start2018-02-16T09:19:41Zlucciomanluccioman@users.noreply.github.com2018-02-16T09:19:41Zurn:sha1:519fc9a6009bd81182f9efb16d734f7a552669a7
Prevent also unnecessary search event cache clean-up on each access to
the crawl monitor page (Crawler_p.html).Limit the number of initially previewed links in crawl start pages.2017-06-17T07:33:14Zlucciomanluccioman@users.noreply.github.com2017-06-17T07:33:14Zurn:sha1:0f80c978d67acb6e3ab92e13d3ed0d21d0d8d4ab
This prevent rendering a big and inconvenient scrollbar on resources
containing many links.
If really needed, preview of all links is still available with a "Show
all links" button.
Doesn't affect the number of links used once the crawl is effectively
started, as the list is then loaded again server-side.Removed unnecessary crawlingDomFilterDepth hidden field.2016-10-05T11:48:22Zlucciomanluccioman@users.noreply.github.com2016-10-05T11:48:22Zurn:sha1:b5eb7a9217497e4f45f751f30650eb3c5b46737e
It had incorrect "-UNRESOLVED_PATTERN-" value (see second part of
mantis 691 http://mantis.tokeek.de/view.php?id=691 )
Note : crawlingDomFilterDepth is apparently unused in current (2016)
YaCy code-base. It was also unnecessary because crawlingDomFilterCheck
hidden field is set to "off".enhanced timezone managament for indexed data:2015-04-15T11:17:23ZMichael Peter Christenmc@yacy.net2015-04-15T11:17:23Zurn:sha1:fed26f33a85658d176f9a38227316a7f5b34fa0a
to support the new time parser and search functions in YaCy a high
precision detection of date and time on the day is necessary. That
requires that the time zone of the document content and the time zone of
the user, doing a search, is detected. The time zone of the search
request is done automatically using the browsers time zone offset which
is delivered to the search request automatically and invisible to the
user. The time zone for the content of web pages cannot be detected
automatically and must be an attribute of crawl starts. The advanced
crawl start now provides an input field to set the time zone in minutes
as an offset number. All parsers must get a time zone offset passed, so
this required the change of the parser java api. A lot of other changes
had been made which corrects the wrong handling of dates in YaCy which
was to add a correction based on the time zone of the server. Now no
correction is added and all dates in YaCy are UTC/GMT time zone, a
normalized time zone for all peers.fix for crawl start check2014-06-15T20:50:04Zorbitermc@yacy.net2014-06-15T20:50:04Zurn:sha1:4177c9cf050036d9c4091dc52d49c009890de0aafixed crawl start links after renaming them in latest commit2014-03-31T22:35:58ZMichael Peter Christenmc@yacy.net2014-03-31T22:35:58Zurn:sha1:ba0e3fb0dc4d4ecc9e6ebf7250a5114be53fa8f9made crawl start pages public since they do not reveal individual2014-03-31T18:42:39Zorbitermc@yacy.net2014-03-31T18:42:39Zurn:sha1:d29b6db270c5d1c59dcd384ac46d1bc799b62241
information and they are also not used as servlet to actually start the
crawl (which is Crawler_p.html).