yacy/source/de/anomic/xml/SurrogateReader.java, branch master YaCy search server with some extra patches https://git.fennell.dev/yacy/atom?h=master 2009-05-26T07:44:22Z - refactoring: added new content package, which will contain connector classes for different types of data sources to import texts into the YaCy index 2009-05-26T07:44:22Z orbiter orbiter@6c8d7289-2bf4-0310-a012-ef5d649a1542 2009-05-26T07:44:22Z urn:sha1:63a0255166ab8d7e27eb072449c936f1c07a7c2d - refactoring: migrated data objects for the new connector classes - added a DAO interface class to specify an abstract interface for database retrieval connector methods git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5977 6c8d7289-2bf4-0310-a012-ef5d649a1542 added a catch Exception to all thread to check if any of them silently dies without any other notification 2009-05-05T06:31:35Z orbiter orbiter@6c8d7289-2bf4-0310-a012-ef5d649a1542 2009-05-05T06:31:35Z urn:sha1:c097531e3d90a73cdcd71ebcd00de8ebfde613bf git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5922 6c8d7289-2bf4-0310-a012-ef5d649a1542 small changes in surrogate reader, wiki code and portal test 2009-04-27T20:30:43Z orbiter orbiter@6c8d7289-2bf4-0310-a012-ef5d649a1542 2009-04-27T20:30:43Z urn:sha1:d079d6dfdb7341d39b054f8edb0b6f43269b03ef git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5894 6c8d7289-2bf4-0310-a012-ef5d649a1542 partly reverting SVN 5818: identical comparator required for join operator 2009-04-17T08:18:01Z orbiter orbiter@6c8d7289-2bf4-0310-a012-ef5d649a1542 2009-04-17T08:18:01Z urn:sha1:9e853e1977fd405344389f90dbe45100648b52ea git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5820 6c8d7289-2bf4-0310-a012-ef5d649a1542 fixed some problems with surrogate reader. This is now ready for testing. 2009-04-16T21:29:41Z orbiter orbiter@6c8d7289-2bf4-0310-a012-ef5d649a1542 2009-04-16T21:29:41Z urn:sha1:7dfe7e7cc6761d22fcb45d80f42073f8931095cf git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5817 6c8d7289-2bf4-0310-a012-ef5d649a1542 removed example lines from SurrogateReader sources; added additional example file 2009-04-16T21:05:34Z orbiter orbiter@6c8d7289-2bf4-0310-a012-ef5d649a1542 2009-04-16T21:05:34Z urn:sha1:3a1364ed5cc20c25264aa48523a050a087fe3267 git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5816 6c8d7289-2bf4-0310-a012-ef5d649a1542 alpha version of surrogate reading and indexing. 2009-04-16T20:47:55Z orbiter orbiter@6c8d7289-2bf4-0310-a012-ef5d649a1542 2009-04-16T20:47:55Z urn:sha1:9050a3c4c547b7bcbe3b0e9505fedc961603a296 see the example file for an explanation. git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5815 6c8d7289-2bf4-0310-a012-ef5d649a1542 full redesign of index access data model: 2009-04-16T15:29:00Z orbiter orbiter@6c8d7289-2bf4-0310-a012-ef5d649a1542 2009-04-16T15:29:00Z urn:sha1:c8624903c64d1525b42053c99e7504a6205a6cdf terms (words) are not any more retrieved by their word hash string, but by a byte[] containing the word hash. this has strong advantages when RWIs are sorted in the ReferenceContainer Cache and compared with the sun.java TreeMap method, which needed getBytes() and new String() transformations before. Many thousands of such conversions are now omitted every second, which increases the indexing speed by a factor of two. git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5812 6c8d7289-2bf4-0310-a012-ef5d649a1542 - added default profile for surrogate indexing 2009-04-16T08:01:38Z orbiter orbiter@6c8d7289-2bf4-0310-a012-ef5d649a1542 2009-04-16T08:01:38Z urn:sha1:bd5f4c78d8d7e84b013ec2ff309a3df62f6d8406 - integrated surrogate indexing into indexing queue process git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5810 6c8d7289-2bf4-0310-a012-ef5d649a1542 added new surrogate reader (surrogates are parsed documents on batches) 2009-04-15T15:30:25Z orbiter orbiter@6c8d7289-2bf4-0310-a012-ef5d649a1542 2009-04-15T15:30:25Z urn:sha1:bc80dc913aa2cd9e9957b2388a6e3d14d2e93991 this will open a new way to insert indexes to YaCy (instead crawling) git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5808 6c8d7289-2bf4-0310-a012-ef5d649a1542