yacy/source/de/anomic/crawler/ZURL.java, branch master

refactoring

2012-09-21T13:48:16Z

redesign of YaCySchema and SolrDoc handling

2012-08-23T07:51:45Z

Abstraction of HandleMap and HandleSet

2012-07-27T10:13:53Z

collection of speed and memory saving hacks

2012-07-13T19:15:38Z

reduction of logging to prevent too much IO caused be logging

2012-07-12T00:08:11Z

added option to record urls that are forwarded to the solr index

2012-06-26T11:54:48Z

refactoring

2012-06-21T22:49:32Z

fixed bug in solr error document

2012-05-14T12:56:21Z

ConcurrentLinkedQueue has a VERY long return time on the .size() method.

2012-02-26T23:42:32Z

See http://docs.oracle.com/javase/6/docs/api/java/util/concurrent/ConcurrentLinkedQueue.html and the following test programm: public class QueueLengthTimeTest { public static long countTest(Queue q, int c) { long t = System.currentTimeMillis(); for (int i = 0; i < c; i++) { q.add(q.size()); } return System.currentTimeMillis() - t; } public static void main(String[] args) { int c = 1; for (int i = 0; i < 100; i++) { Runtime.getRuntime().gc(); long t1 = countTest(new ArrayBlockingQueue(c), c); Runtime.getRuntime().gc(); long t2 = countTest(new LinkedBlockingQueue(), c); Runtime.getRuntime().gc(); long t3 = countTest(new ConcurrentLinkedQueue(), c); System.out.println("count = " + c + ": ArrayBlockingQueue = " + t1 + ", LinkedBlockingQueue = " + t2 + ", ConcurrentLinkedQueue = " + t3); c = c * 2; } } }

Fixed a lot CHMOD rights (no need for execute flag on .java/.html) and introduced local/remote crawl size ratio based check

2011-12-28T23:33:16Z

yacy/source/de/anomic/crawler/ZURL.java, branch master

refactoring

redesign of YaCySchema and SolrDoc handling

Abstraction of HandleMap and HandleSet

collection of speed and memory saving hacks

reduction of logging to prevent too much IO caused be logging

added option to record urls that are forwarded to the solr index

refactoring

fixed bug in solr error document

ConcurrentLinkedQueue has a VERY long return time on the .size() method.

Fixed a lot CHMOD rights (no need for execute flag on *.java/*.html) and introduced local/remote crawl size ratio based check

Fixed a lot CHMOD rights (no need for execute flag on .java/.html) and introduced local/remote crawl size ratio based check