summaryrefslogtreecommitdiff
AgeCommit message (Collapse)Author
2026-01-23Persist recrawl job to restart on peer bootpr0vieh
2026-01-22HTML5 <meta charset="...> supportJohnny1984
Fixes mojibake when crawling some websites.
2026-01-21Revert performance defaults in yacy.initpr0vieh
2026-01-21Add DNS/network errors (-1) to permanent error status codespr0vieh
- DNS errors (NXDOMAIN, SERVFAIL, UnknownHostException) now treated as permanent failures - Updated default permanentStatus from '404,410' to '404,410,-1' - Added documentation explaining -1 status code represents DNS/network failures - Updated UI description in IndexFederated_p.html to reflect DNS error handling - Affects both transferURL and transferRWI DHT operations
2026-01-21Add configurable DHT error URL blocking with retry window and web UIpr0vieh
- Implement proactive DHT error URL rejection for both URL and RWI transfers - Add configurable opt-out via indexReceiveBlockErrors setting (default: true) - Introduce retry window for temporary errors (default: 30 days) - Permanent errors (404, 410) always blocked, configurable via permanentStatus - Add web UI controls in IndexFederated_p.html under Peer-to-Peer section - Bidirectional feedback: receivers reject and report error URLs to senders - Detailed logging shows blocked vs error-blocked counts separately - Uses load_date_dt field to calculate error age for retry decisions
2026-01-20feat: Add two-layer DHT error propagation to prevent broken URL redistributionpr0vieh
Problem: Failed URLs (404, DNS errors, timeouts) are continuously redistributed via DHT, causing infinite recrawl loops and network-wide index pollution. No mechanism exists in YaCy to communicate error status across peers. Solution - Layer 1 (Proactive Rejection): - Receiver checks local Solr index for httpstatus_i != 200 BEFORE accepting RWI entries - Rejects URL immediately if marked as failed previously - Adds rejected URL hash to errorURL response list - Prevents index pollution at ingestion time - Works even if sender doesn't support errorURL protocol (backward compatible) Solution - Layer 2 (Error Feedback): - Receiver reports rejected error URLs back to sender via errorURL response parameter - Sender receives errorURL list and marks those URLs locally as failed - Sender stops re-distributing these URLs to other peers - Network-wide error propagation prevents repeated distribution cycles Implementation Details: - transferURL.java: Implements proactive rejection + error reporting * Checks incoming URL against Solr error status before storing * Collects rejected URL hashes in errorURLs StringBuilder * Returns errorURL list to sender in response - Protocol.java: Processes error URL feedback from receiver * Extracts errorURL from response * Marks reported URLs locally via crawlQueues.errorURL.push() * Logs DHT error reports for monitoring Benefits: - Dramatically reduces network traffic of broken URLs - Prevents wasted crawl resources on unreachable targets - Maintains clean, usable index across distributed network - Defense-in-depth: two independent layers work together - Backward compatible: old peers ignore errorURL parameter Testing: - Log monitoring shows 'DHT: Received X rejected error URL reports from peer Y' - Proactive rejection shows 'blocked X URLs' in transfer logs - Error URLs automatically removed from circulation
2026-01-11Update startYACY.shAckroydAI
Guidance for editing crontab
2026-01-08fix for no search results within RAG local searchMichael Peter Christen
2026-01-08fixed bug in ai.production_models default value and replaced someMichael Peter Christen
outdated LLMs with new ones
2026-01-08replaced codex with regulaMichael Peter Christen
2026-01-06added global search to chatMichael Peter Christen
2026-01-06updated d3.jsMichael Peter Christen
2026-01-04increased default context sizeMichael Peter Christen
2026-01-04added rendering of <think> tags in LLM responses.Michael Peter Christen
2026-01-03more realistic rate limits for chat request ratesMichael Peter Christen
2026-01-03disabled narrow font in model selectionMichael Peter Christen
2026-01-03added default augmentation setting (attachments / local search) in chat.Michael Peter Christen
2026-01-03Merge pull request #747 from Al2Klimov/patch-1Michael Christen
Fix typo in echo message for downloading ivy
2026-01-03Fix: avoid 0-results by using remote metadata results when availableMichael Peter Christen
Use the remote metadata list directly (via addNodes) after storing/committing remote search results. This bypasses the flaky RWI→metadata lookup that can drop all results and yield a ZERO COUNT even when peers return references.
2026-01-03Fix image preview licensing and show 20 images per page, fixesMichael Peter Christen
https://github.com/yacy/yacy_search_server/issues/438 Use separate one‑time licenses for image thumbnails and full previews to prevent preview failures after thumbnail loads, and set the Images tab to request 20 results by default via the content‑domain switch form.
2026-01-03page number computation fixMichael Peter Christen
2026-01-03Handle POST redirects when contacting peers, fix forMichael Peter Christen
https://github.com/yacy/yacy_search_server/issues/684 YaCy’s peer queries use HTTP POST and were not following redirects, so HTTP→HTTPS redirections (e.g., 8090→8443) failed and peers never switched protocol/port. This adds a redirect strategy that allows POST redirects (preserving method/body, except for 303), letting remote peer connections follow HTTPS redirects correctly.
2026-01-02possible fix for search-0-results bug fromMichael Peter Christen
https://github.com/yacy/yacy_search_server/issues/499
2026-01-01fixed windows installer; updated download path for java installerMichael Peter Christen
2025-12-23Fix typo in echo message for downloading ivyAlexander Aleksandrovič Klimov
2025-12-08added AIShield config page for chat access settings. Also added chat to top ↵Michael Peter Christen
menu in case that was wanted by the user. Also added the AILab to the main menu.
2025-12-06added RAG configuration pageMichael Peter Christen
2025-12-06added build page for the AI labMichael Peter Christen
2025-12-05enhanced snippet computation for RAGMichael Peter Christen
2025-11-30fixed missing line ending in token streamingMichael Peter Christen
2025-11-30fixed font nameMichael Peter Christen
2025-11-30added automatic serch result feedback to chat attachmentMichael Peter Christen
2025-11-30added RAG search to promptsMichael Peter Christen
2025-11-30enable focus on composer during token streaming.Michael Peter Christen
2025-11-30added delete button for chat elements and enhanced focus/scrolling behaviorMichael Peter Christen
2025-11-30Merge branch 'master' of https://github.com/yacy/yacy_search_serverMichael Peter Christen
2025-11-30added text attachments and integrated of text attachments in rag proxy handlingMichael Peter Christen
2025-11-29added fontsMichael Peter Christen
2025-11-23added markdown and code formatting to chat assistant answersMichael Peter Christen
2025-11-25added copy and edit buttons in chat historyMichael Peter Christen
2025-11-24another recommended modelMichael Peter Christen
2025-11-24added image attachments, local storage, clear/download/upload/show-system ↵Michael Peter Christen
buttons to chat
2025-11-24Merge branch 'master' of https://github.com/yacy/yacy_search_serverMichael Peter Christen
2025-11-24enhanced chat cssMichael Peter Christen
2025-11-23Merge branch 'master' of https://github.com/yacy/yacy_search_serverMichael Peter Christen
2025-11-23added search result - to - markdown in RAGMichael Peter Christen
2025-11-23added loop unrolling to tensor dot computationMichael Peter Christen
2025-11-23added chat front-end (stub, no connection to search yet)Michael Peter Christen
2025-11-23refactoringMichael Peter Christen
2025-11-23removed dependency of ByteBuffer on FloatBufferMichael Peter Christen
generated by cgit v1.2.3 (git 2.46.0) at 2026-08-08 09:27:06 +0000