diff options
| author | Michael Peter Christen <mc@yacy.net> | 2024-05-19 17:19:09 +0200 |
|---|---|---|
| committer | Michael Peter Christen <mc@yacy.net> | 2024-05-19 17:19:09 +0200 |
| commit | 13fbff0bffc78aa4676e92eb0ff95e5a9e98872f (patch) | |
| tree | 7b9184d4eeb79a0eb096be47676092caf27d3154 /defaults | |
| parent | de941c6fee02ff3cf4610d50bccd8759fd5e556f (diff) | |
Added a RAG Proxy for AI Chat with YaCy
RAG (Retrieval Augmented Generation) is a method to combine a search
engine with a LLM (Large Language Model). When a new prompt is
submitted, a search engine injects knowledge from a search into the
content. This is done using a reverse proxy between the Chat Client and
the LLM. In this case, we used the following software:
LLM Backend - Ollama:
https://github.com/ollama/ollama
Install ollama and then load two required LLM models
with the following commands:
ollama pull phi3:3.8b
ollama pull llama3:8b
Chat Client - susi_chat:
https://github.com/susiai/susi_chat
just clone the repository and the open the file
susi_chat/chat_terminal/index.html
in your browser. This displays a chat terminal.
In this terminal, run the following command:
host http://localhost:8090
This sets the LLM backend to your YaCy peer.
Then start YaCy. It will provide the LLM endpoint to the client
while using ollama in the backend. It then injects search results
only from the local Solr index, not from the p2p network (so far).
Diffstat (limited to 'defaults')
| -rw-r--r-- | defaults/web.xml | 12 |
1 files changed, 11 insertions, 1 deletions
diff --git a/defaults/web.xml b/defaults/web.xml index 9a6fbec39..43a500cc1 100644 --- a/defaults/web.xml +++ b/defaults/web.xml @@ -50,6 +50,11 @@ <servlet-class>net.yacy.http.servlets.SolrServlet</servlet-class> </servlet> + <servlet> + <servlet-name>RAGProxyServlet</servlet-name> + <servlet-class>net.yacy.http.servlets.RAGProxyServlet</servlet-class> + </servlet> + <!-- servlet to provide searchresults via proxy --> <servlet> <servlet-name>URLProxyServlet</servlet-name> @@ -81,7 +86,12 @@ <url-pattern>/solr/webgraph/admin/luke</url-pattern> <!-- uncomment this to use YaCy as remote solr --> <!-- <url-pattern>/solr/collection1/update</url-pattern> --> <!-- only with server.response.compress.gzip=false --> - </servlet-mapping> + </servlet-mapping> + + <servlet-mapping> + <servlet-name>RAGProxyServlet</servlet-name> + <url-pattern>/v1/chat/completions</url-pattern> + </servlet-mapping> <!-- eof hardcoded mappings --> <!-- additional (optional) mappings --> |