#%env/templates/metas.template%# #%env/templates/header.template%# #%env/templates/submenuAI.template%#

LLM Selection

Here you can pick models from an LLM model service to select them as production model. In the "Production Models Matrix" you can then assign each selected model a function inside YaCy

Install your local LLM service! You need either a local ollama or LM Studio instance running on your local host or inside the intranet.

Service Selection
service
  This makes a preset to the Hoststub value
hoststub
  you can probably leave this to the default value
api_key
  (not required for Ollama or LMStudio)
The selected service's context window (num_ctx) is shown and editable in the Services table below. A model's generated-token cap (max_tokens = Ollama num_predict) is set per model in the Production Models Matrix.
 
Services

num_ctx is the context window (in tokens) of the inference service — a per-service value, shared by all models on that endpoint. It is the total budget for prompt plus generated output; YaCy uses it to size prompts so they leave room to generate. The row for the service selected above appears here automatically with its stored (or default) window. This value is advisory: set it to match the window your backend actually serves (Ollama: OLLAMA_CONTEXT_LENGTH, a Modelfile PARAMETER num_ctx, or the Context Length setting). YaCy does not enforce it on the backend.

#{services}# #{/services}#
service hoststub num_ctx
#[service]# #[hoststub]#
Production Models Matrix #{productionmodels}# #{/productionmodels}#
service model hoststub api_key max_tokens search-answers
infoThis model creates answers for search requests		 chat
infoThis model is used in the chat interface and as default for the RAG proxy		 translation
infoThis model can be used to make translations of the web UI		 classification
infoThis model is used to classify prompts to find out what they demand		 search-query
infoThis model produces search queries to YaCy search from prompts in RAG or chat		 qa-pairs
infoThis model can be used to produce query-answer pairs which enhance search from chat prompts		 tldr-shortener
infoThis model is used to make summaries from web content		 log-report
infoThis model evaluates YaCy runtime logs and creates self-enhancement reports		 thinking
infowe detect thinking only to be able to suppress thinking. thinking is not used in YaCy		 tooling
infotooling is required for agentic abilities.		 vision
infothis enables image recognition in the chat		 format
infothis is required for classification		 Actions
#[service]# #[model]# #[hoststub]# #[api_key]# #[max_tokens]# #[thinking]# #[tooling]# #[vision]# #[format]#
#%env/templates/footer.template%#