diff options
Diffstat (limited to 'help/IndexImportWarc_p.md')
| -rw-r--r-- | help/IndexImportWarc_p.md | 82 |
1 files changed, 82 insertions, 0 deletions
diff --git a/help/IndexImportWarc_p.md b/help/IndexImportWarc_p.md new file mode 100644 index 000000000..ec362af98 --- /dev/null +++ b/help/IndexImportWarc_p.md @@ -0,0 +1,82 @@ +--- +page: htroot/IndexImportWarc_p.html +help: help/IndexImportWarc_p.md +title: Warc Import +package: import-export-federation +access: admin +kind: admin-page +backend_java: source/net/yacy/htroot/IndexImportWarc_p.java +--- + +# Warc Import + +## Purpose + +WARC Import reads web-archive files into YaCy. + +Use it to search archived crawls or preserved web collections. + +## What You Can Do Here + +- WARC Import reads web-archive files into YaCy. +- Define the source, target, format, collection, and expected size before starting. +- Monitor progress because large transfers continue beyond the initial request. + +## Page Architecture + +Import and export pages translate external data formats into YaCy documents or move YaCy index data into another store. Most long-running actions create background work that should be monitored afterward. + +| Control | Meaning | Values or examples | +| --- | --- | --- | +| `file` | Selected or uploaded file. | Text value; use the page label and surrounding context to choose the exact content. | +| `url` | URL to inspect, crawl, import, or act on. | URL or URL-derived value; use the exact format shown by the page. | +| `collection` | Collection name used to group indexed documents. | `user` | +| `submit` | Submits the form. | `Import Warc File` | +| `abort` | File. | `Stop` | + +## Correct Use + +Prepare source and target details before starting: file path or URL, format, collection, credentials if needed, and expected size. Imports and exports can continue in the background, so confirm progress on the related monitor or queue page. + +## Access And Safety + +Administrator access is required. YaCy protects `_p` pages as administration pages. + +Protected related endpoint(s): `/IndexImportWarc_p.html`. + +## Automation And API + +Page backend: `source/net/yacy/htroot/IndexImportWarc_p.java`. + +| Endpoint | Method | Access | Backend | +| --- | --- | --- | --- | +| `/IndexImportWarc_p.html` | `POST` | admin | `source/net/yacy/htroot/IndexImportWarc_p.java` | + +### Parameter Guide + +The table explains values that an agent or script must set deliberately. Parameters not relevant to a task should be omitted or left at the page default. + +| Parameter | Meaning and valid values | Care | +| --- | --- | --- | +| `file` | Selected or uploaded file. | Set only when this option is part of the intended request; otherwise omit it and let YaCy use the page default. | +| `url` | URL to inspect, crawl, import, or act on. | Set only when this option is part of the intended request; otherwise omit it and let YaCy use the page default. | +| `collection` | Collection name. Use it to group crawled or imported documents and to search or manage that group later. | Set only when this option is part of the intended request; otherwise omit it and let YaCy use the page default. | +| `submit` | Submit action for the form. Its meaning depends on the surrounding fields. | Changes stored data, configuration, or a running job. Use the authenticated action flow where required and verify the result. | +| `abort` | File. | Set only when this option is part of the intended request; otherwise omit it and let YaCy use the page default. | + +Example request shape: + +```http +POST /IndexImportWarc_p.html +Content-Type: application/x-www-form-urlencoded + +url=...&file=...&collection=...&submit=...&abort=... +``` + +## What To Expect + +Small operations may finish during the request; large imports, exports, harvests, and package operations usually need monitoring. Expect progress, logs, queue entries, or generated files rather than instant final search quality. + +## Related Pages + +- Related transfer work is usually reached through the import/export page for the same format, the index-pack pages, or the queue/status page that reports progress. |
