AI Knowledge Crawler
Research ingestion that follows selected sources, extracts usable material and keeps provenance attached, with explicit outcomes when evidence is missing or a run fails.
Research collection becomes unreliable when traversal grows without bounds or extracted text loses the identity of the page it came from. The earlier crawler treated discovery, extraction and validation as separate responsibilities.
Its traversal coordinator followed only eligible links, fed page material through a dedicated extraction boundary and admitted a result only after validation. The current successor applies that separation to bounded research discovery, preserving immutable candidate records with their sources.
Transient fetch work was retry-capable in the dated system. In the successor, usable, empty and failed runs remain distinct: a run without evidence publishes no candidates and cannot silently replace the last valid evidence. The capability evolved; the two implementations are not one runtime.