Coverage describes how much of the intended collection a crawler actually captured. It is not simply the number of rows in an export: a large result can still miss important sections, dates, or page types.
Define the denominator
Before measuring coverage, state what counts as the target set. It might be URLs from a sitemap, products in a catalog, pages within a date range, or records returned by a defined query. Then count successful, valid results against that scope. If the total is unknown, report a proxy—such as discovered URLs processed—and label it clearly.
Break results down by useful dimensions: section, page type, status code, or crawl date. This helps distinguish a parser failure from pages the crawler never discovered. Track missing required fields separately from pages that were never fetched; they represent different gaps.
Coverage is a measurement, not a guarantee of completeness. Sites change, links may be inaccessible, and the target set may be ambiguous. Preserve the scope and exclusions with each run so future comparisons mean the same thing.
Leave a Reply