Purpose: Single source of truth for the Screaming Frog crawl and ingest configuration. screaming-frog-crawl and screaming-frog-ingest must reference this page so the export-tab string and the required-file list live in one place, not three.

Provenance: Captured 2026-05-25 from screaming-frog-crawl v1.1.0 and screaming-frog-ingest v1.1.0.

Export-tabs string (SF 23.3, verified)

Internal:All,External:All,Response Codes:Client Error (4xx),Response Codes:Server Error (5xx),Response Codes:Redirection (3xx),Page Titles:Missing,Page Titles:Duplicate,Page Titles:Over X Characters,Meta Description:Missing,Meta Description:Duplicate,H1:Missing,H1:Duplicate,Images:Missing Alt Text,Directives:Noindex

Also pass: --save-report "Crawl Overview"

Required output files

internal_all.csv, external_all.csv, response_codes_client_error_(4xx).csv, response_codes_server_error_(5xx).csv, response_codes_redirection_(3xx).csv, page_titles_missing.csv, page_titles_duplicate.csv, page_titles_over_<N>_characters.csv, meta_description_missing.csv, meta_description_duplicate.csv, h1_missing.csv, h1_duplicate.csv, images_missing_alt_text.csv, directives_noindex.csv, crawl_overview.csv, plus crawl.seospider.

IMPORTANT: <N> is the character threshold configured in spider.config. Screaming Frog names the page-titles-over export file after that threshold. Read the threshold from spider.config and build the expected filename from it. Do not hardcode 60. As of 2026-05-25 both screaming-frog skills hardcode page_titles_over_60_characters.csv, which fails the ingest precheck if the threshold is not 60.

PSI toggle

spider.config keeps PSI.auto_connect=true for GUI use. Every headless CLI run must flip it to false and restore it on exit via an EXIT/INT/TERM trap, including on failure. Do not globally disable PSI; that changes the GUI default.

SF 23.3 filter reference

• Page Titles: All, Missing, Duplicate, Over X Characters, Below X Characters, Over X Pixels, Below X Pixels, Same as H1, Multiple, Outside head
• Meta Description: All, Missing, Duplicate, Over X Characters, Below X Characters, Over X Pixels, Below X Pixels, Multiple, Outside head
• H1 / H2: All, Missing, Duplicate, Multiple, Over X Characters, Below X Characters
• Response Codes: All, Success (2xx), Redirection (3xx), Redirection Chains, Client Error (4xx), Server Error (5xx), No Response, Blocked by Robots.txt, Blocked Resource, Internal Indexable, Internal Non-Indexable
• Images: All, Missing Alt Text, Over 100 KB, Missing Size Attributes
• Directives: All, Noindex, Nofollow, Canonicalised, Refresh

Do not invent a filter name. If unsure, run the SF binary with --help export-tabs.