Adds a disk-backed scrape cache under workspace/web_cache keyed by
the SHA-256 of the URL, honoring WEB_SEARCH_SCRAPE_CACHE_TTL so
repeated lookups within the window skip the network entirely.
Fetches now retry twice with exponential backoff before giving up,
and URLs are screened against SCRAPING_BLOCKED_EXTENSIONS and
SCRAPING_BLOCKED_DOMAINS to avoid downloading binaries or
denylisted hosts.
New scraping knobs (enable flag, max chars, cache TTL, blocklists)
are exposed via .env, alongside a provider switch to stepfun with
the step-3.7-flash default model.
marcocosta10000
5697cd0c40