MCP web scraping setup
Define a narrow web-fetch tool boundary with explicit limits and provenance.
Define the tool boundary
Decide which documents a tool is allowed to retrieve and how its caller will receive source URLs, timestamps and errors. Keep retrieved content separate from trusted instructions.
Set explicit budgets
Set request, response-size and concurrency budgets. Treat a tool result as untrusted source content and preserve provenance when passing it to an agent.
Evaluate before connecting
Use fixed fixtures to exercise success, timeout and invalid-content cases. The local recommendation engine ranks only the sample capabilities in the canonical dataset; no LLM or MCP server runs in this deployment.
Compare with context
FetchGauge’s current benchmark data is simulated. Use the methodology to understand the intended protocol, and the calculator to explore its cost assumptions.