Fetch and analyze a domain's robots.txt rules and sitemap files, including declared sitemap URLs and issues.
Analyze robots.txt rules and sitemap files of a domain and report issues.
Easiest way: the APIHub CLI. Credits cover x402, no wallet needed.
# install and set up once
npx @apihubio/cli register
npx @apihubio/cli topup 10
# call this service
npx @apihubio/cli call -X GET "https://proxy.apihub.io/robots-sitemap/v1/check?domain=example.com"Or call the proxy directly (curl / any HTTP client):
curl https://proxy.apihub.io/robots-sitemap/v1/check?domain=example.com \
-H "Authorization: Bearer ahk_your_key"
# If credits available: charged automatically, returns 200
# Otherwise: returns 402 (pay via @x402/fetch or top up credits)Estimate your monthly cost for Robots and Sitemap Checker.