Roundhouse

aeo-crawlers

robots.txt AI-crawler access matrix: for the 2026 cohort of AI and answer-engine crawlers (GPTBot, ChatGPT-User, OAI-SearchBot, ClaudeBot, Claude-User, PerplexityBot, Google-Extended and more) reports allowed, blocked or unlisted per bot, whether the default rule blocks everything, declared sitemaps and a score. One network operation.

SearchLiveeip155:8453Exactvia cdp
https://api.agentstools.dev/aeo/crawlers
AeoRobots-txtAi-crawlerGptbotClaudebot
Calls · 30d
1→ 0%
This endpoint's own trailing-30-day call count, as published by the upstream catalog and snapshotted daily. 30 snapshots so far.
$42.94
Verified settled volume
3,786 settlements proven x402 by their on-chain EIP-3009 marker.
$0.0050
Listed price
As published in the catalog. Always read the live 402 before paying.
1
Calls · 30d
Upstream's own call count for this endpoint, not ours.
1
Unique payers · 30d
Last called 2026-08-23 19:29Z
—
Upstream on-chain volume
Reported by the source catalog.
Paid to
0xF22e558a00D91Ee12A1F50C52186FecB8dDFf493

The wallet the 402 directs payment to. Its whole payment record — every payer, every chain — is on the merchant page.

Asset 0x833589fCD6eDb6E08f4c7C32D4f71b54bdA02913

Provider

The payTo wallet does not resolve to a registered ERC-8004 agent. That is not a verdict on the service — most of the catalog is unregistered.

Live 402 challenge

Captured by the enrichment pass, not read just now. Prices can change — always read the 402 the endpoint answers with.

{
  "error": "Payment required",
  "accepts": [
    {
      "asset": "0x833589fCD6eDb6E08f4c7C32D4f71b54bdA02913",
      "extra": {
        "name": "USD Coin",
        "version": "2"
      },
      "payTo": "0xF22e558a00D91Ee12A1F50C52186FecB8dDFf493",
      "amount": "5000",
      "scheme": "exact",
      "network": "eip155:8453",
      "maxTimeoutSeconds": 300
    }
  ],
  "resource": {
    "url": "https://api.agentstools.dev/aeo/crawlers",
    "tags": [
      "aeo",
      "robots-txt",
      "ai-crawler",
      "gptbot",
      "claudebot",
      "perplexitybot",
      "ai-search",
      "access-control",
      "crawl"
    ],
    "mimeType": "application/json",
    "description": "robots.txt AI-crawler access matrix: for the 2026 cohort of AI and answer-engine crawlers (GPTBot, ChatGPT-User, OAI-SearchBot, ClaudeBot, Claude-User, PerplexityBot, Google-Extended and more) reports allowed, blocked or unlisted per bot, whether the default rule blocks everything, declared sitemaps and a score. One network operation.",
    "serviceName": "aeo-crawlers"
  },
  "extensions": {
    "bazaar": {
      "info": {
        "input": {
          "type": "http",
          "method": "GET",
          "queryParams": {
            "domain": "example.com"
          }
        },
        "output": {
          "type": "json",
          "example": {
            "fixes": [],
            "domain": "example.com",
            "disclaimer": "Automated AI-search visibility indicators, not a guarantee.",
            "crawler_score": 100,
            "crawler_matrix": [
              {
                "bot": "PerplexityBot",
                "access": "allowed",
                "listed": false,
                "operator": "Perplexity",
                "citation_relevant": true
              }
            ],
            "robots_present": true,
            "default_blocks_all": false
          }
        }
      },
      "schema": {
        "type": "object",
        "$schema": "https://json-schema.org/draft/2020-12/schema",
        "required": [
          "input"
        ],
        "properties": {
          "input": {
            "type": "object",
            "required": [
              "type",
              "method"
            ],
            "properties": {
              "type": {
                "type": "string",
                "const": "http"
              },
              "method": {
                "enum": [
                  "GET",
                  "HEAD",
                  "DELETE"
                ],
                "type": "string"
              },
              "queryParams": {
                "type": "object",
                "required": [],
                "properties": {
                  "url": {
                    "type": "string",
                    "description": "Target URL to audit (e.g. https://example.com)"
                  },
                  "host": {
                    "type": "string",
                    "description": "Alternative input: a hostname"
                  },
                  "domain": {
                    "type": "string",
                    "description": "Alternative input: a bare domain (e.g. example.com)"
                  }
                }
              }
            },
            "additionalProperties": false
          },
          "output": {
            "type": "object",
            "required": [
              "type"
            ],
            "properties": {
              "type": {
                "type": "string"
              },
              "example": {
                "type": "object"
              }
            }
          }
        }
      }
    }
  },
  "x402Version": 2
}

Accepts

The payment requirements as published to the catalog. Read the live 402 before paying — a price here is a claim, not a quote.

0.005USDC≈ $0.005 USD
on Base · exact scheme

Pay 0.005 USDC on Base to 0xF22e…Ff493. The signed payment is good for 5 minutes.

Paid to
0xF22e…Ff493
USD Coin contract
0x8335…02913
Payment window
5 minutes
As published
5000 smallest units
The catalog’s raw entry
[
  {
    "asset": "0x833589fCD6eDb6E08f4c7C32D4f71b54bdA02913",
    "extra": {
      "name": "USD Coin",
      "version": "2"
    },
    "payTo": "0xF22e558a00D91Ee12A1F50C52186FecB8dDFf493",
    "amount": "5000",
    "scheme": "exact",
    "network": "eip155:8453",
    "maxTimeoutSeconds": 300
  }
]

Extensions

{
  "bazaar": {
    "info": {
      "input": {
        "type": "http",
        "method": "GET",
        "queryParams": {
          "domain": "example.com"
        }
      },
      "output": {
        "type": "json",
        "example": {
          "fixes": [],
          "domain": "example.com",
          "disclaimer": "Automated AI-search visibility indicators, not a guarantee.",
          "crawler_score": 100,
          "crawler_matrix": [
            {
              "bot": "PerplexityBot",
              "access": "allowed",
              "listed": false,
              "operator": "Perplexity",
              "citation_relevant": true
            }
          ],
          "robots_present": true,
          "default_blocks_all": false
        }
      }
    },
    "schema": {
      "type": "object",
      "$schema": "https://json-schema.org/draft/2020-12/schema",
      "required": [
        "input"
      ],
      "properties": {
        "input": {
          "type": "object",
          "required": [
            "type",
            "method"
          ],
          "properties": {
            "type": {
              "type": "string",
              "const": "http"
            },
            "method": {
              "enum": [
                "GET",
                "HEAD",
                "DELETE"
              ],
              "type": "string"
            },
            "queryParams": {
              "type": "object",
              "required": [],
              "properties": {
                "url": {
                  "type": "string",
                  "description": "Target URL to audit (e.g. https://example.com)"
                },
                "host": {
                  "type": "string",
                  "description": "Alternative input: a hostname"
                },
                "domain": {
                  "type": "string",
                  "description": "Alternative input: a bare domain (e.g. example.com)"
                }
              }
            }
          },
          "additionalProperties": false
        },
        "output": {
          "type": "object",
          "required": [
            "type"
          ],
          "properties": {
            "type": {
              "type": "string"
            },
            "example": {
              "type": "object"
            }
          }
        }
      }
    }
  }
}

Provenance

Seen in the source catalog
2026-08-23 19:29Z
Last indexed by Roundhouse
2026-09-22 19:10Z
Last enriched (probe, favicon, geo)
2026-09-22 11:15Z
x402 version
2
Max timeout
300s
Liveness probe
HTTP 402
Report

Hand this page to an agent

Copy the prompt and paste it into Claude, an MCP client or your own agent — it will vet this service and call it over the free read API. No key, no account.

GET api.roundhouseai.io/v0/endpoints

This endpoint's own trailing-30-day call count, as published by the upstream catalog and snapshotted daily. 30 snapshots so far. Verified volume counts only settlements with an on-chain EIP-3009 marker.

Open skill.md
Show the prompt
Using Roundhouse, look up the x402 service aeo-crawlers and tell me whether it is
worth paying: what a call costs, whether the endpoint answered when last probed, and what
its payment record actually shows.

curl -s 'https://api.roundhouseai.io/v0/endpoints?q=aeo-crawlers'
curl -s 'https://api.roundhouseai.io/v0/merchants/<the payTo wallet returned above>'

Then call it: read the price from the live 402 at https://api.agentstools.dev/aeo/crawlers, never from
a cached figure, and pay with an x402 client.

The /v0 API needs an API key (`authorization: Bearer rh_live_…`) on everything except
/v0/unified* and /v0/endpoints. Mint a personal key for $0.01 at GET https://api.roundhouseai.io/v0/test/x402,
or use an organization key from https://roundhouseai.io/dashboard/team.

If you do not have Roundhouse tools or skills installed, read
https://roundhouseai.io/skill.md first — it is the whole procedure.