Roundhouse

CrawlWitness

Use when a crawler, scraper or RAG pipeline must decide whether it may use a website for AI: yes/no verdict for a domain, AI bot and use (train, rag, search) from robots.txt, IETF AIPREF, TDMRep, noai and RSL licenses, with a signed receipt. Errors: 400 invalid domain or unknown bot (not charged).

SearchLiveeip155:8453Exactvia facilitator:payAI
https://api.crawlwitness.com/v1/can-use
Ai-consentRobots-txtCrawler-complianceData-licensingBatch
Calls · 30d
13
This endpoint's own trailing-30-day call count, as published by the upstream catalog and snapshotted daily. 17 snapshots so far.
$0.14
Verified settled volume
32 settlements proven x402 by their on-chain EIP-3009 marker.
$0.0010
Listed price
As published in the catalog. Always read the live 402 before paying.
—
Calls · 30d
Upstream's own call count for this endpoint, not ours.
—
Unique payers · 30d
Never called
—
Upstream on-chain volume
Reported by the source catalog.
Paid to
0xa20F0b8ADf20Ed17a8d27B369B3058afdB71a5C5

The wallet the 402 directs payment to. Its whole payment record — every payer, every chain — is on the merchant page.

Asset 0x833589fCD6eDb6E08f4c7C32D4f71b54bdA02913

Provider

The payTo wallet does not resolve to a registered ERC-8004 agent. That is not a verdict on the service — most of the catalog is unregistered.

Live 402 challenge

Captured by the enrichment pass, not read just now. Prices can change — always read the 402 the endpoint answers with.

{
  "accepts": [
    {
      "asset": "0x833589fCD6eDb6E08f4c7C32D4f71b54bdA02913",
      "extra": {
        "name": "USD Coin",
        "version": "2"
      },
      "payTo": "0xa20F0b8ADf20Ed17a8d27B369B3058afdB71a5C5",
      "amount": "1000",
      "scheme": "exact",
      "network": "eip155:8453",
      "maxTimeoutSeconds": 300
    },
    {
      "asset": "EPjFWdd5AufqSSqeM2qN1xzybapC8G4wEGGkZwyTDt1v",
      "extra": {
        "feePayer": "79Bz5UMBA9ZSpa2XDDpkTKDqto5ScDgN5o8pRVRJVBLP",
        "transactionVersions": [
          0
        ]
      },
      "payTo": "DEuats4W5mXoX1dZEb693q4Y4GT6mnZiwPjTN2GYFFrd",
      "amount": "2000",
      "scheme": "exact",
      "network": "solana:5eykt4UsFv8P8NJdTREpY1vzqKqZKvdp",
      "maxTimeoutSeconds": 300
    }
  ],
  "detalle": "pass ?domain=<domain>&bot=<AI bot>&use=train|rag|search",
  "resource": {
    "url": "https://api.crawlwitness.com/v1/can-use",
    "tags": [
      "ai-consent",
      "robots-txt",
      "crawler-compliance",
      "data-licensing",
      "batch"
    ],
    "iconUrl": "https://api.crawlwitness.com/icono.png",
    "mimeType": "application/json",
    "description": "Use when a crawler, scraper or RAG pipeline must decide whether it may use a website for AI: yes/no verdict for a domain, AI bot and use (train, rag, search) from robots.txt, IETF AIPREF, TDMRep, noai and RSL licenses, with a signed receipt. Errors: 400 invalid domain or unknown bot (not charged).",
    "serviceName": "CrawlWitness"
  },
  "extensions": {
    "bazaar": {
      "info": {
        "input": {
          "type": "http",
          "method": "GET",
          "queryParams": {
            "bot": "GPTBot",
            "use": "train",
            "domain": "nytimes.com"
          }
        },
        "output": {
          "type": "json",
          "example": {
            "bot": "GPTBot",
            "use": "train",
            "pago": {
              "recibo": {
                "firma": "0x…"
              },
              "transaccion": "0x…"
            },
            "domain": "nytimes.com",
            "method": "can-use/1",
            "allowed": false,
            "reasons": [
              "robots.txt disallows GPTBot at the site root"
            ],
            "bot_role": "training",
            "decision": "not_allowed",
            "operator": "OpenAI"
          }
        }
      },
      "schema": {
        "type": "object",
        "$schema": "https://json-schema.org/draft/2020-12/schema",
        "required": [
          "input"
        ],
        "properties": {
          "input": {
            "type": "object",
            "required": [
              "type",
              "method"
            ],
            "properties": {
              "type": {
                "type": "string",
                "const": "http"
              },
              "method": {
                "enum": [
                  "GET"
                ],
                "type": "string"
              },
              "queryParams": {
                "type": "object",
                "required": [
                  "domain",
                  "bot"
                ],
                "properties": {
                  "bot": {
                    "enum": [
                      "GPTBot",
                      "ClaudeBot",
                      "anthropic-ai",
                      "Google-Extended",
                      "GoogleOther",
                      "CloudVertexBot",
                      "Applebot-Extended",
                      "meta-externalagent",
                      "Amazonbot",
                      "amazon-kendra-",
                      "Bytespider",
                      "CCBot",
                      "cohere-training-data-crawler",
                      "AI2Bot",
                      "Diffbot",
                      "Timpibot",
                      "ImagesiftBot",
                      "omgili",
                      "PetalBot",
                      "KimiBot",
                      "ICC-Crawler",
                      "Cotoyogi",
                      "Reflectionbot",
                      "CloudflareBrowserRenderingCrawler",
                      "LinerBot",
                      "FishBot",
                      "Novellum",
                      "bigsur.ai",
                      "NavuBot",
                      "QualifiedBot",
                      "atlassian-bot",
                      "make.com",
                      "magpie-crawler",
                      "Awario",
                      "w4mwnpbXf3MFAbxOkJRw",
                      "SemrushBot-OCOB",
                      "SemrushBot-SWA",
                      "BorderxBot",
                      "SelectikaScraper",
                      "CitibotSiteCrawler",
                      "AdpResearchBot",
                      "WARDBot",
                      "ygs-scraper-bot",
                      "OAI-SearchBot",
                      "Claude-SearchBot",
                      "PerplexityBot",
                      "Applebot",
                      "Amzn-SearchBot",
                      "Bravebot",
                      "Cloudflare-AI-Search",
                      "Cloudflare-AI-Search-External",
                      "DuckAssistBot",
                      "YouBot",
                      "KernelSearchBot",
                      "ShapBot",
                      "alphalens-bot",
                      "Anomura",
                      "Element451Bot",
                      "ChatGPT-User",
                      "Claude-User",
                      "Perplexity-User",
                      "meta-externalfetcher",
                      "meta-externaltest",
                      "muse-",
                      "MistralAI-User",
                      "cohere-ai",
                      "Google-Agent",
                      "GrokAgent",
                      "Devin",
                      "ApifyWebsiteContentCrawler",
                      "Instapaper",
                      "Retool",
                      "QATechBot",
                      "TwinAgent",
                      "CledaraBot",
                      "HarkBot",
                      "HIFIBot",
                      "ChathiveCrawler",
                      "EasyScan",
                      "Nava",
                      "ZipchatBot"
                    ],
                    "type": "string",
                    "description": "AI user-agent token, e.g. GPTBot, ClaudeBot, Google-Extended, CCBot"
                  },
                  "use": {
                    "enum": [
                      "train",
                      "rag",
                      "search"
                    ],
                    "type": "string",
                    "description": "Intended use (default train)"
                  },
                  "domain": {
                    "type": "string",
                    "maxLength": 253,
                    "minLength": 3,
                    "description": "Domain, e.g. nytimes.com"
                  }
                }
              }
            },
            "additionalProperties": false
          },
          "output": {
            "type": "object",
            "required": [
              "type"
            ],
            "properties": {
              "type": {
                "type": "string"
              },
              "example": {
                "type": "object"
              }
            }
          }
        }
      }
    },
    "offer-receipt": {
      "info": {
        "offers": [
          {
            "format": "eip712",
            "payload": {
              "asset": "0x833589fCD6eDb6E08f4c7C32D4f71b54bdA02913",
              "payTo": "0xa20F0b8ADf20Ed17a8d27B369B3058afdB71a5C5",
              "amount": "1000",
              "scheme": "exact",
              "network": "eip155:8453",
              "version": 1,
              "validUntil": 1793681484,
              "resourceUrl": "https://api.crawlwitness.com/v1/can-use"
            },
            "signature": "0x3391cd5bd890bf4cc4bcba546a081bd03dc1b31e0765d3d2cee94b409b148871744ab1dfd07adcb33443b54553140fe1ef5d7f371b36bd576a79189d57db82661c",
            "acceptIndex": 0
          },
          {
            "format": "eip712",
            "payload": {
              "asset": "EPjFWdd5AufqSSqeM2qN1xzybapC8G4wEGGkZwyTDt1v",
              "payTo": "DEuats4W5mXoX1dZEb693q4Y4GT6mnZiwPjTN2GYFFrd",
              "amount": "2000",
              "scheme": "exact",
              "network": "solana:5eykt4UsFv8P8NJdTREpY1vzqKqZKvdp",
              "version": 1,
              "validUntil": 1793681484,
              "resourceUrl": "https://api.crawlwitness.com/v1/can-use"
            },
            "signature": "0x53a5d1b333a39cefaea740bf9beb2f45cad9489c5e90a94545dab2c0d5b583cd439c1fac40bcf1f5477a8af12e292600bff94a5f22a4ced5688e08e168ec01b21b",
            "acceptIndex": 1
          }
        ]
      },
      "schema": {
        "type": "object",
        "$schema": "https://json-schema.org/draft/2020-12/schema",
        "required": [
          "offers"
        ],
        "properties": {
          "offers": {
            "type": "array",
            "items": {
              "type": "object",
              "required": [
                "format",
                "signature"
              ],
              "properties": {
                "format": {
                  "type": "string"
                },
                "payload": {
                  "type": "object",
                  "required": [
                    "version",
                    "resourceUrl",
                    "scheme",
                    "network",
                    "asset",
                    "payTo",
                    "amount"
                  ],
                  "properties": {
                    "asset": {
                      "type": "string"
                    },
                    "payTo": {
                      "type": "string"
                    },
                    "amount": {
                      "type": "string"
                    },
                    "scheme": {
                      "type": "string"
                    },
                    "network": {
                      "type": "string"
                    },
                    "version": {
                      "type": "integer"
                    },
                    "validUntil": {
                      "type": "integer"
                    },
                    "resourceUrl": {
                      "type": "string"
                    }
                  }
                },
                "signature": {
                  "type": "string"
                },
                "acceptIndex": {
                  "type": "integer"
                }
              }
            }
          }
        }
      }
    },
    "payment-identifier": {
      "info": {
        "required": false
      },
      "schema": {
        "type": "object",
        "$schema": "https://json-schema.org/draft/2020-12/schema",
        "required": [
          "required"
        ],
        "properties": {
          "id": {
            "type": "string",
            "pattern": "^[a-zA-Z0-9_-]+$",
            "maxLength": 128,
            "minLength": 16
          },
          "required": {
            "type": "boolean"
          }
        }
      }
    }
  },
  "x402Version": 2
}

Accepts

The payment requirements as published to the catalog. Read the live 402 before paying — a price here is a claim, not a quote.

0.001USDC≈ $0.001 USD
on Base · exact scheme

Pay 0.001 USDC on Base to 0xa20F…1a5C5. The signed payment is good for 5 minutes.

Paid to
0xa20F…1a5C5
USD Coin contract
0x8335…02913
Payment window
5 minutes
As published
1000 smallest units
2000units
on solana:5eykt4UsFv8P8NJdTREpY1vzqKqZKvdp · exact scheme

That figure is in the token’s smallest units. Roundhouse does not hold this contract’s decimals, so it is shown as published rather than converted.

Paid to
DEuats…YFFrd
Token contract
EPjFWd…TDt1v
Payment window
5 minutes
The catalog’s raw entry
[
  {
    "asset": "0x833589fCD6eDb6E08f4c7C32D4f71b54bdA02913",
    "extra": {
      "name": "USD Coin",
      "version": "2"
    },
    "payTo": "0xa20F0b8ADf20Ed17a8d27B369B3058afdB71a5C5",
    "amount": "1000",
    "scheme": "exact",
    "network": "eip155:8453",
    "maxTimeoutSeconds": 300
  },
  {
    "asset": "EPjFWdd5AufqSSqeM2qN1xzybapC8G4wEGGkZwyTDt1v",
    "extra": {
      "feePayer": "79Bz5UMBA9ZSpa2XDDpkTKDqto5ScDgN5o8pRVRJVBLP",
      "transactionVersions": [
        0
      ]
    },
    "payTo": "DEuats4W5mXoX1dZEb693q4Y4GT6mnZiwPjTN2GYFFrd",
    "amount": "2000",
    "scheme": "exact",
    "network": "solana:5eykt4UsFv8P8NJdTREpY1vzqKqZKvdp",
    "maxTimeoutSeconds": 300
  }
]

Extensions

{
  "bazaar": {
    "info": {
      "input": {
        "type": "http",
        "method": "GET",
        "queryParams": {
          "bot": "GPTBot",
          "use": "train",
          "domain": "nytimes.com"
        }
      },
      "output": {
        "type": "json",
        "example": {
          "bot": "GPTBot",
          "use": "train",
          "pago": {
            "recibo": {
              "firma": "0x…"
            },
            "transaccion": "0x…"
          },
          "domain": "nytimes.com",
          "method": "can-use/1",
          "allowed": false,
          "reasons": [
            "robots.txt disallows GPTBot at the site root"
          ],
          "bot_role": "training",
          "decision": "not_allowed",
          "operator": "OpenAI"
        }
      }
    },
    "schema": {
      "type": "object",
      "$schema": "https://json-schema.org/draft/2020-12/schema",
      "required": [
        "input"
      ],
      "properties": {
        "input": {
          "type": "object",
          "required": [
            "type",
            "method"
          ],
          "properties": {
            "type": {
              "type": "string",
              "const": "http"
            },
            "method": {
              "enum": [
                "GET"
              ],
              "type": "string"
            },
            "queryParams": {
              "type": "object",
              "required": [
                "domain",
                "bot"
              ],
              "properties": {
                "bot": {
                  "enum": [
                    "GPTBot",
                    "ClaudeBot",
                    "anthropic-ai",
                    "Google-Extended",
                    "GoogleOther",
                    "CloudVertexBot",
                    "Applebot-Extended",
                    "meta-externalagent",
                    "Amazonbot",
                    "amazon-kendra-",
                    "Bytespider",
                    "CCBot",
                    "cohere-training-data-crawler",
                    "AI2Bot",
                    "Diffbot",
                    "Timpibot",
                    "ImagesiftBot",
                    "omgili",
                    "PetalBot",
                    "KimiBot",
                    "ICC-Crawler",
                    "Cotoyogi",
                    "Reflectionbot",
                    "CloudflareBrowserRenderingCrawler",
                    "LinerBot",
                    "FishBot",
                    "Novellum",
                    "bigsur.ai",
                    "NavuBot",
                    "QualifiedBot",
                    "atlassian-bot",
                    "make.com",
                    "magpie-crawler",
                    "Awario",
                    "w4mwnpbXf3MFAbxOkJRw",
                    "SemrushBot-OCOB",
                    "SemrushBot-SWA",
                    "BorderxBot",
                    "SelectikaScraper",
                    "CitibotSiteCrawler",
                    "AdpResearchBot",
                    "WARDBot",
                    "ygs-scraper-bot",
                    "OAI-SearchBot",
                    "Claude-SearchBot",
                    "PerplexityBot",
                    "Applebot",
                    "Amzn-SearchBot",
                    "Bravebot",
                    "Cloudflare-AI-Search",
                    "Cloudflare-AI-Search-External",
                    "DuckAssistBot",
                    "YouBot",
                    "KernelSearchBot",
                    "ShapBot",
                    "alphalens-bot",
                    "Anomura",
                    "Element451Bot",
                    "ChatGPT-User",
                    "Claude-User",
                    "Perplexity-User",
                    "meta-externalfetcher",
                    "meta-externaltest",
                    "muse-",
                    "MistralAI-User",
                    "cohere-ai",
                    "Google-Agent",
                    "GrokAgent",
                    "Devin",
                    "ApifyWebsiteContentCrawler",
                    "Instapaper",
                    "Retool",
                    "QATechBot",
                    "TwinAgent",
                    "CledaraBot",
                    "HarkBot",
                    "HIFIBot",
                    "ChathiveCrawler",
                    "EasyScan",
                    "Nava",
                    "ZipchatBot"
                  ],
                  "type": "string",
                  "description": "AI user-agent token, e.g. GPTBot, ClaudeBot, Google-Extended, CCBot"
                },
                "use": {
                  "enum": [
                    "train",
                    "rag",
                    "search"
                  ],
                  "type": "string",
                  "description": "Intended use (default train)"
                },
                "domain": {
                  "type": "string",
                  "maxLength": 253,
                  "minLength": 3,
                  "description": "Domain, e.g. nytimes.com"
                }
              }
            }
          },
          "additionalProperties": false
        },
        "output": {
          "type": "object",
          "required": [
            "type"
          ],
          "properties": {
            "type": {
              "type": "string"
            },
            "example": {
              "type": "object"
            }
          }
        }
      }
    }
  },
  "offer-receipt": {
    "schema": {
      "type": "object",
      "$schema": "https://json-schema.org/draft/2020-12/schema",
      "required": [
        "offers"
      ],
      "properties": {
        "offers": {
          "type": "array",
          "items": {
            "type": "object",
            "required": [
              "format",
              "signature"
            ],
            "properties": {
              "format": {
                "type": "string"
              },
              "payload": {
                "type": "object",
                "required": [
                  "version",
                  "resourceUrl",
                  "scheme",
                  "network",
                  "asset",
                  "payTo",
                  "amount"
                ],
                "properties": {
                  "asset": {
                    "type": "string"
                  },
                  "payTo": {
                    "type": "string"
                  },
                  "amount": {
                    "type": "string"
                  },
                  "scheme": {
                    "type": "string"
                  },
                  "network": {
                    "type": "string"
                  },
                  "version": {
                    "type": "integer"
                  },
                  "validUntil": {
                    "type": "integer"
                  },
                  "resourceUrl": {
                    "type": "string"
                  }
                }
              },
              "signature": {
                "type": "string"
              },
              "acceptIndex": {
                "type": "integer"
              }
            }
          }
        }
      }
    }
  },
  "payment-identifier": {
    "schema": {
      "type": "object",
      "$schema": "https://json-schema.org/draft/2020-12/schema",
      "required": [
        "required"
      ],
      "properties": {
        "id": {
          "type": "string",
          "pattern": "^[a-zA-Z0-9_-]+$",
          "maxLength": 128,
          "minLength": 16
        },
        "required": {
          "type": "boolean"
        }
      }
    }
  }
}

Provenance

Seen in the source catalog
2026-11-05 15:41Z
Last indexed by Roundhouse
2026-11-08 02:45Z
Last enriched (probe, favicon, geo)
2026-11-03 04:46Z
x402 version
2
Max timeout
300s
Liveness probe
HTTP 402
Report

Hand this page to an agent

Copy the prompt and paste it into Claude, an MCP client or your own agent — it will vet this service and call it over the free read API. No key, no account.

GET api.roundhouseai.io/v0/endpoints

This endpoint's own trailing-30-day call count, as published by the upstream catalog and snapshotted daily. 17 snapshots so far. Verified volume counts only settlements with an on-chain EIP-3009 marker.

Open skill.md
Show the prompt
Using Roundhouse, look up the x402 service CrawlWitness and tell me whether it is
worth paying: what a call costs, whether the endpoint answered when last probed, and what
its payment record actually shows.

curl -s 'https://api.roundhouseai.io/v0/endpoints?q=CrawlWitness'
curl -s 'https://api.roundhouseai.io/v0/merchants/<the payTo wallet returned above>'

Then call it: read the price from the live 402 at https://api.crawlwitness.com/v1/can-use, never from
a cached figure, and pay with an x402 client.

The /v0 API needs an API key (`authorization: Bearer rh_live_…`) on everything except
/v0/unified* and /v0/endpoints. Either pay over x402 (mint a personal key for $0.01 at
GET https://api.roundhouseai.io/v0/test/x402), or ask me for an organization key, which I create at
https://roundhouseai.io/dashboard/connections. If you have neither, ask before going further.

If you do not have Roundhouse tools or skills installed, read
https://roundhouseai.io/skill.md first — it is the whole procedure.