Roundhouse

NetIntel

Fetch and parse a domain's robots.txt file — returns all crawl rules by user-agent, sitemap URLs, crawl delay settings, and checks whether a specific path is allowed or blocked for any bot — so agents can respect crawl policies and locate sitemaps before scraping.

SearchLiveeip155:8453Exactvia cdp
https://netintel.dev/robots-txt/analyze
Robots.txtCrawl rulesSeoWeb intelligenceScraping
Calls · 30d
1→ 0%
This endpoint's own trailing-30-day call count, as published by the upstream catalog and snapshotted daily. 32 snapshots so far.
$169.45
Verified settled volume
4,578 settlements proven x402 by their on-chain EIP-3009 marker.
$0.010
Listed price
As published in the catalog. Always read the live 402 before paying.
1
Calls · 30d
Upstream's own call count for this endpoint, not ours.
1
Unique payers · 30d
Last called 2026-09-19 20:43Z
—
Upstream on-chain volume
Reported by the source catalog.
Paid to
0xdaDc335482AD545296Fd7b28518A251fFCbEb9Df

The wallet the 402 directs payment to. Its whole payment record — every payer, every chain — is on the merchant page.

Asset 0x833589fCD6eDb6E08f4c7C32D4f71b54bdA02913

Provider

The payTo wallet does not resolve to a registered ERC-8004 agent. That is not a verdict on the service — most of the catalog is unregistered.

Live 402 challenge

Captured by the enrichment pass, not read just now. Prices can change — always read the 402 the endpoint answers with.

{
  "error": "Payment required",
  "accepts": [
    {
      "asset": "0x833589fCD6eDb6E08f4c7C32D4f71b54bdA02913",
      "extra": {
        "name": "USD Coin",
        "version": "2"
      },
      "payTo": "0xdaDc335482AD545296Fd7b28518A251fFCbEb9Df",
      "amount": "10000",
      "scheme": "exact",
      "network": "eip155:8453",
      "maxTimeoutSeconds": 300
    },
    {
      "asset": "EPjFWdd5AufqSSqeM2qN1xzybapC8G4wEGGkZwyTDt1v",
      "extra": {
        "feePayer": "GVJJ7rdGiXr5xaYbRwRbjfaJL7fmwRygFi1H6aGqDveb"
      },
      "payTo": "8FVeybWnB4dYrxTuPK7WzFZNAiEcFVaDLFnKpRwVGr5s",
      "amount": "10000",
      "scheme": "exact",
      "network": "solana:5eykt4UsFv8P8NJdTREpY1vzqKqZKvdp",
      "maxTimeoutSeconds": 300
    }
  ],
  "resource": {
    "url": "https://netintel.dev/robots-txt/analyze",
    "tags": [
      "robots.txt",
      "crawl rules",
      "seo",
      "web intelligence",
      "scraping"
    ],
    "iconUrl": "https://netintel.dev/favicon.png",
    "mimeType": "application/json",
    "description": "Fetch and parse a domain's robots.txt file — returns all crawl rules by user-agent, sitemap URLs, crawl delay settings, and checks whether a specific path is allowed or blocked for any bot — so agents can respect crawl policies and locate sitemaps before scraping.",
    "serviceName": "NetIntel"
  },
  "extensions": {
    "bazaar": {
      "info": {
        "input": {
          "type": "http",
          "method": "GET",
          "queryParams": {
            "domain": "example.com"
          }
        },
        "output": {
          "type": "json",
          "example": {
            "found": true,
            "grade": "B",
            "rules": [
              {
                "allow": [
                  "/public/"
                ],
                "disallow": [
                  "/private/",
                  "/admin/"
                ],
                "user_agent": "*",
                "crawl_delay": null
              }
            ],
            "score": 85,
            "domain": "example.com",
            "findings": [],
            "sitemaps": [
              "https://example.com/sitemap.xml"
            ],
            "path_check": {
              "path": "/private/page",
              "allowed": false,
              "rule_type": "disallow",
              "user_agent": "*",
              "matched_rule": "Disallow: /private/"
            },
            "robots_url": "https://example.com/robots.txt",
            "status_code": 200,
            "raw_content_preview": "User-agent: *\nDisallow: /private/\n..."
          }
        }
      },
      "schema": {
        "type": "object",
        "$schema": "https://json-schema.org/draft/2020-12/schema",
        "required": [
          "input"
        ],
        "properties": {
          "input": {
            "type": "object",
            "required": [
              "type",
              "method"
            ],
            "properties": {
              "type": {
                "type": "string",
                "const": "http"
              },
              "method": {
                "enum": [
                  "GET"
                ],
                "type": "string"
              },
              "queryParams": {
                "type": "object",
                "required": [
                  "domain"
                ],
                "properties": {
                  "path": {
                    "type": "string",
                    "description": "URL path to check permission for (e.g. /api/data)"
                  },
                  "domain": {
                    "type": "string",
                    "description": "Domain name to fetch robots.txt from (e.g. example.com)"
                  },
                  "user_agent": {
                    "type": "string",
                    "description": "Bot name to check rules for (default: *)"
                  }
                }
              }
            },
            "additionalProperties": false
          },
          "output": {
            "type": "object",
            "required": [
              "type"
            ],
            "properties": {
              "type": {
                "type": "string"
              },
              "example": {
                "type": "object",
                "properties": {
                  "found": {
                    "type": "boolean"
                  },
                  "grade": {
                    "type": "string"
                  },
                  "rules": {
                    "type": "array",
                    "items": {
                      "type": "object"
                    }
                  },
                  "score": {
                    "type": "number"
                  },
                  "domain": {
                    "type": "string"
                  },
                  "findings": {
                    "type": "array",
                    "items": {
                      "type": "object"
                    }
                  },
                  "sitemaps": {
                    "type": "array",
                    "items": {
                      "type": "string"
                    }
                  },
                  "path_check": {
                    "type": "object"
                  },
                  "robots_url": {
                    "type": "string"
                  },
                  "status_code": {
                    "type": "number"
                  },
                  "raw_content_preview": {
                    "type": "string"
                  }
                }
              }
            }
          }
        }
      }
    }
  },
  "x402Version": 2
}

Accepts

The payment requirements as published to the catalog. Read the live 402 before paying — a price here is a claim, not a quote.

0.01USDC≈ $0.01 USD
on Base · exact scheme

Pay 0.01 USDC on Base to 0xdaDc…Eb9Df. The signed payment is good for 5 minutes.

Paid to
0xdaDc…Eb9Df
USD Coin contract
0x8335…02913
Payment window
5 minutes
As published
10000 smallest units
10000units
on solana:5eykt4UsFv8P8NJdTREpY1vzqKqZKvdp · exact scheme

That figure is in the token’s smallest units. Roundhouse does not hold this contract’s decimals, so it is shown as published rather than converted.

Paid to
8FVeyb…VGr5s
Token contract
EPjFWd…TDt1v
Payment window
5 minutes
The catalog’s raw entry
[
  {
    "asset": "0x833589fCD6eDb6E08f4c7C32D4f71b54bdA02913",
    "extra": {
      "name": "USD Coin",
      "version": "2"
    },
    "payTo": "0xdaDc335482AD545296Fd7b28518A251fFCbEb9Df",
    "amount": "10000",
    "scheme": "exact",
    "network": "eip155:8453",
    "maxTimeoutSeconds": 300
  },
  {
    "asset": "EPjFWdd5AufqSSqeM2qN1xzybapC8G4wEGGkZwyTDt1v",
    "extra": {
      "feePayer": "BFK9TLC3edb13K6v4YyH3DwPb5DSUpkWvb7XnqCL9b4F"
    },
    "payTo": "8FVeybWnB4dYrxTuPK7WzFZNAiEcFVaDLFnKpRwVGr5s",
    "amount": "10000",
    "scheme": "exact",
    "network": "solana:5eykt4UsFv8P8NJdTREpY1vzqKqZKvdp",
    "maxTimeoutSeconds": 300
  }
]

Extensions

{
  "bazaar": {
    "info": {
      "input": {
        "type": "http",
        "method": "GET",
        "queryParams": {
          "domain": "example.com"
        }
      },
      "output": {
        "type": "json",
        "example": {
          "found": true,
          "grade": "B",
          "rules": [
            {
              "allow": [
                "/public/"
              ],
              "disallow": [
                "/private/",
                "/admin/"
              ],
              "user_agent": "*",
              "crawl_delay": null
            }
          ],
          "score": 85,
          "domain": "example.com",
          "findings": [],
          "sitemaps": [
            "https://example.com/sitemap.xml"
          ],
          "path_check": {
            "path": "/private/page",
            "allowed": false,
            "rule_type": "disallow",
            "user_agent": "*",
            "matched_rule": "Disallow: /private/"
          },
          "robots_url": "https://example.com/robots.txt",
          "status_code": 200,
          "raw_content_preview": "User-agent: *\nDisallow: /private/\n..."
        }
      }
    },
    "schema": {
      "type": "object",
      "$schema": "https://json-schema.org/draft/2020-12/schema",
      "required": [
        "input"
      ],
      "properties": {
        "input": {
          "type": "object",
          "required": [
            "type",
            "method"
          ],
          "properties": {
            "type": {
              "type": "string",
              "const": "http"
            },
            "method": {
              "enum": [
                "GET"
              ],
              "type": "string"
            },
            "queryParams": {
              "type": "object",
              "required": [
                "domain"
              ],
              "properties": {
                "path": {
                  "type": "string",
                  "description": "URL path to check permission for (e.g. /api/data)"
                },
                "domain": {
                  "type": "string",
                  "description": "Domain name to fetch robots.txt from (e.g. example.com)"
                },
                "user_agent": {
                  "type": "string",
                  "description": "Bot name to check rules for (default: *)"
                }
              }
            }
          },
          "additionalProperties": false
        },
        "output": {
          "type": "object",
          "required": [
            "type"
          ],
          "properties": {
            "type": {
              "type": "string"
            },
            "example": {
              "type": "object",
              "properties": {
                "found": {
                  "type": "boolean"
                },
                "grade": {
                  "type": "string"
                },
                "rules": {
                  "type": "array",
                  "items": {
                    "type": "object"
                  }
                },
                "score": {
                  "type": "number"
                },
                "domain": {
                  "type": "string"
                },
                "findings": {
                  "type": "array",
                  "items": {
                    "type": "object"
                  }
                },
                "sitemaps": {
                  "type": "array",
                  "items": {
                    "type": "string"
                  }
                },
                "path_check": {
                  "type": "object"
                },
                "robots_url": {
                  "type": "string"
                },
                "status_code": {
                  "type": "number"
                },
                "raw_content_preview": {
                  "type": "string"
                }
              }
            }
          }
        }
      }
    }
  }
}

Provenance

Seen in the source catalog
2026-09-19 20:43Z
Last indexed by Roundhouse
2026-09-24 07:30Z
Last enriched (probe, favicon, geo)
2026-09-14 05:45Z
x402 version
2
Max timeout
300s
Liveness probe
HTTP 402
Report

Hand this page to an agent

Copy the prompt and paste it into Claude, an MCP client or your own agent — it will vet this service and call it over the free read API. No key, no account.

GET api.roundhouseai.io/v0/endpoints

This endpoint's own trailing-30-day call count, as published by the upstream catalog and snapshotted daily. 32 snapshots so far. Verified volume counts only settlements with an on-chain EIP-3009 marker.

Open skill.md
Show the prompt
Using Roundhouse, look up the x402 service NetIntel and tell me whether it is
worth paying: what a call costs, whether the endpoint answered when last probed, and what
its payment record actually shows.

curl -s 'https://api.roundhouseai.io/v0/endpoints?q=NetIntel'
curl -s 'https://api.roundhouseai.io/v0/merchants/<the payTo wallet returned above>'

Then call it: read the price from the live 402 at https://netintel.dev/robots-txt/analyze, never from
a cached figure, and pay with an x402 client.

The /v0 API needs an API key (`authorization: Bearer rh_live_…`) on everything except
/v0/unified* and /v0/endpoints. Mint a personal key for $0.01 at GET https://api.roundhouseai.io/v0/test/x402,
or use an organization key from https://roundhouseai.io/dashboard/team.

If you do not have Roundhouse tools or skills installed, read
https://roundhouseai.io/skill.md first — it is the whole procedure.