WebToolMage

Parsing is local — no live site fetch claimed.

WebToolMage

URL inspector

Break a URL into scheme, host, path, query and hash.

Does not claim remote crawl status unless a live fetch is performed (not in these pilots).

Result

{
  "href": "https://example.com/path?q=1#frag",
  "protocol": "https:",
  "host": "example.com",
  "pathname": "/path",
  "search": "?q=1",
  "hash": "#frag"
}

Method

inspectUrl uses the URL constructor. robots validator checks User-agent/Disallow/Allow/Sitemap lines.

Assumptions

  • URL API parsing.
  • Robots line rules.
  • No paid network API.

Questions

Does this fetch my site?

Not in these pilots. Paste content instead.

Is robots crawl advice legal advice?

No. It is syntax validation only.

Robots.txt tester·US English version

URL inspector · WebToolMage