Page rendering
Can AI read your vehicle pages?
Your bot protection is selective, and this scan can prove it: your pages are in the web search index, so the crawlers that build the indexes AI search answers from are getting through, and a real browser loads your site without ever seeing a challenge, while our plain automated request was refused. Engines that answer from those indexes can still retrieve and cite your pages, so this wall is not making you invisible to AI search. What our refused request does show is that automated readers outside your allowlist are turned away at the door — and which AI fetchers are on that allowlist is a setting in your bot product, not a guarantee. Allowlisting the AI crawlers explicitly (in Cloudflare this is one rule) removes the doubt without weakening protection against real threats, and a VIN Index listing closes the gap from the other side: an inventory surface assistants can always read, with your site's security untouched.
VIN Index republishes every car as a page a crawler reads start to finish, without running a line of JavaScript.
See the detail
What we checked
Whether your vehicles are present in the raw HTML an AI crawler receives, and whether the named AI crawlers are allowed in.
How we tested
We fetch your page with no JavaScript (the way ChatGPT’s and Claude’s crawlers read), count the readable vehicles, and probe whether each engine’s crawler gets through.
What it means
If your cars only appear after JavaScript runs, most AI crawlers see an empty page and have nothing to cite when a buyer asks.
- Your pages in the web search index
- Found
- Real browser render
- Got through, no challenge shown
- Automated crawler request
- Challenged / blocked
- Bytes returned to a no-JS crawler fetch
- 17 chars of HTML
- Vehicles readable in that HTML
- None
