Page rendering
Can AI read your vehicle pages?
Your site refuses OAI-SearchBot, ChatGPT-User, Claude-User, PerplexityBot at the network layer, even though your robots.txt invites them in — so engines that depend on those crawlers (ChatGPT through Bing, and Perplexity) can't read your pages directly and fall back to third-party listings like AutoTrader. But you still allow Googlebot, bingbot, so engines that read those indexes — Google's AI Overviews and Gemini, and Claude through Brave — can find and cite your own pages. The fix is to allowlist the blocked crawler user-agents (almost always a Cloudflare/WAF bot rule) so every engine reaches you, not just some.
VIN Index republishes every car as static, server-rendered HTML, so a crawler reads your whole lot without running a line of JavaScript.
See the detail
What we checked
Whether your vehicles are present in the raw HTML an AI crawler receives, and whether the named AI crawlers are allowed in.
How we tested
We fetch your page with no JavaScript (the way GPTBot and ClaudeBot read), count the readable vehicles, and probe whether specific AI crawler user-agents get through.
What it means
If your cars only appear after JavaScript runs, most AI crawlers see an empty page and have nothing to cite when a buyer asks.
- PerplexityBot request
- Refused at the network layer
- Claude-User request
- Refused at the network layer
- ChatGPT-User request
- Refused at the network layer
- OAI-SearchBot request
- Refused at the network layer
- Bytes returned to GPTBot
- 778,753 chars of HTML
- Vehicles readable in that HTML
- Yes
