How do I know whether a request claiming to be GPTBot is real?
The user agent string is free text and anyone can send it. OpenAI publishes the IP address ranges its crawlers use, as do Perplexity, Microsoft, Google and Apple. A request is real when its address falls in the published range for that crawler; otherwise it is an impostor, whatever the string says. Agent Tracking does this check for every log line and lists impostors as unverified instead of counting them.
By Gorden Wübbe · Updated 2026-09-08
Why the string is not enough
Scrapers copy the GPTBot user agent because sites tend to allow it. A robots.txt rule or a rate limit keyed on the string therefore treats the copy the same as the original. The only reliable signal is the network address, which the vendors publish precisely so that sites can tell.
Where the ranges come from
OpenAI publishes separate JSON lists for GPTBot, ChatGPT-User and OAI-SearchBot. Perplexity, Microsoft, Google and Apple publish theirs. The lists change; Agent Tracking refreshes them nightly and checks each log line against the list for the crawler it claims.
Doing it by hand
Take the address from the log line, fetch the vendor's list, and test whether the address is inside any of the CIDR blocks. For one line that is a minute; for a night's log it is a script. The log import does it for every line and shows the result per agent as verified and unverified counts.
curl -sS -X POST https://agenttracking.co/api/logs/example.com \ -H "Authorization: Bearer wmt_your_token" \ -H "Content-Type: text/plain" --data-binary @/var/log/nginx/access.log
What to do with an impostor
Nothing automatic; the tool never blocks. But an agent whose unverified count is a large share of its total is worth a rule at the edge or in robots.txt, and the verified share tells you how much real crawler traffic you would keep.
In short
Do all crawlers publish ranges?
No. Where a vendor publishes none, the row is marked as not verifiable, and the fetches are counted from the string alone with that caveat.
Is reverse DNS an alternative?
For Googlebot and Bingbot, yes. OpenAI and Perplexity rely on published ranges instead, which is why the check uses ranges throughout.