Who Reads a Tariff Calculator? Our Access Logs Say Two-Thirds Are AI Crawlers
Computed from six days of ExportSonar production access logs (80,603 raw lines, 2026-10-09T19:15Z through 2026-10-15T03:21Z UTC, pulled 2026-10-15) classified with our openly documented traffic-audit pipeline; every figure on this page is recomputed from data/calculator_demand_map.json at build time.
Every operator of a web tool publishes screenshots of happy users. Almost nobody publishes the boring, decisive statistic: who actually loads your pages. We run a free US tariff calculator, and our raw nginx access logs let us answer the question precisely. The result surprised us even though we watch the site daily: of 7,090 calculator requests in the six-day window, 4,728 — 66.7% came from declared AI and LLM agents, led by GPTBot (2,441) and ClaudeBot (2,228). Browser-like human traffic was 1,887 requests, or 26.6%. The machine already reads this tool more than its intended audience does — which is exactly why we treat the calculator as an AI-citation landing page, and why honest log hygiene matters more, not less.
This article is the third publication built from our own telemetry, after the
Section 301 line-exposure map and the ruling-reversal census. It differs from
anything a keyword tool can offer: Google Trends cannot segment by user agent,
referrer structure, or ?hs= query parameter. Raw server logs can.
The dataset: six days, six files, one classifier
The corpus is the full production access-log family pulled from the server on October 15: six files, 80,603 raw lines, of which 80,488 parse under the combined-log format with the site's custom tail fields, collapsing to 78,128 unique request events after global deduplication. The window runs from 2026-10-09T19:15Z to 2026-10-15T03:21Z UTC — about 5.3 days, so daily averages quoted below are conservative.
We deliberately reused the classification pipeline from our earlier traffic-baseline audit rather than inventing a new one: each request is assigned to a layer by explicit user-agent tokens (declared AI agents such as GPTBot, ClaudeBot, OAI-SearchBot, ChatGPT-User and PerplexityBot; search crawlers such as Googlebot and Bingbot; SEO/monitoring tools; our own scanner; and a residual browser-like bucket that we call humanoid because some disguised automation inevitably survives there). That last caveat cuts both ways, and we flag it again later where it changes the reading.
Three layers, one census
| Layer | Calculator requests | Share | Who this is |
|---|---|---|---|
| Declared AI / LLM agents | 4,728 | 66.7% | GPTBot 2,441 · ClaudeBot 2,228 · OAI-SearchBot 59 |
| Browser-like (humanoid) | 1,887 | 26.6% | Chrome/Firefox/Safari families; may include some disguised automation |
| Googlebot / GoogleOther | 388 | 5.5% | Classic search indexing |
| Everything else | 87 | 1.2% | SEO tools, our own scanner, misc agents |
Census basis: 7,090 HTTP 200 GET requests to /calculator.html after dedupe; shares rounded to one decimal.
Two observations follow. First, the calculator is a genuine AI-citation surface: when ChatGPT or Claude answers "what is the duty rate on X from China", the agents behind those answers fetch the calculator about 875 times per day on average, roughly two and a half times the browser-like rate of 354. Second, the humanoid layer is an over-count of humans, not an under-count, because botnets increasingly ship Mozilla-compatible user agents; the true human share sits at or below 26.6%.
What 6,004 deep-linked lookups reveal
The calculator accepts pre-filled queries of the form
/calculator.html?hs=<4-digit>&origin=<country>. In the
window these deep links fired 6,004 times, covering
1,228 distinct four-digit headings, with
origin=CN on 5,876 of them (97.9%). Referrer headers split cleanly:
3,810 arrived with no referrer at all (bookmarks, API-style fetching,
chat clients stripping referrers) and 2,194 were referred from our own
/hs/ chapter pages — proof that the chapter-browse funnel
feeds the calculator without any paid promotion.
| HS heading | Example commodity (verbatim from our HTS extract) | Deep-link hits | Rated lines | Median general rate | Highest rate |
|---|---|---|---|---|---|
| 0106 | Primates | 10 | 21 | 0.0% | 4.8% |
| 0303 | Sockeye salmon (red salmon) (Oncorhynchus nerka) | 10 | 86 | 0.0% | 15.0% |
| 0304 | Tilapias (Oreochromis spp.) | 10 | 187 | 0.0% | 6.0% |
| 0306 | Rock lobster and other sea crawfish | 10 | 78 | 0.0% | 7.5% |
| 0602 | Unrooted cuttings and slips | 10 | 16 | 1.9% | 4.8% |
| 0703 | Onion sets | 10 | 3 | 20.0% | 20.0% |
Rate profile joined to our full HTS extract (23,193 rows, USITC REST vintage 2026-10-09); ties in the hits column broken by HS code ascending.
The headline finding is what the table does not show: a steep demand curve. It is almost perfectly flat, and the next section explains why that flatness is itself the story.
The demand-map caveat: enumeration is not demand
Our working title for this piece was bolder — something like "the 30 headings buyers actually calculate". The data refused to support it. Three IP addresses account for 66.7% of all deep links (4,004 of 6,004), walking headings in long consecutive runs — classic enumeration behavior. Exclude those three and the picture flattens completely: 2,000 deep links spread over 1,228 headings, a median of one hit per heading, maximum seven.
That correction is the most useful thing in this article. Any competitor can buy a keyword list; nobody should confuse a crawler's crawl budget with a buyer's shopping list. The honest conclusions are structural, not ordinal: nearly every four-digit heading gets probed eventually (all 1,228 four-digit headings in the schedule), origin=CN dominates beyond its already dominant trade share, and referral flow concentrates on chapter pages rather than search.
Run this analysis on your own logs
The method transfers to any site with server-log access:
- Pull the complete rotated log family, not just the live file — five of our six corpus files were compressed rotations.
- Parse with a single regex validated against fixtures, including the empty-request 400-line edge case, or your counters will silently disagree.
- Deduplicate globally before counting; overlapping rotation windows otherwise double-count boundary hours.
- Classify user agents into named layers with token lists you publish, and keep a dedicated bucket for your own monitors — ours would have polluted the humanoid layer with 6,462 requests.
- Recompute every ranking twice: once raw, once with high-volume IPs excluded. If a ranking survives both cuts it describes demand; if it collapses, it described enumeration.
- Only then write prose — and let a script assert every number in that prose against the dataset, the way this page is generated.
What this means for importers
If you research tariffs the way most buyers do — asking an assistant, or browsing chapter pages until something clicks — the calculator experience you get is increasingly shaped for machines first. That is not a downgrade: it means the numbers you are shown must be reproducible from public data, which is why every rate here joins back to the official schedule and why we log data provenance per row. Start from the chapter index, drill into a heading, then open the tariff calculator with the heading pre-filled; the trade-remedy chapter map explains which additional layers stack on top of the MFN rate, and non-standard rate text decodes the specific/compound strings the calculator parses. For the monitoring layer — knowing when your headings' rates change — see plans and pricing, and create an account at signup to receive alerts.
Frequently asked questions
- What share of traffic to a tariff calculator comes from AI crawlers?
- In our six-day production window, 4,728 of 7,090 calculator requests (66.7%) came from declared AI and LLM agents: GPTBot 2,441, ClaudeBot 2,228 and OAI-SearchBot 59. Human-browser requests were 1,887, or 26.6%.
- Which HS headings do deep-linked calculator visits target?
- Deep links carried 6,004 four-digit HS lookups covering 1,228 distinct headings, 97.9% of them with origin=CN. After excluding three enumerating IP addresses responsible for 66.7% of deep links, the remainder spread thinly: 2,000 lookups over 1,228 headings, median one hit each, maximum seven.
- How can you tell real buyer demand apart from crawler enumeration in access logs?
- Classify user agents into layers first (declared AI bots, search crawlers, known scanners, your own monitors, browser-like residuals), deduplicate repeated lines, then recompute the ranking after excluding high-volume enumerating IPs. A heading ranked top purely on raw volume can collapse from thousands of hits to a handful once three IP addresses are removed.
- Why does origin=CN dominate calculator deep links?
- 5,876 of 6,004 deep-linked calculator requests (97.9%) carried origin=CN, mirroring China's share of US import lines and the fact that Section 301 additional duties stack country-specific percentages on top of the MFN rate, which is what the calculator computes.
Sources: USITC Harmonized Tariff Schedule · nginx HTTP server log-module documentation