Visitor Classification

Every request that reaches the Appear edge is classified in under 50ms using a combination of user-agent analysis, IP reputation data, and behavioral signals.

How classification works

Classification determines which response profile to serve: human, generic crawler, or a named AI system. The decision is made at the edge before any content is fetched, ensuring zero latency impact for human visitors.

Identified AI crawlers

  • OAI-SearchBot / GPTBot — OpenAI crawler roles
  • PerplexityBot — Perplexity AI
  • ClaudeBot / Claude-SearchBot / anthropic-ai — Anthropic / Claude
  • Googlebot and Google-Extended — Google Search and Google AI policy surfaces
  • Bytespider — ByteDance / TikTok AI
  • Other known AI agents — continuously updated

Classification signals

  • User-agent string matching against known crawler signatures
  • IP address cross-referenced against known AI platform ASNs
  • Request pattern analysis (no cookie, no JS, no referrer)
  • Behavioral signals that distinguish automated from human traffic
  • Manual override rules configurable per domain in dashboard

Human traffic handling

Requests classified as human are proxied directly to your origin server with no modification. A human visitor sees exactly what they would see if Appear did not exist. There is no caching, no content modification, and no latency added beyond standard DNS resolution.

Search engine crawlers

Traditional search engine crawlers (Googlebot, Bingbot) are identified separately from AI crawlers. Googlebot renders JavaScript and sees your real site as-is. It is not served pre-rendered content. Your traditional search rankings are determined entirely by what Googlebot sees, which is unchanged by Appear.