Heuristics First, AI Second

When you have a classification problem, the natural instinct is to reach for a language model. Feed it the input, describe the categories, get back a label. The results are usually good. The approach feels modern. And for the first hundred items, the cost is negligible.

Then you start processing thousands of items per day, and the math changes.

The cost of sending everything to the model

Language models charge per token. A typical email might be 500-2,000 tokens. Classify it with an API call and you are spending somewhere around $0.001-0.003 per message. That sounds trivial until you multiply it by the volume of a real inbox.

A moderately active business email account receives 50-150 messages per day. At the low end, that is 1,500 classifications per month. The cost stays manageable, but the latency does not, each classification requires a round trip to an API or a local model inference.

More fundamentally, most of those API calls are wasted. The email from [email protected] is a notification. Every time. The email with "Your order has shipped" in the subject is transactional. Every time. Sending these to a language model is like hiring an expert to answer questions you already know the answers to.

Rules handle the obvious cases

We built an email classification system with a fast, rule-based heuristic classifier that checks signals in order:

• Sender domain, a maintained list of known domains mapped directly to categories
• Sender patterns, addresses starting with noreply@, billing@, support@ map to notification or transactional
• Subject patterns, phrases like "receipt," "order confirmation," "password reset" match known templates
• Body signals, unsubscribe links, tracking pixels, "view in browser" links, and the absence of reply quotes help distinguish newsletters from conversation
• Calendar attachments, .ics files map directly to the calendar category

Each check contributes to a confidence score across six categories: conversation, newsletter, notification, transactional, marketing, and calendar. When the combined signals produce a high-confidence result, the classification is final. No model needed.

This handles roughly 80% of incoming email. The heuristic runs in microseconds. The result is deterministic. And the cost is zero.

AI handles the ambiguous remainder

The remaining 20% are the messages where signals conflict or are absent. A long email from an unfamiliar sender with one unsubscribe link, is it a newsletter or a genuine conversation from someone whose email client appends an unsubscribe footer? A notification from a service the heuristic list has never seen?

These are the cases where a language model earns its cost. The uncertain messages get queued and processed in background batches. The model sees the full message content and metadata, the heuristic's tentative classification, and a description of the categories. It returns its own classification, which either confirms or overrides the heuristic's guess.

The result gets cached by sender and pattern, so similar future messages benefit without another model call. At roughly $0.0003 per classification and only 20% of messages requiring it, the AI cost is a small fraction of what it would be if every email went through the model.

Why the conservative default matters

The system defaults to "conversation" when signals are ambiguous. This is a deliberate choice about which kind of error is acceptable.

There are two ways to be wrong:

• False negative: a newsletter ends up in the inbox. The user sees it, recognizes it, moves on. Minor inconvenience.
• False positive: a real email from a person gets classified as a newsletter and filtered out. The user might never see it. Missed meeting, missed opportunity.

These errors are not symmetrical. The cost of a false positive is dramatically higher. So the system is tuned to err on the side of showing things, not hiding them.

This principle, that different error types have asymmetric costs, and the system should be biased toward the less costly error, applies well beyond email. Content moderation systems that over-filter lose legitimate speech. Lead scoring systems that are too aggressive discard real prospects.

The handoff in practice

The full flow:

1. Email arrives
2. Heuristic classifier runs immediately, microseconds
3. High confidence → classification is final, email appears in the right category instantly
4. Low confidence → tentative "conversation" label, email visible in the inbox
5. Uncertain message joins a background queue
6. When the queue reaches a batch threshold, it goes to the AI provider
7. AI returns classifications, results update the UI and get cached

From the user's perspective, most emails are categorized instantly. A small number may shift categories a few seconds later when the AI batch completes. The inbox is never blocked waiting for a model.

This pattern works beyond email

The heuristic-first, AI-second pattern applies to any classification task where a significant portion of inputs have clear, rule-based signals, and the cost of running every input through a model is disproportionate to the value.

Content moderation fits. Known banned phrases and obvious spam can be caught with rules. The edge cases, sarcasm, context-dependent language, are where the model helps.

Support ticket routing fits. Tickets mentioning "billing" or "refund" can be routed by keyword. Ambiguous descriptions need understanding.

Lead scoring fits. A form submission from a known competitor domain is not a real lead. A submission with a corporate email and specific pricing questions probably is.

The principle is the same: use cheap, fast, deterministic rules for the cases where the answer is obvious, and reserve the model for the cases where it actually helps. The result is a system that is faster, cheaper, and more predictable, without sacrificing accuracy where accuracy matters most.