Anomaly Detection: When Your Data Breaks (And You Actually Notice)

It’s Thursday morning. You haven’t checked your analytics dashboard in four days.

You open it up for your weekly review and notice something odd. Your conversion rate is down 40% compared to last week.

You panic. You call your marketing team. “What happened? Why didn’t anyone tell me this was broken?”

Your marketing director looks confused. “It wasn’t broken yesterday. I checked.”

You do some digging. Turns out, your Stripe integration broke on Tuesday. For 72 hours, conversions weren’t being tracked. Your paid ads were running fine, customers were buying, but your data pipeline was silently failing.

By the time you noticed, you’d burned $40,000 in ad spend on campaigns you couldn’t measure.

This is why we obsess over Anomaly Detection.

What Is Anomaly Detection?

Anomaly Detection is automated monitoring that screams at you when something deviates from normal.

It’s not a person checking dashboards. It’s a system that knows what “normal” looks like and alerts you the second something looks wrong.

In this case:

Normal: Conversion rate between 2-2.5%
Actual (Tuesday): Conversion rate dropped to 0.1%
Alert triggered at 2 PM: “CRITICAL: Conversion rate dropped 90% in last 24 hours”
Your team investigates immediately instead of discovering it on Thursday.

That’s the difference between a $40,000 loss and a 30-minute investigation.

Types of Anomalies We Monitor

Not all anomalies are created equal. Some are catastrophic. Some are just noise.

Type 1: Data Pipeline Breaks (CRITICAL)

This is when your data stops flowing entirely.

Your Stripe integration stops working
Your Facebook Pixel fires incorrectly
Your Google Analytics loses tracking
Your CRM stops syncing

When this happens, your dashboards show incomplete data. You’re making decisions based on lies.

We set up alerts for:

“Zero conversions in the last 6 hours” (almost always a pipeline break)
“Conversion count drops more than 50% vs. historical average”
“Ad spend is recorded but no conversions are recorded” (pixel failure)

These alert immediately. No human waiting until Thursday to notice.

Type 2: Traffic Anomalies (HIGH)

Your website traffic changes unexpectedly.

Traffic spikes 200% (why? Good news or bot attack?)
Traffic drops 50% (competitor launched something? Your site down?)
One geographic region suddenly has zero traffic (targeting broke?)

We set up ranges based on your historical patterns.

If your normal daily traffic is 5,000-6,000 visits, and suddenly you see 15,000, that’s a 3x spike. We alert you immediately because you need to understand why.

Is it good news (PR coverage, viral post) or bad news (your site was featured on a botnet list)?

Type 3: Revenue Anomalies (HIGH)

Your revenue per customer changes unexpectedly.

Average order value drops 30%
Revenue per paid visitor drops (lower quality traffic acquired)
Customer acquisition cost suddenly doubles (targeting degraded? Bid wars?)

These are early warnings that something is shifting. Not a catastrophe, but worth investigating quickly.

Type 4: Ratio Anomalies (MEDIUM)

The relationship between two metrics breaks.

Normally, for every 100 clicks, you get 2.5 conversions (2.5% conversion rate)
Today, you have 1,000 clicks but only 5 conversions (0.5% conversion rate)
Revenue is the same, so it’s not a pipeline break
But something shifted in your funnel

This might be a landing page issue, a mobile breakage, or a targeting change. You want to know about it.

How We Set Baselines

The key to anomaly detection is knowing what “normal” looks like.

We calculate baselines using historical data. Typically, we look at the last 90 days and calculate:

Average value (mean)
Standard deviation (how much it usually varies)
Seasonal patterns (December converts better than January for e-commerce)

Then, we create alert thresholds.

Here’s an example for conversion rate:

Historical average (last 90 days): 2.3%
Standard deviation: 0.4%
Normal range: 1.5% - 3.1% (mean ± 2 standard deviations)

If today’s conversion rate hits 1.2%, that’s below the normal range. We alert.

If it hits 3.5%, that’s above the normal range. We also alert, because high conversion rates are often data errors (pixel double-firing, for example).

The Challenge of Seasonality

This is where it gets tricky.

If you run an e-commerce business, November and December will naturally have higher conversion rates and lower CAC (more people shopping for holiday gifts).

A naive anomaly detection system would scream, “ALERT: Revenue spiked 200%!” on Black Friday. You don’t need that alert.

So, we build seasonality into the baseline.

We compare each day to the same day from the previous year. Or we account for day-of-week patterns (Sundays convert differently than Tuesdays).

The formula becomes: “Is today’s metric unusually high or low compared to similar days in the past?”

Red Flags We Actually Care About

Here are the anomalies that warrant immediate human attention:

1. Sudden Zero Traffic / Conversions This is almost always a tracking failure. Your site might be fine, but your data is broken.

2. Conversion Rate Drops > 50% This suggests a real problem: broken funnel, site down, or targeting misconfiguration.

3. Cost Per Acquisition Doubles Either your targeting degraded, or competitors are bidding more aggressively. Either way, you need to know.

4. Revenue Up But Traffic Down This is unusual and worth understanding. Did AOV increase? Did you raise prices? Did you unintentionally exclude some traffic?

5. Pixel Fire Rate Drops Below Expected If 20% of your users normally block tracking, but suddenly 80% do, something changed. Maybe Apple released a new privacy feature. Maybe your tag manager is broken.

When NOT to Alert (The False Positive Problem)

If you alert on every tiny fluctuation, your team will ignore all alerts.

We’re very selective. We aim for 99%+ specificity, meaning less than 1% of alerts are false positives.

Here’s what we don’t alert on:

Small fluctuations within normal range (“Conversion rate is 2.1% today instead of 2.3%—within normal variance”)
Predictable changes (“Traffic is lower on Sunday because it’s always lower on Sunday”)
Expected changes (“You paused Facebook ads, so traffic dropped 30%—we expected that”)

We only alert on the stuff that requires human investigation.

Operationalizing Anomaly Alerts

An alert is useless if nobody acts on it.

So, we integrate alerts into your workflow:

Slack notification: Immediate message to your #analytics channel
Email escalation: If nobody acknowledges the alert within 15 minutes, an email goes to your CFO
Dashboard highlight: When you log in, anomalies are flagged in red
Weekly anomaly report: Every Friday, you get a summary of anomalies detected and what they meant

The goal isn’t to panic you. It’s to ensure nothing slips through the cracks for four days.

Real-World Example

Let me walk you through a real scenario.

Tuesday, 2 PM: Your Stripe integration breaks. Conversions stop being recorded.

Traditional approach: You don’t notice until Thursday morning. By then, you’ve burned $40k in ad spend with no recorded ROI.
Anomaly detection approach: At 2:15 PM, an alert pops up: “CRITICAL: Conversion count dropped 95% in last 6 hours. At 2:20 PM, your team investigates. At 2:45 PM, they redeploy the integration. You’ve lost 45 minutes of data instead of 48 hours. Total damage: $500 instead of $40k.

That’s the power of automation.

The Takeaway

You can’t monitor everything manually. Humans are terrible at pattern recognition when you’re drowning in data.

We automate the boring work—constant monitoring—so you can focus on the interesting work—fixing problems when they arise.

An anomaly detection system is the difference between a business that reacts and a business that’s reactive.