OMA held flat at −0.04pp — but underneath, the mix is highly turbulent

ID: a recurring pattern, not a one-off

ID OMA accuracy fell from 90.22% → 86.81% in W16 (−3.40pp). This is the third significant decline in four weeks: W13 92.79% → W14 89.63% → W15 90.22% → W16 86.81%. Cumulative drop: −5.97pp from W13 baseline.

The market is also gaining global share (8.70% → 8.85%) while accuracy worsens — the interaction effect is small but negative. Suggests either Indonesia-specific moderation quality is degrading, or the additional volume is concentrated in harder-to-judge content.

Action: Request structured Indonesia retrospective. The trend is now clear enough to need a dedicated investigation.

LATAM: pure rate-effect dominance

LATAM accuracy fell from 85.54% → 82.51%, a 3.03pp drop. Weight grew slightly (8.79% → 9.24%) which marginally amplified damage via interaction (−0.014pp).

The rate component (−0.266pp) is by far the largest driver. Investigate whether a regional policy change, language model update, or sampling shift hit the LATAM portfolio specifically in W16.

MENA2: bounce-back from a chronic underperformer

MENA2 accuracy jumped 77.79% → 84.63% (+6.84pp). Weight contracted slightly (4.39% → 4.00%), so this is overwhelmingly a rate story.

Looking back, MENA2 has been a problem region — this single-week recovery is the largest market gain in the dataset. Whether it's durable depends on whether the W14–W15 issue was a one-off (sample anomaly, transient labeling problem) or whether deeper calibration work lifted the floor.

Confirm with the regional team whether structural changes were made.

Tobacco's outsized impact

At 10.81% of W16 sample weight, Tobacco & Nicotine is the 2nd-largest single policy (after Youth Regulated Goods at 12.29%). Its accuracy moves the global needle directly.

Multi-week trajectory: clear recovery from W14 trough

• W13: 84.66% acc, 13.23% wt — recent peak
• W14: 76.07% acc, 11.06% wt — −8.59pp single-week collapse
• W15: 79.04% acc, 12.23% wt — partial recovery (+2.97pp)
• W16: 81.20% acc, 10.81% wt — continued recovery (+2.16pp)

Tobacco quality has rebounded ~5.13pp from the W14 trough, but is still 3.46pp below its W13 baseline. The trajectory is clearly positive.

Shift-share decomposition (W16 vs W15)

• Rate effect: +0.264pp — accuracy gain at prior weight
• Weight effect: +0.099pp — shrinking a below-mean segment helps
• Interaction: −0.031pp — small, accuracy ↑ while weight ↓
• Total: +0.334pp (≈ −952% of the global Δ)

What to watch

Sample volume: 1,878 → 1,537 cases (−18%). Some of the weight contraction may reflect a sampling change. Verify the methodology hasn't changed.

Below-mean accuracy persistence: at 81.20%, Tobacco is still 4.80pp below the global mean. If volume rebounds before quality recovers further, the helpful weight-effect direction will reverse — Tobacco could flip back to a major drag.

Action: Lock in the recovery — confirm whether the W14 trough was an isolated event and whether the 3-week rebound has structural support, not just regression-to-mean.

Violent Behaviors: triple-negative, all three effects against

Accuracy collapsed 76.78% → 57.46% (−19.32pp). Weight grew (1.47% → 1.77%), so the additional volume entered a now-failing segment — interaction effect (−0.058pp) compounds the damage.

This is one of the largest reputational-risk policy categories. A 19pp accuracy drop combined with growing volume is a serious signal — escalate immediately.

Personal Info High Risk: catastrophic single-week drop

Accuracy fell 84.12% → 53.12% (−31.01pp) on stable weight (~0.69%). The pure rate effect (−0.208pp) entirely explains this row's contribution.

A 31pp drop on a privacy-related, high-stakes policy is alarming. Possible drivers: policy interpretation change, new content vector (e.g., new types of doxxing patterns), or model/labeler retraining gone wrong. Investigate before W17.

A.S.B: the largest single offset

Adult Sexualized Behaviors recovered 54.88% → 58.75% (+3.87pp). Weight contracted 5.77% → 5.00% (−0.77pp). Both effects are favorable: rate (+0.224pp) and weight (+0.239pp) — shrinking a below-mean segment helps.

This single policy contributed +0.433pp — by itself, more than 12× the global Δ in the offsetting direction. Worth understanding what drove the accuracy jump (calibration, content shift, sampling) since A.S.B is a chronic problem area.

Disparaging Religion: most severe rate drop

This policy collapsed by 75.96pp on a tiny sample weight (~0.08–0.13%). Global impact is "only" −0.099pp (246%), but the rate magnitude is unprecedented.

Almost certainly a sample/policy/labeling artifact — a 76pp single-week swing is implausible as a true accuracy change. Verify the W16 sample is representative; if it is, escalate as a critical operational failure.

Volume change: Tobacco shrank faster than the pipeline

OMA total volume W15 → W16: 5,645,117 → 4,544,936 (−19.6%).
Tobacco volume W15 → W16: 288,581 → 197,907 (−31.4%).

If Tobacco had shrunk at the same rate as the pipeline, it would have been ~232K (still 12.23% share). The fact that it dropped to 198K means Tobacco genuinely lost share — about 11.6% relative share contraction, or 1.42 percentage points absolute.

Share change: real but not dramatic

12.23% → 10.81% looks like a meaningful drop. But put in context: Tobacco's share has been oscillating in the 11–13% band for 4 weeks (W13 13.23%, W14 11.06%, W15 12.23%, W16 10.81%). The W16 figure is at the bottom of that band but not an outlier.

Why both directions help OMA right now

Tobacco accuracy (81.20%) is 4.84pp below the global mean (86.00%). For below-mean segments:

• Accuracy ↑ directly lifts the global average via the rate effect.
• Share ↓ removes a drag from the mix via the weight effect.
• Both happening together is the ideal direction for an underperforming policy.

This is why the W16 contribution (+0.334pp) is so much bigger than what the +2.16pp accuracy gain alone would predict.

Three components, all favorable in net

• Rate effect: +0.264pp = 12.23% × (+2.16pp) — pure accuracy gain at prior weight
• Weight effect: +0.099pp = (−1.42pp) × (79.04% − 86.04%) — shrinking a below-mean segment helps
• Interaction: −0.031pp = (−1.42pp) × (+2.16pp) — small, since accuracy ↑ while weight ↓ (opposite directions)
• Total: +0.334pp

If Tobacco had only improved accuracy (+2.16pp) without the share drop, contribution would be just +0.264pp. The share contraction adds another +0.099pp — about 30% extra leverage. The interaction is small because the two factors moved in opposite directions, which limits the joint effect.

The favorable direction can reverse fast

Scenario A — Volume rebounds before accuracy fully recovers: If W17 sees Tobacco volume pop back to ~12% share (a ~+1.2pp share gain) while accuracy plateaus at ~81%, the weight effect flips to negative. A +1.2pp share increase × (81.0% − 86.0%) ≈ −0.06pp drag. Combined with no rate gain, Tobacco could contribute close to zero or slightly negative.

Scenario B — Accuracy recovery stalls or reverses: Tobacco hit 84.66% in W13 — that's the recent ceiling. Without structural fixes, regression to ~79% (the W15 level) is plausible. A −2pp accuracy fall × 11% weight ≈ −0.22pp drag in a single week.

Scenario C — Both reverse: Volume back to 13% AND accuracy back to 79% would create a triple-negative similar to what happened in W14 (−8.59pp accuracy collapse). The W16 +0.334pp tailwind would flip to ~−0.40pp drag — a 0.7pp swing on a single policy.

Action: Monitor whether the W14 trough was caused by an isolated event (sample anomaly, transient labeling problem) or a structural issue. The 3-week recovery looks real but is on a thin base.

Two different "weights" — don't conflate them

The OMA dashboard has two volume-like metrics for each policy:

• Weight (raw volume): total moderation traffic for that policy. W16: 197,907 cases. This represents real production decisions made by the moderation system.
• Annotation sample: how many cases were sampled for human evaluation. W16: 1,537 cases. This is the OHA/OMA evaluation effort.

Tobacco annotation sample: 1,878 → 1,537 (−18.2%). Tobacco's share of total annotation samples: 6.73% → 6.76% — essentially flat. So at the evaluation/sampling level, Tobacco's representation didn't change.

The 31% volume drop is in moderation traffic, not in evaluation effort. This means: Tobacco is being moderated less in production, not just sampled less. Likely drivers: content trends, policy enforcement changes, or seasonality.

MENA2: pure rate-effect explosion

Tobacco accuracy in MENA2 jumped 54.25% → 86.21% (+31.96pp). This is the largest single-market accuracy swing in the entire dataset.

Mechanism breakdown:

• Rate effect: +1.73pp (5.4% within-Tobacco share × +31.96pp accuracy gain)
• Weight effect: +0.35pp (share dropped 5.4%→4.0%, beneficial since MENA2 was below Tobacco's 79% mean)
• Interaction: −0.45pp (share ↓ × accuracy ↑ — opposing directions cancel)
• Total: +1.63pp = 75% of Tobacco's overall +2.16pp gain

Why MENA2 mattered so much: a) starting accuracy was extremely low (54%), so improvement headroom was huge; b) it carried a meaningful share (5.4%) of Tobacco moderation, so each accuracy point of improvement multiplied; c) volume contracted 49% so the additional gain wasn't diluted.

Verify: a 32pp single-week swing in one market is implausible as organic improvement. Likely candidates: labeling guideline change, sample composition shift, or regional moderator team retrained. Investigate before W17.

VN: ideal-direction expansion

Tobacco volume in VN: 5,776 → 16,065 (+178%). Within-Tobacco share rose 2.00% → 8.12% (+6.12pp). Accuracy rose 87.50% → 98.04% (+10.54pp).

This is the best-case pattern: more volume, higher accuracy. All three shift-share components are positive: rate (+0.21pp), weight (+0.52pp — VN was ABOVE Tobacco mean, so growing helps), interaction (+0.64pp).

Why this matters: if VN sustains 98% Tobacco accuracy at 8% share, it becomes a structural Tobacco anchor. Track whether the volume surge is a one-off (data backlog catch-up?) or new baseline.

ID: the largest single drag within Tobacco

ID accounted for 17.17% of Tobacco moderation in W15 — the largest single market. In W16: share crashed to 10.21% (−6.96pp absolute, −41% relative) AND accuracy fell from 88.14% → 83.66% (−4.47pp).

Mechanism: Volume dropped 59% (49,542 → 20,210). Accuracy dropped on top of the volume drop. ID was above Tobacco's mean, so losing share also hurt via weight effect.

• Rate effect: −0.77pp
• Weight effect: −0.63pp (above-mean segment shrinking is bad)
• Interaction: +0.31pp (partially offset because share ↓ × accuracy ↓ aligns negatively, which mathematically becomes positive)
• Total: −1.09pp = removed 50% of what could have been a much bigger Tobacco gain

Tobacco's +2.16pp gain would have been roughly +3.25pp without the ID drag. Indonesia is now the single largest variable in Tobacco's trajectory.

MENA1: textbook triple-negative

MENA1 Tobacco share more than doubled (5.03% → 11.57%, +6.54pp absolute). Accuracy collapsed (88.12% → 74.00%, −14.13pp). Volume +57.9% (14,502 → 22,896).

This is the worst possible direction for a Tobacco market: more volume into a now-failing segment.

• Rate effect: −0.71pp (5% share × −14.13pp accuracy)
• Weight effect: +0.59pp (share grew, but MENA1 was ABOVE Tobacco mean at 88% W15 — so partially offset)
• Interaction: −0.92pp (share ↑ × accuracy ↓ — the worst combination)
• Total: −1.04pp drag

Pattern: Why did MENA1 Tobacco volume surge while quality crashed? Possible: sudden enforcement campaign or content trend in the region pushed more Tobacco cases into review faster than moderator capacity could absorb. Compare to the broader MENA1 market data (which dropped −1.81pp overall) — Tobacco is the dominant contributor to MENA1's regional decline.

What makes a market matter for Tobacco's number

A market's contribution to Tobacco's accuracy delta depends on three factors:

1. Share of Tobacco's volume. The bigger the slice, the more leverage. ID (17%), EN (10%), DE (8%), UA (8%) lead. Low-share markets like LV, EE, NO can have huge accuracy swings with negligible global impact.
2. Accuracy gap from Tobacco's overall mean (~80%). Markets sitting well below mean (LATAM 49%, BR 55%, DE 58%, MENA2 54%) are "headroom" markets — small accuracy gains translate disproportionately. Markets near or above mean (PK 92%, KR 96%) have less upside per unit of effort.
3. Share-direction × accuracy-direction alignment. Best case: above-mean market growing share AND improving (VN). Worst case: above-mean market shrinking AND declining (ID). The interaction term captures this combinatorial effect.

Why MENA2 dominated: low starting accuracy (huge headroom) + meaningful share (5%+) + share contraction at the right time (compounded the rate effect).

Why ID dragged so much: largest share by far (17%), accuracy was above mean at W15 (so losing share hurt), and accuracy also fell (compounding interaction).

Where Tobacco moderation actually happens

EMEA: 47.55% of Tobacco's W16 mix (47.55% share, 94K cases). Accuracy 74.39% — well below Tobacco's 81.20% mean and far below the region-aggregate Tobacco baseline. EMEA Tobacco improved +2.16pp this week, driven almost entirely by MENA2 + DE recovery.

APAC: 44.06% of Tobacco's mix (87K cases). Accuracy 89.79% — well above Tobacco's mean. APAC is the high-quality anchor, but its accuracy slipped −0.12pp this week as ID drag offset VN/MY/TH gains.

AMS: 8.38% of Tobacco's mix (17K cases). Smallest share but biggest WoW accuracy improvement at +15.86pp (LATAM and BR both jumped 8–25pp). Despite the small footprint, AMS contributed +0.84pp to Tobacco's gain (39%).

Implications:

• EMEA is structurally the weakest Tobacco region — sustained EMEA improvement would yield the biggest global lift.
• APAC's flatness this week is mostly an ID story; if ID stabilizes, APAC reverts to a +0.5pp/week tailwind.
• AMS is volatile (LATAM Tobacco at 49% W15 is implausibly low) — the W16 jump may be partially correction-to-mean, not real improvement.

Youth Sexualized Behaviors: rate × growing share = compounding

Multi-week trajectory: W13 70.00% (sample 16) → W14 41.14% (sample 11) → W15 70.00% (sample 51) → W16 47.66% (sample 32).

The W14–W15 oscillation suggests this category sits on the noise threshold, but the W15 figure (70% on 51 samples) is the most reliable W15 baseline — and W16's 47.66% on 32 samples is large enough that the −22pp gap likely reflects a genuine quality issue.

The compounding factor: share grew 9.08% → 15.27% (+6.19pp). Volume jumped 16,656 → 25,077 (+50%). So more cases entered a now-failing category — interaction effect within ID.

Action: The growing volume + falling accuracy is the worst direction for any policy. Investigate whether the volume growth is content-driven (e.g., a viral trend) or operational (changed routing rules), and whether moderator capacity for this category scaled accordingly.

Youth Non-Sexualized Nudity: 81.80% → 66.91%, sample 169 → 98

This is the most statistically reliable drag in ID this week. Even after W16's sample reduction, 98 samples gives reasonable confidence in the −14.89pp signal.

Multi-week: W13 73.25% (sample 43) → W14 94.67% (sample 27) → W15 81.80% (sample 169) → W16 66.91% (sample 98). The category is volatile but the W15→W16 movement is supported by a meaningful sample on both sides.

Share also grew (8.36% → 10.48%, +2.12pp), so the rate × interaction compounded.

Action: This is one of the few ID policy signals that is sample-backed and trend-coherent. Add to the ID quality investigation immediately.

Why Tobacco's role in ID is different from its global role

Globally, Tobacco is a +2.16pp gainer. But in ID specifically:

• Accuracy ↓: 90.47% → 83.66% (−6.81pp)
• Share ↓: 26.33% → 14.34% (−11.99pp absolute, −45% relative)
• Volume ↓: 48,288 → 23,555 (−51%)

This is the rare combination where weight effect more than offsets rate effect. Tobacco was an above-mean segment in ID (90% vs ID's 90.22% mean was barely above). Cutting its share in half removes some volume from a now-mediocre segment, which in shift-share terms is roughly neutral.

4-week trend (Tobacco accuracy in ID): 96.47% → 96.24% → 88.14% → 83.66%. This is a real degradation pattern, and ID is the single biggest contributor to the global Tobacco x Markets drag (−1.09pp on the Tobacco delta).

The ID Tobacco volume halving is itself worth investigating: is it a real moderation traffic decrease (content trend) or a routing/sampling change?

An enforcement campaign signature?

Volume in this policy went 804 → 7,410 in a single week (a 9× jump). Accuracy went 83.02% → 43.38% (−39.6pp).

This pattern is unusual: a category that was near-marginal in W15 suddenly carries 9× volume at much worse quality. Hypothesis: a content trigger (viral trend, regulatory directive, or routing rule change) flooded this category with cases the moderation pipeline wasn't calibrated for.

Sample is small (12 → 11) so the magnitude is uncertain, but the volume pattern alone is worth investigating — that kind of step-change usually has a discrete cause.

Indonesia's accuracy across W13–W16

• W13: 92.79% — high baseline
• W14: 89.63% (−3.16pp WoW)
• W15: 90.22% (+0.58pp recovery)
• W16: 86.81% (−3.41pp)
• Cumulative: −5.97pp from W13 baseline

The recovery in W15 was partial and W16 erased it plus more. ID is now its lowest level in the analyzed window.

The decline is increasingly concentrated in Youth-related policies (per W16 data). This points to either: a) a Indonesia-specific content trend (more youth-content moderation cases hitting the system), b) a localization issue with policy guidelines for youth content in Bahasa Indonesia, or c) a moderator team rotation that affected youth-category calibration.

Important data limitations

1. The "Title Accuracy" column in this dataset isn't directly comparable to OMA accuracy. Sum of policy contributions reaches −14.4pp while ID's actual OMA delta is −3.41pp. The metric definitions diverge — use this table for relative comparison among ID policies, not for precise contribution accounting.

2. Sample sizes are very small. Most policies have W16 samples between 1 and 30. Only 4 policies cross the 30-sample threshold. This is fundamentally a small-N analysis at the policy × market level.

3. Several policies show 0% or 100% accuracy persistently. These are likely pipeline/data-integrity artifacts, not real moderator behavior. Excluded from the dragger table.

4. Within-ID share doesn't sum to 100%. The dataset's "Weight Proportion" column appears to be normalized within sub-buckets, not within ID overall. Volume-derived share has been used in this analysis instead.