If your team took down 40% more harmful content last quarter than the quarter before, is that good news?
Most Trust & Safety teams would present it that way. The honest answer is that you don't know yet.
It could mean detection improved. It could mean harm on the platform grew. It could mean automation got more aggressive, and legitimate users are paying for it.
Volume alone can't tell you which.
This is one of the most common gaps in how T&S programs report to leadership. We measure how much work the system does, like removals, reports actioned, automation rate, and time to decision. Those numbers are easy to produce, and they almost always go up and to the right.
But what gets measured gets resourced. When a program is rewarded for activity, it optimizes for activity.
The metrics that matter are harder to build:
- Is exposure to severe harm going down?
- After enforcement, do bad actors stop, or just come back on new accounts?
- Are we catching high-risk behavior earlier?
- What are appeals telling us about where our policy or automation is wrong?
Operational metrics show the machine is running. Outcome metrics show it's working. A mature program reports both, and is willing to tell leadership when they disagree. That's the real leadership test. It takes discipline to walk into a business review and say, "Our numbers are up, and I'm not convinced we're safer."
But that's the conversation that earns credibility, directs investment to the right places, and keeps Trust & Safety from being judged like a cost center measured by throughput.
Activity is easy to measure. Impact is harder.