Free and open · By Steven Macchia, Trust & Safety leader
The T&S Handbook
How do you build and run a Trust & Safety team, from the first hire to a regulated program at scale?
A practitioner's handbook in 19 chapters for founders, product leaders and new Trust & Safety leads. No sign-up.
General information, not legal advice. Check with your own legal team before acting on anything here.
19 of 19 chapters drafted
Updated Oct 4, 2026 · What's new
Contents
19 chapters in 4 parts. Read Part 1 in order. After that, go to the chapter for the problem in front of you.
Part 1Before the first hire
- 01What Trust & Safety is for
What is this function for, and how do you know it's working?
Draft - 02Know your risks
How will people misuse this product, and which harms should you tackle first?
Draft - 03The first 90 days
You're the first safety hire. What do you do first?
Draft
Part 2Build
- 04Writing policy and an enforcement ladder
How do you write rules people can follow and reviewers can apply the same way?
Draft - 05Detection and prevention
How do you find harm before users have to report it, and stop it before it happens?
Draft - 06Child safety and age assurance
How do you keep children safe on a product adults use too?
DraftUpdated Oct 3 - 07Standing up review operations
How do you build a review operation that's fast, consistent and affordable?
Draft - 08Hiring and structuring the team
Who do you hire, in what order, and where should the team report?
Draft - 09Choosing vendors and tools
When should you buy, build or use open source, and how do you choose?
Draft
Part 3Run
- 10Quality, calibration and appeals
How do you know decisions are right, and fix them when they aren't?
Draft - 11Measuring what matters
Is the program making people safer, and how would you prove it?
Draft - 12Severe harm escalations
What happens in the first hour after you find child sexual abuse material, a threat to life or a request from law enforcement?
Draft - 13Crisis response
When something goes badly wrong in public, who decides what, and how fast?
Draft - 14Moderator wellbeing
How do you protect the people who look at the worst content?
Draft - 15Working with Product, Legal, Comms and leadership
How do you get safety built in rather than bolted on?
Draft
Part 4Scale and govern
- 16Regulation and compliance
Which laws apply to your service, and what do they make you do?
Draft - 17Transparency reports and enforcement notices
How do you explain your decisions to users and the public?
Draft - 18AI in Trust & Safety
Where does AI help moderation, and how do you keep AI products safe?
Draft - 19Budgets, roadmaps and making the case
How do you get the people and money the program needs?
Draft
What it believes
Ten principles behind every chapter
Each one comes from something Steven has written. Open a principle for the full line and the posts behind it. The posts are collected on the writing page.
01Measure impact, not activity.
More removals can mean better detection, more harm or more good users caught by mistake. Report outcomes next to operations, and say so when they disagree.
From: Activity is easy to measure. Impact is harder. · Harassed players look like your best-retained users02Automate as far as the evidence supports, and no further.
Where a wrong decision can't be reversed or someone's safety is at risk, automation prepares the case and a person closes it.
From: Automation rate isn't a measure of maturity03Automation earns its scope.
It expands into a new area only when its overturn rate matches human review, and it rolls back when that rate moves.
From: Appeal overturns: the early warning for automation04Harm is a pattern, not a message.
Grooming, raids and scams build over time, so the response should build as signals stack.
From: Grooming is a pattern, not a message · Games are where kids socialize now, and regulators know it · Watch the rate, not the volume05Put friction where the risk is.
Safer defaults and targeted limits stop most harm before anything needs removing, and most users never feel them.
From: Age assurance: what should each check unlock? · Watch the rate, not the volume · Run the abuse pre-mortem at design review
06A ban is one move, not a closed case.
Offenders come back on new accounts and move across platforms, so map the network and share signals.
From: A banned account is not a closed case · Nudify apps: harm no single platform sees in full07Age assurance is the foundation.
Every protection for children assumes you know who's a child, and each level of assurance should unlock only what it can support.
From: Age assurance: what should each check unlock? · Saturday reading: child safety gaps in games, Ofcom in court and California's new laws08Get in at design review, and earn the invite.
Before launch, changing a default is an edit. After launch, it's taking something away. Be selective about what gets a full review, and answer in days.
From: Run the abuse pre-mortem at design review · Launches get a security review. Almost none get an abuse review. · You can't moderate your way out of a systems problem09Be able to prove it works.
Regulators, courts and attorneys general have moved from "do you have a policy?" to "can you prove it works?" That takes an owner for every threshold and a written record of every change.
From: The era of voluntary child safety is ending · Grooming is a pattern, not a message · Age assurance: what should each check unlock?10Safety is a retention and revenue question.
Prove it with your own data, cut correctly, so the program isn't judged as a cost center by its throughput.
From: Harassed players look like your best-retained users · Activity is easy to measure. Impact is harder.
Who it's for
Anyone who just became responsible for keeping people safe
- A founder or product leader who has just realized safety is now part of their job
- The first safety hire, building from nothing
- A new Trust & Safety leader inheriting a team, or growing one into new markets and new laws
- Anyone moving into Trust & Safety from operations, policy, legal or data
Examples lean on gaming, social and user-generated content platforms, Steven's background, and on child safety, where the stakes are highest.
How to read it
Each chapter covers one part of the job: what good looks like at your stage, how to do it, the mistakes that cost teams most, a template to start from, and the T&S Workbench tool that does the work with you.
Every chapter shows what good looks like at three stages
Goes with
The handbook explains the job. T&S Workbench gives you free, private tools to do it, and every chapter links the ones that go with it.
What's new
Updated Oct 4, 2026
The handbook grows as Steven writes. Posts are linked to the chapters they inform, and chapters are drafted and revised from them. Every change, by date →
Latest changes
- Revised15 chapters (1, 2, 3, 4, 7, 8, 9, 10, 11, 12, 13, 14, 15, 17, 18)
Added Steven's own calls from an interview: where Trust & Safety should report, what to automate first, the one number to track from day one, who makes the 2am call, and more. Practical choices that vary by platform are now laid out as options.
- Drafted18 chapters (1, 2, 3, 4, 5, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19)
First full drafts of the other 18 chapters, built on Steven's posts, the handbook's principles and the Workbench's open content, with every legal and factual claim checked against its source. Stories from Steven's own work come next.
- Drafted6. Child safety and age assurance
First full draft: what each level of age assurance should unlock, safe defaults, grooming responses that build as signals stack, the account decision, and how to measure prevention.
Latest posts
Saturday reading: child safety gaps in games, Ofcom in court and California's new laws
Australia's eSafety found Fortnite and Minecraft still mostly rely on self-declared age. Age assurance is the foundation, because every other safeguard assumes you know who's a kid.
Watch the rate, not the volume
In Stream's live sports chat data, volume swung 7.5x but the rate of racist content held steady. Alert on rate jumps to spot raids, and slow down new accounts instead of locking the room.
Run the abuse pre-mortem at design review
At design review, changing a default is a quick edit. After launch it means taking something away. T&S earns the invite by being selective and fast, and accepted risks get an owner and a date.
Harassed players look like your best-retained users
Raw retention data hides the cost of toxicity because harassed players are the most engaged. A matched cohort shows it, and safety exposure belongs on the retention dashboard.