Writing · Safety by design and operations

A T&S policy gets tested every day, on cases nobody wrote into it

A T&S policy gets tested every day, on cases nobody wrote into it.

It only works once it's in the hands of the people and tools making the calls. That means reviewer guidance with real examples, product requirements, and enforcement logic built into the tools. (And there's some great products building this in to their reasoning right now.)

And it's never finished. New features ship, abuse changes, and reviewers hit edge cases every week. Those edge cases are the best source of policy updates you'll get, so give them a way to flow back up. Update the guidance on a regular schedule and write down why each change was made.

The test is consistency. Same case, same call, no matter who reviews it or whose account it is. That includes big creators and top spenders.

Regulators expect this now too. The EU's Digital Services Act says providers have to apply their terms in a "diligent, objective and proportionate manner" (Article 14(4)).

As a T&S leader, you own both the policy and how it plays out. You should be able to walk your CEO or general counsel through it, case by case and in the numbers.

#TrustAndSafety #ContentModeration #Leadership #Compliance

Related

  1. What I'd tell myself in year one of Trust & Safety

    Five lessons from more than ten years in the field: treat safety as a systems problem built into the product, don't treat a ban as a closed case, measure what users experience rather than how busy the team was, let automation earn its scope, and give every risk one named owner.

    1 min read · Essay · Safety by design and operations

  2. If a regulator asked for your pre-launch risk record tomorrow

    Ofcom's investigation into whether Meta risk-assessed Instagram Instants before launch is a test any service with UK users could face. Do the assessment at design review with a named sign-off, decide how disappearing content gets reported, and file a dated record you could hand over within a day.

    1 min read · Essay · Safety by design and operations

  3. Launch week is the hardest test a safety program gets

    Most of what goes wrong in launch week is decided before day one. Pre-approve friction for new and unverified accounts, triage by severity, watch the rate of abuse rather than raw counts, and agree with product how much friction you'll accept, then review it after week one.

    1 min read · Essay · Safety by design and operations