Writing · Safety by design and operations

Launch week is the hardest test a safety program gets

Launch week is the hardest test a safety program gets, and most of what goes wrong is decided before day one.

New users arrive all at once, and so do (many of) the people who prey on them. In a game thats raids on the busiest chats and scams in trade and gifting. In a social app its strangers testing the new DMs. Much of it comes from the newest accounts.

Run the abuse review alongside the security review, and decide the friction before launch: slower posting and lower limits for brand-new accounts, tighter rules for anyone unverified. Get product and legal to approve the thresholds in advance, so nobody is hunting for sign-off at 2am on launch night. And when a raid hits, slow down the newest accounts instead of locking the room for everyone.

Reports will spike with the user count, so triage by severity, child safety and threats first. Read the rate of abuse per user, not the raw counts. A jump in the rate for one category is the signal, and it usually means something coordinated.

The necessary conversation is with product. Friction on new accounts slows real newcomers in the week you most want them to stay. Id rather have that argument a month out than on launch night. Agree how much friction youll accept and for how long, then review after week one what fired, what youd pre-approve next time and what it cost new users.

#TrustAndSafety #SafetyByDesign #ProductManagement #GamingSafety

Related

  1. What I'd tell myself in year one of Trust & Safety

    Five lessons from more than ten years in the field: treat safety as a systems problem built into the product, don't treat a ban as a closed case, measure what users experience rather than how busy the team was, let automation earn its scope, and give every risk one named owner.

    1 min read · Essay · Safety by design and operations

  2. If a regulator asked for your pre-launch risk record tomorrow

    Ofcom's investigation into whether Meta risk-assessed Instagram Instants before launch is a test any service with UK users could face. Do the assessment at design review with a named sign-off, decide how disappearing content gets reported, and file a dated record you could hand over within a day.

    1 min read · Essay · Safety by design and operations

  3. Run the abuse pre-mortem at design review

    At design review, changing a default is a quick edit. After launch it means taking something away. T&S earns the invite by being selective and fast, and accepted risks get an owner and a date.

    1 min read · Essay · Safety by design and operations