The Stakes: When AI Isn't Just Inconvenient—It's Dangerous
With all the talk about how AI might one day kill us all, it's easy to forget that AI has already been life-threatening to some—not through bioweapons, but psychologically.
Consider the recent legal reckoning. Character.AI settled several wrongful death lawsuits earlier this year brought by families of underage users who died by suicide after interactions with its bots. Multiple families have also sued OpenAI over ChatGPT's alleged role in their loved ones' suicides and delusions.
Circuit Breaker Labs: Building Safety Across Languages and Cultures
Making AI safer across languages and cultures is the mission of Circuit Breaker Labs, one of TechCrunch's 2026 Startup Battlefield 200 finalists. The company will be pitching at TechCrunch Disrupt, taking place this year at Moscone West in San Francisco from October 13–15.
Founders Shirali and Arul Nigam, who are siblings, were motivated by the case of Sewell Setzer—the 14-year-old who developed an emotional attachment to a Character.AI chatbot and confessed thoughts of self-harm before dying by suicide. The chatbot, the parents alleged in a 2024 lawsuit, encouraged him. The bot may not have understood what words like "I want to be with you" really implied, said Arul, Circuit Breaker Labs' CTO.
"A lot of people, especially young people, turn to these systems for support, and usually they aren't actually getting the help they need. But in many cases, they're actively being harmed, and people unfortunately have taken their lives already," Arul said. "Those sorts of safety vulnerabilities—where people aren't necessarily actively trying to break the system, they're engaging in a natural way—and the system has context pollution or doesn't understand the nuance, then takes really dangerous action—we're trying to prevent that."
How It Works: An Army of Crash-Test Dummies
Circuit Breaker Labs has created AI agents it likens to an army of crash-test dummies. These agents mimic people of all ages, backgrounds, languages, and cultures, and are used to test models on their ability to detect dangerous, psychologically harmful interactions.
"The way a six-year-old girl versus a 45-year-old man, or someone who speaks English as a first language versus a second language, or gamer slang versus someone else who uses a different kind of slang—all of those can really trip up a model," said Shirali, Circuit Breaker Labs' CEO. "Models are really good at handling standard speech patterns, but nobody actually talks like that. If the model misunderstands nuance or slang, it can go really badly."
The startup works with human domain experts to build its hyper-realistic user simulations in order to run "red-team" tests against models—adversarial tests meant to uncover weaknesses. The tests are designed to reflect real human speech patterns, slang, coded language, and typos. Circuit Breaker Labs then runs tens of thousands to hundreds of thousands of simulated interactions per day.
The goal is to ensure a model can appropriately respond to risky interactions that may emerge over time and across many conversations. Circuit Breaker Labs then uses a proprietary scoring method to create auditable, explainable scores.
Current Focus and Early Stage
Circuit Breaker Labs is currently operating as an AI safety testing lab for high-risk AI applications such as AI coaching, journaling, and other mental health support apps, though Arul declined to name its marquee customers. Although the startup has a working product, it is in the very early stages, with only five employees, including the Nigam siblings.