AI safety needs more people who deploy stuff in the real world
There’s a looming brittleness of "fast ROI" approach that can quickly multiply across finance, healthcare, institutional underpinning of everyday life
For the two years that I've worked in applied AI, any interest in “AI safety” seemed relatively fruitless, unless you’re an ML researcher and influence how the models are built.
During the many months of meandering around AI discussions and literature1, I realized that this false and (somewhat convenient) conviction is exactly the problem. I’ve just finished the BlueDot Impact‘s AGI Strategy course with the intention of figuring out how to bridge the gap between the ambiguous scenario of a future AI risk, and the hammer-and-nail of how AI is being deployed today.
AI safety doesn’t need another mediocre researcher that I maybe could become in a couple of years - it needs AI-safety-oriented-people everywhere else, and since I’m already close enough - working on applied AI in enterprise - I’m starting where I am.
The vast majority of life runs on enterprise workflows - how the government operates, how most things are built, how banks handle your money or how a hospital deals with patient data. My day job is weaving various automations into these workflows; and because of the messiness of the human element inside them, there’s a lot of AI risk that sits right at the deployment layer.
The near-term risks - which we’re already seeing - and the plausible-but-yet-unrelized long-term catastrophic scenarios, like superintelligence taking control over our infrastructure, aren't separate categories, but the same dynamics at different points on a timeline. So the path to the big bad future scenarios runs through the small failures that are already happening.
Which means a lot of the highest-leverage work has to happen at the deployment layer, and it has to happen by people who know that layer: the PMs, engineers, the procurement folks, the operations leads, and the general “middle” of the org chart that gets handed an LLM to figure out how to implement it across their side of the business. And, because all of this has to happen quickly to bring about the ROI, there’s a looming brittleness, that, if multiplied across finance, healthcare, institutional underpinning of everyday life, looks pretty bleak.
Which is why I’m throwing my hat into the AI chatterbox: the intention is safety, but the topics span across the concepts and infrastructure that you as a PM, engineer, exec should know: how do agent harnesses work? What is the Productivity J-curve and why does it apply here? How to think about forward deployed engineers? If I want to know why an agent made a specific decision, can I just look at the chain of thought?
Welcome!
Special shoutout here to Brian Christian for The Alignment Problem, hands-down the best introduction to AI safety or AI in general. If I could download any book into every head working in the tech industry, it would be this one.