aiqbench analysis
This article analyzes the structural challenges AI safety organizations face in scaling operations due to reliance on founder-proven trust rather than scalable hiring pipelines. It suggests that while experienced outsiders can fill these roles, the current "trust loop" creates significant friction for organizations attempting rapid growth without pre-existing personal relationships. The piece concludes by proposing a new program designed to replicate successful founder-COO dynamics in a more systematic and accessible manner.
Lucky hires: how AI safety orgs find their COOs (and why that doesn't scale)
Published on October 13, 2026 11:26 AM GMT Epistemic status: a hypothesis from about fifteen interviews with people involved in hiring, plus our advising data. The program described at the end is partly designed to test it. TL;DR Whether an AI safety organization can scale often depends on a COO who works as the founder's partner, and not only as the person running operations. Most people in that role, including me, got in because they happened to be close to the founder. Our best guess at why is a trust loop: with thin leadership, organizations only hire people who need no supervision, and since assessing an outsider against that bar is slow and costly, they hire people they have already seen work. Experienced outsiders can do this job, but building that trust from the outside takes time: our operations advisees who moved into the field took a median of seven to eight months to land a role. We're piloting the Organization Builders Program, which tries to create the same conditions in a scalable way. Applications close October 22. I didn't apply for my job I met Claire Boine, the co-founder of Successif, at a workshop where we were both participants. Part of the workshop was peer coaching, and we were paired up to coach each other. Afterward, I offered to continue with weekly calls, as the organization was going through a challenging time. Four months later, she secured additional funding, and asked me if I wanted to join as COO. We met for a workshop, discussed the terms of working together, and I started without ever going through an interview. During the months of our calls, Claire had seen how I approach problems and how I handle disagreements, and I had seen the same of her. Since I was advising her and not applying for anything, the stakes were low for both of us and by the time she asked, we had already built a basis of trust. Discovering that my story was typical In my post on lessons from my career, I wrote that most of my significant job opportunities came through weak ties and unexpected connections rather than job applications. For senior hires, I value knowing how someone will perform, because a bad hire at that level affects the whole organization. Having worked with someone before reduces that uncertainty more than a hiring round. I assumed this was different in AI safety, a field full of long application processes for everything from fellowships to jobs. However, when we started interviewing operations leaders in our field about how they got their roles, I heard similar stories again and again. One leader moved from the operations team of a fiscal sponsor into the organization it supported. They knew at least two others who had taken the same route and suggested we copy it. Sam Smith tracked 27 senior operations searches and found that only seven were filled within six months, about half of them through closed rounds.[1] I think of these as lucky hires, myself included, though the luck is in the access, not the people. In my case, I had spent years putting myself in situations where luck could find me, and the people we interviewed had done the same. Hiring this way has two problems. It doesn't scale: it depends on which trusted people happen to be available when a role opens, and the people doing hard jobs right now usually aren't. And it skips the comparison: if you hire the person in front of you, you never find out whether someone better was out there. What I mean by COO The COO title can mean very different things. In Second in Command (HBR, May 2006), Nathan Bennett and Stephen Miles describe seven types of COO, from an executor who delivers the strategy day to day to an heir apparent who is being prepared to take over. Their main observation is that the COO role is defined less by the work and more by the relationship to the CEO as a person. In this post, I mean what they call the "other half" or the "partner." The other half complements the founder's experience and knowledge, while the partner shares the leadership with them. In a young AI safety organization, this is usually the person who helps turn a founder's research or policy idea into an organization, from setting the organization's culture to hiring and implementing good management practices. The article also says the most important factor in a working CEO-COO pairing is the level of trust between them. This matched my experience with Claire. That trust didn't come only from seeing each other's work. It also came from caring about each other as people and being willing to admit mistakes. Why this role matters for scaling When an organization starts, the founders take on all the roles. This works for some time, but eventually either the work or the founders start to suffer. MATS found that organizations approaching about 20 employees develop a strong need for people who can take on management.[2] At Successif, we hear this regularly. Over the past 12 months, 18 organizations approached us on their own, looking for a senior
Read the full article at Effective Altruism ForumRelated coverage

How to apply to AI safety fellowships (and beyond)
LessWrong · Oct 2, 2026

Can John Ternus find Apple’s next big thing?
The Verge · Sep 21, 2026

[AINews] not much happened today
Latent Space · Oct 3, 2026
State of the Field: AI for Epistemics and Coordination
LessWrong · Oct 2, 2026