The Org and the Loop
Skip-able background: I don’t post a lot, in fact the last time was about 8 years ago. Then I was still trying to make Biz Ops happen. It didn’t really, but I do think I was spot on about what we’re seeing now in terms of AI agents and the sorts of people who should be managing them. I’m also coming back from a long period of focusing mainly on building to more of a Biz Ops type role and so thinking about a lot of these things at a more strategic level.
I’m not going to make the case that something interesting and new is happening in software development and that it will likely spill over into lots of other arenas. There are plenty of people making it already. This is for people who are bought in and trying to figure out how to best leverage new tools to do work in new ways.
This started in my head as The Org vs. The Loop, but I don’t think that is accurate. There are definitely two distinct ways of approaching the problem, but I don’t think they are opposed. Both are necessarily present in any agentic deployment, it's just a question of what we focus on.
But what am I talking about? The Org approach seeks to mimic a human software development process with agents augmenting or completely replacing some of the humans. This seems to be the most common way people are integrating AI into their work. Agents doing the job of product owner, designer, coder, QA, dev ops, etc. It is easy to understand, relatively low change management risk, and focused on efficiency gains vs. fundamental disruption. This is probably a good thing given how much disruption is already built into agentic deployments.
The Loop* is, in a lot of ways, weird and uncomfortable. Using hundreds or thousands of bot iterations to get something usable feels wasteful and inherently inhuman. It contains all of the worst of throwing massive waves of electricity and silicon at a problem to get “good enough” slop and then assuming the customer won't know the difference because everyone else is doing the same.
However, the Org doesn’t really avoid the Loop, it just sets the handoffs and feedback points roughly where they are in a human software org. It’s a reverse The Matrix. Rather than the machines putting us in a simulation of a generic mid-90s city, we’re putting machines in a simulation of a mid-20s software dev shop. But, what works for human software dev is unlikely to be the most efficient way to organize these weird little guys we have working for us.
Recommended by LinkedIn
So, if we’re using loops anyway, what might be the best way to organize them? The place that agentic workflows really shine is when there are natural points of feedback already out in the world and we can just plug them into the system. An agent working against robust test suites or well-documented APIs really feels like magic. Slop-forking anything and everything is terrifying but real and we can’t ignore it. Agents do best with lots and lots of feedback, way more than a human can provide or would tolerate.
In the absence of direct feedback via specs or existing tests, it falls on your team – human or bot – to provide feedback. There was a pre-LLM pattern for reinforcement learning called Generative Adversarial Networks or GANs. This was the first place I heard the term generative in the context of ML. Back in the deep learning days, people training image recognition bots ran into the unimaginable problem of exhausting labeled images of cats on the internet and had to make their own. So, train one bot (the generator) to make images of cats and train another (the discriminator) to judge if the image really contains a cat or not. And there you have a proto-loop of explicitly creating and providing feedback, but it is at the other extreme of the Ralph loop: way over-specified.
If it feels like I’m beating around the bush and not answering the question, you’re right. I don’t know the answer or if there is one. I feel that the right level of adversarial looping is orders of magnitude higher than you’d want in your human org. Handoffs and context switching are the biggest overhead for humans and trivial for agents. So, how do we organize for that?
Again, I don’t know. The answer is probably unique for your org and probably less pop-culture influenced than my suggestions. But it is also probably weirder than what you’re trying now. That is my main recommendation: Don’t just blindly recreate what works for your human org. Let the Loop get bigger and weirder than you feel comfortable with, but also loop the Loop and constantly give it feedback so your team can learn and grow.
*Geoffrey Huntley of Ralph Wiggum loop fame and Dan Shapiro writing about Dark Factories are must-reads for understanding how weird things can get these days. I’m sure there are lots more, let me know in the comments.
