Introducing "VEO" - The Virtual Engineering Organization

Hey everybody, John Cox here from SevenPico.

If you’re seeing this email, don’t worry, you didn’t end up on a new list. We just revamping the newsletter we already send every month. New look. Brand new focus.

I’ll tell you why…

For more than a year now, Dave, Brandon, Eric, and I have been experimenting with some cutting edge stuff – using swarms of AI agents, controlled by expert human operators, to build software products in a completely new way.

We call it the Virtual Engineering Organization (VEO), and I believe this is the future of how all product teams will function.

The upside is immense. We’ve measured engineering throughput at 8-16x a typical human engineer (more on this below).

But the learning curve is steep, the mistakes are costly, and the tools and infrastructure for running a team this way don’t really exist (yet). 

So moving forward, we’ll use this space to share what we’re learning as we go so that you can keep your team on the cutting edge too. We’ll show you the good, the bad, and the ugly. What’s working. What’s not.

If you’re interested in the future of product development, you’re going to love it.

The name of this newsletter has a double meaning too. 

Veo is also latin for “I see.”

We’re seeing a complete recreation of the way software products are built. If you’re seeing it too, this space is for you.

-John Cox,

President & Co-founder, SevenPico

Back in the 1950s, IBM had this great ad for their (refrigerator-sized) electronic calculators, touting the idea that buying one was as powerful as hiring 150 extra engineers.

With AI, we’re once again in a “150 extra engineers” moment, but the numbers are bigger, and not everybody is going to get there at the same speed.

Plenty of Human Engineering Orgs are individually leveraging AI tools these days. Software developers use coding assistants to debug faster, or save time writing a new SQL query, bypassing the trip to Stack Overflow.

Engineers that use these tools well can see improvements in throughput (used poorly, they can actually decrease throughput, but that’s a topic for another email).

But while any given developer may be using one or more AI tools, they are not necessarily getting orders of magnitude more leverage.

The Virtual Engineering Organization (VEO) makes one fundamental change: Each human “operator” oversees a team of AI agents, each with specialized roles, that is each capable of several times the throughput of a human programmer.

With each agent writing or reviewing code faster than their human counterpart, and operators able to manage two or more agents at a time, the overall throughput gains stack up fast.

I’m simplifying a little here. 

Obviously, in the first example, each additional tool doesn’t necessarily add 50% additional throughput.

Similarly, in the case of a V.E.O., there’s actually a slight decay in throughput gains as you add each additional agent because managing them is mentally demanding for now (more on this below).

But the takeaway is clear…

The operator of a Virtual Engineering Org has way more leverage than even a very good engineer using individual AI tools to augment their work.

You may be wondering where I’m getting these numbers from. Fair question.

On a recent project, we decided to benchmark VEO performance against that of a typical Human Engineering Org using modern tools.

Our system starts by producing a series of product management artifacts.

They include design documents, each of which are broken into a series of sprint documents. Those in turn break down into feature documents, and ultimately, ticket documents.

The task management system then estimates time needed for each of these. 

That time is currently estimated as though it’s a human engineer working on each ticket (most tools are still built to assume human workers, but that won’t last long).

So, we chose a sprint, sanity-checked the time estimates, and had the AI “punch in and out” each time it worked on a task.

After the sprint – maybe twenty or so tickets in all – we totaled up the time spent “on the clock” by agents.

Overall, it was one eighth the total time estimate from the task manager. Their throughput was 8x faster than reasonable estimates for human engineers.

You still need skilled technologists to manage these agents, so the need for high quality product people isn’t going anywhere.

And for now, the main constraint is that even the best humans can only manage so many at once.

So far, we find two agents is very doable for a single operator. We’ve stretched it to four agents running in parallel, but with the tools right now, that’s more of a burst threshold.

You’re multi-tasking. And what’s more, you’re multi-tasking on something that's intricate and deep and technically complicated. So it’s like trying to do several hard things at once – possible, but hard to sustain.

But the tools will get better, and as they do, operators will manage more and more of these specialized agents.

Our current “vision” is to reach 100.

A single operator managing one hundred specialized agents, each producing, reviewing, and revising code at 8x the speed of a human today. That’d give each operator the power of 799 extra engineers.

Now that’d make for one hell of an IBM ad.