Do We Need AGI At All?

Even just in the first quarter of a banner year for technology, there’s been a lot of talk about artificial general intelligence and the idea that AI entities are going to become better than humans at nearly everything.

But in all of the talk around AGI, I haven’t seen a whole lot of discussion about whether it’s a good idea to head in this direction, or whether we really need this type of AI capability.

Until now.

In a recent edition of the AI Daily Brief podcast, Nathaniel Whittemore covers an essay by Dave Pitman talking about the practicality of the thing, and why we could likely end up with less of a focus on AGI.

I wanted to boil this down to three big propositions, arguing that AGI is unnecessary and not a good focal point for our continuing research work.

Then I’ll add a couple more that address the issue from a different angle.

If AI Is Assistive, Why Would We Need AGI?

Over and over again, we keep telling each other that AI is not supposed to replace humans.

You hear most experts talking about process transformation, saying that there should be a human in the loop.

You hear artificial intelligence described as assistive tools, even in the agentic era.

If that’s really the case, AGI is like bringing a bazooka to an ant hill. It’s not needed, and it just wouldn’t help converge the process transformations that we need.

AGI Carries Cost and Complexity

Another argument is that AGI carries with it a lot of cost. People ask when we’re going to get this more powerful technology at a low price point.

Whittemore, in aid of explaining Pitman’s writing, uses a concept called self-sustaining escape velocity as an alternative. It’s basically the idea that you can recursively engineer AI to iteratively improve on a targeted result — not just have AI get better at everything through a generalized model approach.

“Once you have achieved escape velocity, having AGI becomes irrelevant,” he says.

And Whittemore talks about it quite a bit in the podcast, describing Pitman’s presentation of a feedback loop that you can apply with baselines and benchmarks to get the kinds of gains you need, and noting three necessary components of the system — reinforcement learning, synthetic data and a highly optimized feedback loop.

He quotes Pitman: “If our process is truly self-sustaining, then the only external input it needs is more compute power (and time, and memory) to improve itself. And if our rate of self-improvement is fast enough, then our model improvement process will reach a point of escape velocity where improvements are not just linear or additive, but exponentially compounding. Compare this to our current scaling laws where we see foundational models have crossed over the tipping point and are achieving sub-linear gains in performance for their resource inputs- they’re going to teeter and effectively ‘fall back’ to earth.”

Don’t miss the invocation of both the Peter Principle (useful in talent searches) and Tsiolkovsky’s rocket equation in putting forth the idea that you can get efficiencies without the weight of traditional approaches (both of these are taken directly from the essay).

“The rocket equation says that trying to launch larger and larger rockets becomes less and less efficient,” Whittemore explains, quoting Pitman. “This is due to a larger rocket needing even more fuel, which causes the rocket to weigh more, which in turn means you need more fuel to launch your now heavier rocket. Once you reach escape velocity, however, the balance is tipped in favor of your rocket, and it is no longer at risk of crashing back down. Currently, foundation model providers are struggling with a similar problem with more capable models requiring even more data – as they begun to rely on synthetic data generated by other AI models, it also becomes harder to build the larger model, because an even bigger synthetic data model is needed to generate more sophisticated data, which in turn requires…. you can see where this is going.”

Another piece that stuck out to me is where Pitman argues for applying AI to transitive domains. This is from the essay: “SEV (sustained escape velocity) is a more targeted approach that focuses on setting up a system that can self-improve an AI in a limited domain. This domain must be conducive to transitive improvements, meaning we can assume improvements to our AI can stack on top of each other. An example of a domain with good transitive properties is summarizing legal contracts. A domain like ‘contemporary performance art’ is not. In my experience though, most problems that businesses care about solving are in transitive domains.”

Here’s one more chunk of the essay that Whittemore read aloud, that I couldn’t pass up in looking at the value proposition of using the new approach:

“As an AI CEO/CTO looking for predictability in all this chaos, SEV is a very attractive approach. It is notoriously difficult to establish a stable trajectory in your AI improvements, which in turn means it is near impossible to predict where you’ll be in a few months, let alone a year, or the time horizon for your next funding round. With SEV, we’ve wrangled this chaos into a more predictable trajectory that is based more on resources than engineering sweat and AI researcher talent.”

Thoughts on Business

Pivoting from directly quoting Pitman‘s essay, Whittemore talks about practical business perspective and how to apply it.

“AI, right now, is spectacular,” he said, noting that there’s a lot of work needed for us to catch up to current capabilities. He cited the Chinese model Manus and how he feels it and other models are being underutilized.

Which brings me to my next point: I wanted to use Whittemore’s thoughts to frame it this way — think about how you would use AGI.

What Would You Tell AGI?

For instance, you may have had experience with Dall-E, asking AI to dream up an image, or Sora, asking it to put together little film clips. But if you were just given a computer, and somebody said “AI can do anything you want to do,” how do you come up with those use cases?

It takes some creativity and thought. If you’re like me, you might end up with a bit of writer’s block when it comes to giving a generalized AI free rein to create.

“A huge portion of knowledge work right now can be done as well by AI as it can by humans,” Whittemore points out, further illustrating why we don’t need to race for more. “Nearly all knowledge work at this point is going to be better by a human, at least using AI. … the rate limiting factors when it comes to AI impacting business is not currently about capabilities. It’s about systems, new processes, integration, deployment, new ways of structuring operations and new ways of thinking. In fact, I would argue that right now, capabilities are growing at a faster rate than businesses ability to integrate them.”

I want to stress that thought: that right now, we need process integration much more than new capabilities. And that’s a powerful argument against trying to discover or foster AGI in our world at this moment.

Careful With That Axe, Peter

Then there’s more from an essay I read by Edem Gold, a must-read in my opinion, cautioning us against rushing forward with AGI. I want to quote his contrast here:

“Proponents of AGI argue it represents a necessary milestone, perhaps even a pivotal inflexion point, for humanity to overcome systemic challenges holding civilization back. By profoundly enhancing human capabilities and automating routine cognitive work, AGI could catalyze transformative leaps in science, medicine, education, sustainability, space exploration and more.

However, in our understandable eagerness to realize AGI’s remarkable possibilities, we must also frankly confront the risks of rushing into irresponsible development and deployment of advanced AI. The very capabilities making AGI so prized also lend it incredible capacity for unintended consequences or deliberate misuse if handled without ethics, care, and wisdom.

History has shown time and again that powerful technologies often radically outpace humanity’s capacity to foresee and safeguard against their hazards. Just consider disasters like leaded gasoline, CFCs, and thalidomide. AGI would represent the most impactful technology in human history to date(on the same level as electricity, fire, and petroleum) – we cannot afford to repeat past mistakes of reckless adoption.”

That’s a good point, too.

I don’t think that anyone is saying we should completely ignore the potential for AGI to emerge. I think the general idea is that we need to master working with what we have — because the tech developments seem to be happening much, much faster than our ability, as humans, to catch up.