I think I agree with just about everything here. Except the bolded.Disclaimer: This isn’t intended to be combative in tone. If it comes off that way, I’m sorry.
Weirdly I think we probably agree on the potential scale of impact but disagree on much of the rest of it.
I’ll start with most trivial, pedantic disagreement. I think the notion that AI is incomprehensible and opaque is greatly overblown.
Not to be overly facile, but virtually everything in existence is incomprehensible to humans at some level of abstraction.
With AI I think this belief stems from our inability to describe the behavior of the system as we would using a transfer function in a linear system.
Sure, the output is both emergent and non-deterministic, but the process of how it’s created and a conceptual description of what it is doing are embarrassingly simple at the fundamental level (ANNs). It is also basically unchanged over the last 40 years (and the most basic element was invented ~70 years ago!).
While I agree governance and control are the foremost concerns, I don’t think this has anything to do with AI itself
The structural problem is with how we’ve organized our civilization. The concentration of power that already exists is what is causing the abdication by governments of their duty to provide oversight. AI didn’t create that, we all allowed that to happen.
My hunch is that access to data centers needed to keep up in the arms race to build the biggest models will continue to be monetized, not restricted. Those massive models will either become economically viable or they’ll shrink to fit the power budgets to make them profitable enough. I’d bet on a hybrid model of training in the cloud and running inference at the edge becoming the most popular paradigm.
The damage I’m most worried about is due less to control, but to widespread access.
From what I gather the “agentic” AI you’re all talking about currently is integration of LLMs with business tooling wrapped in somewhat bespoke programmatic “agents” to provide domain-specific executive function. Does that sound right?
Putting that firepower in the hands of roving gangs of CFO warlords may well cause economic genocide on a massive scale at precisely a time when social safety nets and cohesion are weakened.
Or maybe it’ll trigger a social remodeling that rebalances power in a more equitable manner?
What I hear when I talk to my "smart friends" and listen to those that have been in this area of study for a significant chunk of the true application of ANNs in modern AI (Geoffrey Hinton, Yann Lecun, Et al.) is that we definitely know HOW to make a statistical model and the infrastructure necessary to feed and power it. At the same time they are saying they definitely DON'T KNOW HOW the model chooses the connections necessary to produce the work that is, in an ever increasing fashion, impressing and baffling their creators.
They can not explain exactly how it does everything it does. That is not a fact that makes me think that we have no worries at all.
Thank you for the complete thoughts though. They were very enlightening.