What Will More Intelligence Actually Do For Us?

In lots of sci-fi books, as soon as artificial superintelligence arrives, it bootstraps itself to even more godlike intelligence in an explosive “singularity” that rapidly transforms the entire physical universe. Lots of people, especially “AI safety” and “effective altruist” types, expected things to play out basically the same way in reality. But looking around, not much has changed since we entered the intelligence explosion. There’s a huge data center boom, and most people use AI on a daily basis, but we still live basically the same lives — driving to work or taking the train, sitting in front of a computer, scrolling on our phones, collecting a paycheck. People are staying in their jobs longer, but employment hasn’t been disrupted in a significant way:

A lot of people I know are surprised by this. Ruxandra Teslo writes:

Walking around the world today one might notice that it is weirdly unchanged…To many, this is surprising. Just the other day I was at a conference where someone remarked that if he could have seen today’s AI capabilities a few years ago, he would have been astonished — and would have assumed the world by now would look far more transformed, with much higher GDP growth.

Teslo blames bottlenecks — governance and other “frictions” — for the slow economic impact. But some others are advancing a more radical hypothesis — that intelligence itself is subject to diminishing returns.

One of these is Francois Chollet, an AI researcher who specializes in measuring AI’s capabilities. In a highly controversial series of tweets back in March, he conjectured that intelligence might be subject to diminishing returns:

One of the biggest misconceptions people have about intelligence is seeing it as some kind of unbounded scalar stat, like height. "Future AI will have 10,000 IQ", that sort of thing. Intelligence is a conversion ratio, with an optimality bound. Increasing intelligence is not so much like "making the tower taller", it's more like "making the ball rounder". At some point it's already pretty damn spherical and any improvement is marginal.

Now of course smart humans aren't quite at the optimal bound yet on an individual level, and machines will have many advantages besides intelligence -- mostly the removal of biological bottlenecks: greater processing speed, unlimited working memory, unlimited memory with perfect recall... but these are mostly things humans can also access through externalized cognitive tools.

In fact, this is a possibility I myself had raised in a post a year earlier:

It seems possible that humans are simply incredibly specialized in a few types of cognitive tasks — extracting patterns from sparse data, synthesizing various patterns into “intuition” and “judgement”, and communicating those patterns in language — and that we’ve basically approached the theoretical maximum in those narrow areas…That would explain why AI has gotten much better at things like math and coding and forecasting over the last year, but why the basic chatbot interface doesn’t seem much more “intelligent”. It would also explain why when you talk to Terence Tao about math, it’s like talking to a superhuman, but when you talk to him about where to get lunch or which movies are the best, he’ll just sound like a fairly smart normal dude. AI will eventually get better than Tao at math…but it may never get much better than the most thoughtful, eloquent humans at deciding where to get lunch or recommending movies. It may simply not be mathematically possible to get much better than we already are at that sort of thing.

Why would intelligence top out like this? Well, if we think of intelligence as the ability to extract information from data, then even an infinitely advanced model endowed with infinite compute will be limited by the fact that there’s a limited amount of information that can be extracted from the data.

For one thing, data itself is in limited supply. You can’t transform the world unless you can (in some generalized sense) understand it, and you can’t understand the world unless you can measure it, and our ability to measure the world is inherently limited and finite. [...]

So although we don’t know yet, it’s possible that humans were already hitting the point of diminishing returns with regards to individual cognitive capacity, and that superintelligent machines will never be as far beyond us as we are beyond dogs. But even if that’s true, I can think of at least three reasons why machine superintelligence could still deliver huge productivity gains. [...]

Distributed tacit knowledge

The German company Zeiss makes the best glass on the planet. If one of the mirrors that Zeiss makes for ASML’s EUV chipmaking machines were the size of Germany, the biggest bump on that mirror would be just one millimeter high. Only a few other companies — and maybe no other company on Earth — can match that. Zeiss’ mirrors also have a number of other amazing properties, like not distorting much due to temperature changes.

How does Zeiss make glass this good? No one knows — not even the people at Zeiss. If the technology were capable of being written down on a blueprint, China would have hacked Zeiss and stolen it, the way Huawei hacked Cisco and Nortel. If the technology were capable of being explained by a former Zeiss employee, or even several former Zeiss employees, China would have paid those people many millions of dollars to spill the beans.

Zeiss’ technology basically can’t be stolen, because it’s tacit and distributed. It consists of a vast number of little tricks and techniques that a huge number of individual employees use on a daily basis. These people don’t always even realize all those little things they’re doing that make the glass come out so good. And each employee knows a different set of tricks and techniques. The knowledge exists at the level of the organization itself, and is thus very hard to steal or recreate.

This is true of lots of corporate technology. A big part of the reason China can cut off the supply of rare earths to the rest of the world any time it wants to is that other countries aren’t very good at refining rare earths. Rare earths are difficult to separate from each other in solutions; it takes a ton of little chemistry tricks to do it cheaply at scale. Chinese refiners have spent four decades building up those little tricks and techniques; American or Japanese refiners won’t simply be able to replicate their efficiency overnight, and so it’ll continue to cost much more to produce rare earths outside China.

Except in the age of AI, this might change. Suppose American rare earth refiners give their employees a bunch of equipment to record everything they do — smart glasses, gloves, and so on — in addition to sensors distributed throughout their plants. AI will be able to synthesize all that information and very rapidly suggest small ways to improve the production process. Many of those little experiments will fail; others will succeed and will quickly be adopted, allowing another round of experimentation and improvement to begin very quickly. Crucially, AI’s ability to do this doesn’t depend on its raw intelligence — only on its ability to handle huge amounts of data very quickly.

In other words, in the age of AI, distributed tacit knowledge might not be nearly as big of a barrier to technological diffusion. This could improve economy-wide productivity, as lagging firms catch up to leading firms much more quickly. A more equal distribution of productivity would also make the economy more competitive, creating more surplus for consumers (though possibly reducing the incentive for firms to innovate, by making technology less excludable).

AI’s ability to quickly produce distributed tacit process knowledge might also supercharge productivity growth at the frontier. Imagine if any company could optimize any production process five times faster than today. The whole economy would speed up, as components got cheaper, turnaround times and product cycles got shorter, and scale-up got much faster.

And as with the previous example, improving the production of distributed tacit knowledge wouldn’t depend on AI’s raw intelligence. It would spring from AI’s ability to act like a computer — to interface directly with sensors, to handle lots of data, to perceive tiny details, and to do everything very very quickly.

by Noah Smith, Noahpinion |  Read more:

Image: Zeiss
[ed. Another thing I've wondered about: historians make a living unearthing little known facts and connecting dots from sources that are deeply buried in paper and microfiche respositories (and early data storage technologies - like 8 and 5 1/4 inch floppy disks). Millions of memos and correspondences that were once widely distributed and now sitting in dusty boxes or warehouses, archived somewhere. Items that could help significantly in understaning more about human judgement and decision-making. How much of this has been scraped for training? Very little, I'd presume.]