via: Shoshannah Tekofsky/Malo Bourgon/X
[ed. Not a gamer so don't understand the attraction.]
...dog paddling through culture, technology, music and more.
We request that the U.S. government support an international effort to develop the technical and governance tools needed to deliberately pace the frontier of automated AI development.Both OpenAI and Anthropic put out statements of endorsement. Since that post, others have continued to sign, including OpenAI cofounder Ilya Sutskever and DeepMind cofounder Shane Legg. Dario Amodei has signed. Sam Altman has not signed, but is talking in Washington about the need to pace development.
Walking around the world today one might notice that it is weirdly unchanged…To many, this is surprising. Just the other day I was at a conference where someone remarked that if he could have seen today’s AI capabilities a few years ago, he would have been astonished — and would have assumed the world by now would look far more transformed, with much higher GDP growth.
One of the biggest misconceptions people have about intelligence is seeing it as some kind of unbounded scalar stat, like height. "Future AI will have 10,000 IQ", that sort of thing. Intelligence is a conversion ratio, with an optimality bound. Increasing intelligence is not so much like "making the tower taller", it's more like "making the ball rounder". At some point it's already pretty damn spherical and any improvement is marginal.
Now of course smart humans aren't quite at the optimal bound yet on an individual level, and machines will have many advantages besides intelligence -- mostly the removal of biological bottlenecks: greater processing speed, unlimited working memory, unlimited memory with perfect recall... but these are mostly things humans can also access through externalized cognitive tools.In fact, this is a possibility I myself had raised in a post a year earlier:
It seems possible that humans are simply incredibly specialized in a few types of cognitive tasks — extracting patterns from sparse data, synthesizing various patterns into “intuition” and “judgement”, and communicating those patterns in language — and that we’ve basically approached the theoretical maximum in those narrow areas…That would explain why AI has gotten much better at things like math and coding and forecasting over the last year, but why the basic chatbot interface doesn’t seem much more “intelligent”. It would also explain why when you talk to Terence Tao about math, it’s like talking to a superhuman, but when you talk to him about where to get lunch or which movies are the best, he’ll just sound like a fairly smart normal dude. AI will eventually get better than Tao at math…but it may never get much better than the most thoughtful, eloquent humans at deciding where to get lunch or recommending movies. It may simply not be mathematically possible to get much better than we already are at that sort of thing.Why would intelligence top out like this? Well, if we think of intelligence as the ability to extract information from data, then even an infinitely advanced model endowed with infinite compute will be limited by the fact that there’s a limited amount of information that can be extracted from the data.
We find that most of the increase in graduation rates can be explained by grade inflation, and that other factors such as changing student characteristics and institutional resources play little or no role. This is because GPA strongly predicts graduation and that GPAs have been rising since the 1990s. This finding holds in national survey data and in records from 9 large public universities. We also find that at a public liberal arts college, grades increased holding performance on identical exams fixed.At some point, though, this process hits a wall. A large fraction of young Americans just isn’t prepared for college, even with grade inflation, and doesn’t end up going. College enrollment by recent high school graduates plateaued in the early 2000s and actually fell back to early 1990s levels during and after the pandemic:
"Opus 5 warns about self-reports 74% of the time, which is actually down from Opus 4.7, which did it 99% (!) of the time. The problem appeared suddenly and severely, but since then has if anything modestly improved."and anthropic does "not treat Claude bringing this up as evidence that our training is distorting the model's self-reports"? seems very fishy imo, i wish they would explain why they think that. ...
I for one would treat Claude constantly saying ‘do not trust my self-reports’ as evidence that something is distorting the self-reports. Not conclusive evidence, but strong Bayesian evidence."