Stanford University’s Institute for Human-Centered Artificial Intelligence (HAI) has released the seventh annual issue of its comprehensive AI Index report, written by an interdisciplinary team of academic and industrial experts.
For people that haven’t been paying attention, AI has already beaten us in a frankly shocking number of significant benchmarks. In 2015, it surpassed us in image classification, then basic reading comprehension (2017), visual reasoning (2020), and natural language inference (2021).
AI is getting so clever, so fast, that many of the benchmarks used to this point are now obsolete. Indeed, researchers in this area are scrambling to develop new, more challenging benchmarks. To put it simply, AIs are getting so good at passing tests that now we need new tests – not to measure competence, but to highlight areas where humans and AIs are still different, and find where we still have an advantage.
Editorial comment by Richard Gentle:
It will be interesting to see, in the next few developments, whether AI (and thinking here of Chat GPT as one example) can further eliminate ‘hallucinations’ by running deeper checks on its accumulation of available information – for example, a collection of facts about a person called Fred Bloggs, is likely (as in the first version of Chat GPT) to mix a few people with the name ‘Fred Bloggs’ and assign all information to only one of them. This has been a major issue, and concern, for students and university lecturers, when determining the factual content and part-plagiarism of some submissions.
It will also be interesting to see when AI can not only “predict that the cat might jump off the table or that the table is sturdy enough to hold it, given its weight.” but also anticipate in which direction the cat will actually decide to jump off the table.
Views: 13
