Melanie Mitchell, an AI researcher at the Santa Fe Institute, is also excited to see an entirely new approach. “We really haven’t seen this come from the deep learning community that much,” she said. She also agrees with LeCun that large language models cannot be the whole story. “They lack the memory and internal models of the world that really matter,” she said.
Natasha Jaques, a researcher at Google Brain, thinks language models should still play a role. “We know that large language models are super efficient and that there is so much human knowledge that it is odd that language is completely missing from LeCun’s proposals,” she said.
Jaques, who is working on ways to get AIs to share information and abilities with each other, points out that humans don’t have to have direct experience of something to learn about it. We can change our behavior simply by being told something, such as not to touch a hot pan. “How do I update the world model Yann is suggesting if I don’t have the language?” she asked.
There’s another problem, too. If they work, LeCun’s ideas will create a powerful technology that can be as transformative as the internet. However, his proposal does not discuss how the model’s behaviors and motives would be controlled, or who would control them. Abhishek Gupta, founder of the Montreal Institute of AI Ethics and an expert on responsible AI at Boston Consulting Group, says this is a strange omission.
“We should be thinking more about what it takes for AI to work well in a society, and that requires thinking about ethical behavior, among other things,” says Gupta.
However, Jaques notes that LeCun’s proposals are still far more ideas than practical applications. Mitchell said the same: “There’s certainly little risk that this will soon become human-level intelligence.”
LeCun will agree. His aim was to seed a new approach in the hope that others would build on it. “This is something that will take a lot of effort by a lot of people,” he said. “I’m putting this out there because I think this is ultimately the way to go.” If nothing else, he wants to convince people that large language models and reinforcement learning are not the only way forward.
“I hate seeing people waste their time,” he said.
