According to the top AI scientist at Meta, AI has sparked a renaissance in tech sector R&D.
Although there are now many intriguing commercial applications that might boost productivity, Yann LeCun argues that for AI to genuinely evolve, it must learn to plan.
According to Yann LeCun, chief AI scientist for Meta, the success of the deep learning age of artificial intelligence has resulted in something resembling a renaissance in business R&D in information technology.
LeCun said at a small conference of reporters and executives via Zoom this month that the kinds of approaches his team has been developing have had a “far broader commercial effect, much more wide-ranging” than was the case in earlier phases of artificial intelligence.
As a result, a lot of research funding has been garnered, and industrial research has been renewed.
Microsoft Research was the only business company that “had any sort of significance in information technology” as recently as twenty years ago, according to LeCun.
Then, according to LeCun, “Google Research came to the fore, and FAIR [Facebook AI Research], which I started, and a few other laboratories sprouting up and revived the concept that industry could perform fundamental research” in the 2010s.
LeCun stated that the reason for this uptick in business R&D is “the enormous potential of what may happen in the future, and what happens in the present, owing to those innovations.”
According to LeCun, the value of applied AI is driving the development of a dual-track system in which corporate R&D keeps up longer-term moonshot programs and another track that directs research toward useful commercial applications.
The development of intelligent virtual assistants with human-level intelligence is what we eventually desire, therefore it makes perfect sense for a firm like Meta to have a huge research lab with ambitious long-term ambitions. However, the technology that has already been produced is also valuable.
LeCun cited Google’s Transformer natural language processing program, introduced in 2017, as an example. The transformer has served as the foundation for numerous programs, such as OpenAI’s ChatGPT.
“For example, content moderation and speech detection in multiple languages has been completely revolutionized over the last two or three years by large, Transformers pre-trained in a self-supervised manner,” said LeCun.
LeCun added, “It’s made incredible, astounding development, and it’s owing to the most recent advances in AI research.
The Collective[i] Forecast, an online, interactive conversation series run by Collective[i], which describes itself as “an AI platform built to maximize B2B sales,” asked LeCun to give a one-and-a-half-hour session.
LeCun was responding to a question posed by ZDNET on the impact the industry and business’s extraordinary interest in AI is having on the field’s fundamental research.
LeCun characterized his outlook on the potential for applied AI to benefit society as “optimistic.” He pointed out that even in cases where AI is unable to fully realize specific objectives, it nevertheless has positive side effects.
LeCun used the example of automated driving systems that, although not fully autonomous, have had the benefit of introducing features that increase road safety and save lives.
According to LeCun, every new automobile in Europe must now include ABS or automated emergency braking. “In the US, it’s not necessary. but many automobiles have it.
“The same systems that also allow the automobile to drive itself on the highway, right?” he said, referring to ABS. He pointed out that the braking system cuts crash by 40%.
Therefore, despite all you read about, say, the Tesla that hit a truck or anything, such things do save lives to the degree where they are necessary.
LeCun said that the application of AI in research and health at present to improve people’s lives is “one of the things I find quite encouraging about AI.”
Many experimental systems increase the accuracy of diagnoses using MRI, X-rays, and other things, of which a few hundred have received FDA clearance, according to LeCun. “The health will be greatly impacted by this.”
“We have technologies today that would be able to create proteins to cling to a particular spot, which means we can develop medications in a fundamentally different approach than we’ve done in the past,” said LeCun.
LeCun added that AI offers “enormous promise for advancement in materials science.” Having high-capacity batteries that are affordable and don’t need the usage of rare materials that are only found in one location is something needed to combat climate change.
LeCun mentioned one such material effort called Open Catalyst, which collaborates with Carnegie Mellon University to employ AI to produce “new catalysts for use in renewable energy storage to aid in tackling climate change.” Open Catalyst was founded by colleagues at FAIR.
The concept is to employ solar panels to cover a small desert and then store the energy generated by those panels, for instance in the form of hydrogen or methane, according to LeCun. He said that the present methods for storing hydrogen or methane products are “either efficient, or scalable, but not both.”
Could we find a new catalyst possibly with the aid of AI—that would make the process more effective or scaleable by not requiring some unusual new material? It’s worth a shot even if it might not succeed.
LeCun argued that despite all of these exciting commercial and applied possibilities for AI, the pursuit of animal- or human-level intelligence falls short of the field’s larger goal.
LeCun said that while basic scientific advancements haven’t always been as numerous or as rich, the tremendous research advances that underlie today’s applications, like Transformers, were made possible in the deep learning age by the unparalleled availability of data and processing.
The more recent wave was brought on by a few conceptual advances—which, to be honest, weren’t all that significant or impressive—but more importantly, by the sheer volume of data and processing that made it feasible to scale such systems up.
Evidence that scaling AI, or adding additional layers of configurable parameters, directly increases program performance includes things like Large Language Models, such as GPT-3, the computer program on which ChatGPT is built.
In terms of GPT-3 and similar technologies, he remarked, “It turns out they perform pretty well when you scale them up.”
By depending solely on growth without considering alternative strategies, the sector can eventually see decreasing returns, according to LeCun.
Simply make things larger, and they will operate better, has been the slogan of many businesses, including OpenAI, he added. However, I believe that we have reached our boundaries at this time.
We don’t seem to be able to train a fully autonomous self-driving [automobile] system by just training bigger neural nets on more data; it doesn’t seem to get there, said LeCun, despite scaling ever-larger models.
Despite how remarkable they are, LeCun claimed that systems like ChatGPT are “not especially new” and “nothing groundbreaking” since they cannot plan.
They are entirely reactive, according to LeCun. The human-typed prompt is equivalent to “you give them a context of a few thousand lines,” and “from that, the system just creates the following token, entirely reactively.”
There is only reactionary planning or breaking down of a big activity into simpler ones, according to LeCun.
LeCun cited the instance of Microsoft’s integration of the OpenAI software Co-Pilot into the GitHub code-management system as an illustration. Such systems have a pretty serious restriction, he noted. They are essentially being utilized as a predictive keyboard on steroids.
You begin writing your program and describe what it should accomplish in the comments. The tools that will finish the program are based on huge language models.
Similar to how cruise control in vehicles assists with highway driving, such auto-complete functions. Because Co-Pilot can write faults in code that are undetectable, “your hands need to stay on the wheel at all times.
LeCun asked, “How can we go from systems that create code that occasionally executes but occasionally doesn’t? “And the answer to this is that all of those systems today are fully reactive and incapable of planning.”
And you don’t need this for intelligent conduct, either.
LeCun said that to achieve intelligent behavior, a system must be able to foresee the effects of its behavior and possess “some form of the internal world model, a mental representation of how the world is going to change as a result of its activities.”
LeCun wrote on the need for planning-capable systems in a thoughtful article last summer, which he extensively addressed with ZDNet in September.
According to LeCun, the rebirth of corporate information technology R&D has not yet resulted in productivity, the most coveted consequence of technology, but that may change in the following ten years.
LeCun noted that economists view AI as a “general-purpose technology,” which means that it “will slowly disseminate in all corners of the economy and industry and affect all economic activity” by various effects such as creating new jobs, dislodging other jobs, etc., “and lead to increased productivity because it fosters innovation,” citing research by Erik Brynjolfsson of Stanford University’s Human-Centered Artificial Intelligence group. In other words, productivity in the economy is the same as an innovation that builds on the invention.
“What Eric, in particular, has been emphasizing is that historically speaking, it takes around 15, 20 years to see a meaningful influence on the productivity of a technological revolution. At least until very recently, we have not noticed an increase in productivity owing to AI.
So, based on his prognosis, it will probably occur within the next ten years.
Given its attractiveness to young academics, LeCun suggested that the rebirth of corporate fundamental R&D in information technology may have some staying power.
LeCun noted that one tendency he and his team have noticed is that young, gifted individuals are increasingly aspiring to work in AI research because it is the “cool thing to do,” as opposed to going into finance in the past. “I believe kids should pursue science.”
Related Post
