A robot operated by artificial intelligence (AI) understands human instructions and acts without programming. Google recently showed off such a demo. At its core is a large-scale language model that uses machine learning to learn the relationships between words and actions from vast amounts of text on the web.
Google research scientist Xia Fei sat in the center of her bright and airy kitchen, typing commands into her laptop. The laptop has one arm and wheels and is connected to a robot that looks like a large floor lamp.
“I’m hungry,” Shea typed. The robot then immediately traverses the room toward a nearby counter. Then, with a large piece of plastic shaped like a pincer, he carefully picked up a bag of multigrain chips, wheeled it over to Shea, and handed him the snack.
The most impressive demonstration, held in August 2022 at Google’s Robotics Lab in Mountain View, California, showed how the robot responded to Shea’s commands without being programmed by a human coder. I knew what to do. The software that controlled the robot learned from millions of pages of text plucked from the web how to translate human words into sequences of actions.
That means, unlike virtual assistants like Alexa or Siri, you don’t have to use specific, pre-approved expressions to give commands. If you tell a robot, “My throat is dry,” it will try to find a drink for you. Just say, “Oops, I spilled my drink,” and he’ll get you a sponge.
“Robots need to acquire the ability to adapt and learn from experience in order to deal with different real-world situations,” Google senior research scientist Karol Hausmann explained during the demonstration.. The demonstration also showed the robot fetching a sponge and wiping up a spilled drink.
For machines to interact with humans, they must be able to understand how words can be combined to produce different meanings. “It’s up to robots to understand all the nuances and complexities of language,” says Hausmann.
that it was a large language model problem
Google’s demonstration was the first step toward a long-term goal of creating robots that can interact with humans in complex environments.
Over the past few years, researchers have discovered that if you feed large amounts of text from books and the web to large-scale machine learning models, programs with superior language skills, such as OpenAI’s text generator GPT-3, have discovered that it is possible to produce By processing various forms of documents online, software gains the ability to summarize texts, answer questions about texts, generate articles on specific subjects, and even converse properly with humans. You can.
Tech giants such as Google make extensive use of these large-scale language models for search and advertising purposes. Numerous companies offer this technology through cloud APIs, and new services have emerged that apply the linguistic power of artificial intelligence (AI) to tasks such as code generation and writing ad copy.
Former Google engineer Blake Lemoyne was recently fired after warning that chatbots, dubbed LaMDA, may be conscious. The vice president, who still works at Google, wrote in an op-ed for the British weekly The Economist that chatting with LaMDA made him feel like he was “talking to something intelligent. ”
Despite these advances, AI programs are still prone to confusion and parroting gibberish. Language models trained on web sentences can’t understand facts either, and often reproduce the bigotry and hateful language in their training data.
This suggests that careful engineering may be required to reliably guide robots without causing them to go out of control.
Read Also: Is The New Robot Working In Amazon’s Warehouse Really “Safe”:
Power of advanced language model “PaLM”
The robot Hausmann demonstrated uses PaLM (Pathways Language Model), the most powerful language model Google has ever released.
PaLM can perform a variety of tricks, such as explaining in natural language how it came to its conclusions when answering questions. When a robot performs a given task, it uses the same approach to generate a series of steps and carry them out.
The hardware used by the Google researchers was developed by a company called Everyday Robots. Everyday Robots, spun out of Google’s parent company Alphabet’s “X” division, is working on an ambitious research project aimed at developing a “robot butler.”
A new program developed by Google researchers uses PaLM’s language processing power. This translates a word or command it hears into a series of appropriate actions that the robot can perform, such as “open a drawer” or “pick up chips.”
The library of actions the robot can perform was built through a separate training process, in which a human remotely controlled the robot and taught it to perform actions such as picking up objects. Robots are limited in the tasks they can perform in their environment, so they don’t behave erratically if the language model gets it wrong.
PaLM’s language skills allow the robot to understand even relatively vague commands. For example, when tasking a robotic arm with moving colored blocks and a bowl, Google research scientist Andy Tsung told the robot, “My wife is the blue block and I am the green block.” Imagine, please bring us closer.” The robot responds to requests by placing blue blocks next to green blocks, and so on.
“The application of large-scale language models to robotics is a very exciting goal,” says Stephanie Telex, an assistant professor at Brown University who specializes in robot learning and robot-human collaboration. But expanding the range of tasks that robots can perform and doing more of what humans ask them to do is still a “big open question,” says Telex.
Brian Ichter, a Google research scientist working on the project, admits that Google’s kitchen robot can still get confused by “a lot of things.” Simply changing the lighting or moving an object can cause the robot to perceive the object incorrectly. This shows how trivial tasks for humans can be difficult for robots.
It’s also not clear whether the system can handle complex sentences and instructions as smoothly as it did with short instructions in the demonstration.
Advances in AI are already enabling robots to do more things than ever before. For example, industrial robots can recognize products in factories and find faults. Many researchers are also exploring ways for robots to learn from practice and observation in the real world or in simulations. But even the best-looking demonstrations often work only in limited circumstances.
Will there be a better way to understand the real world?
Ichter says the project could lead to ways for language models to better understand the real world. AI language software often makes mistakes because it lacks common sense knowledge. Humans should be able to use that knowledge to understand the ambiguity of language.
“Language models have never experienced the real world at all. They just reflect the collective data of words read on the Internet,” says Iktor.
It will be a long time before Google’s research projects become actual products. But many of Google’s rivals have recently taken a renewed interest in home robots.
Amazon unveiled a home robot called Astro in September 2021, but it has far more limited capabilities than Google’s robot. And in August, Amazon announced it would acquire iRobot, the creators of the popular Roomba robot vacuum cleaner.
Elon Musk has promised that Tesla will develop a humanoid robot, but not many details about the project have been revealed. It might be more of a recruitment pitch than a product announcement.
