Google has released an new open source benchmark that it says will allow robots to perform tasks by writing their own code in response to human-written instructions.
The company has launched a new website (opens in new tab) to reveal “Code as Policies” (CAP), whereby prompts written in plain English can be interpreted into language-model generated programs (LMPs) written in Python code.
CAP is the successor to PaLM-SayCan (opens in new tab), a project that similarly allowed a physical helper robot to be directed through commands in plain English. CAP promises to allow for more complex tasks to be completed with increased accuracy, in part due to allowing machines to write their own code.
Self-coding robots
In a blog post (opens in new tab) discussing CAP’s release, Google Research Intern Jacky Liang and Research Scientist Andy Zeng describe the motivation for the technology, and what it might mean for the future.
“What if when given instructions from people, robots could autonomously write their own code to interact with the world? […] Given natural language instructions, current language models are highly proficient at writing not only generic code but, as we’ve discovered, code that can control robot actions as well.”
But it might not be the time to throw away your laptop for programming just yet. In testing, Google’s researchers demonstrated simple commands with a similar structure. Test robots were able to “draw a 5 cm hexagon around the middle” [of a whiteboard], and “put the blocks in a horizontal line near the top” [of a square boundary].
In the accompanying paper (opens in new tab), titled “Code as Policies: Language Model Programs for Embodied Control”, the project team concede that CAP currently doesn’t have the ability to deal with especially abstract or complex commands or perceive descriptions of trajectory. The team’s approach also didn’t account for impossible commands being fed though CAP.
In theory, the open source nature of Google’s “robot-centric” implementation of LMPs in Python could result in much quicker implementation of solutions to these issues. The CAP website also contains releases (opens in new tab) via Github, and an interactive demo (opens in new tab) via Google Colab to describe how robots “write” code in response to commands.
English
Arabic
Belarusian
Bengali
Bosnian
Bulgarian
Chinese (Simplified)
Croatian
Czech
Danish
Dutch
Estonian
Filipino
Finnish
French
Georgian
German
Greek
Gujarati
Hausa
Hebrew
Hindi
Hmong
Hungarian
Igbo
Indonesian
Italian
Japanese
Javanese
Kazakh
Korean
Kurdish (Kurmanji)
Kyrgyz
Lao
Latvian
Lithuanian
Macedonian
Malagasy
Norwegian
Persian
Polish
Portuguese
Punjabi
Romanian
Russian
Serbian
Slovak
Slovenian
Spanish
Swedish
Thai
Turkish
Ukrainian
Vietnamese
Yoruba