Researchers at the University of Cambridge have made significant strides in the realm of robotic cooking. By programming a robotic chef with a repertoire of eight simple salad recipes and exposing it to instructional cooking videos, they successfully trained the robot to recreate the demonstrated recipes. The breakthrough, published in the journal IEEE Access, showcases the valuable role of video content in advancing automated food production and potentially revolutionizing the deployment of robot chefs.

Figure 1. Robotic Chef. (Credit: University of Cambridge)

Figure 1 shows Robotic Chef. While the concept of robotic chefs has long been a staple of science fiction, implementing their culinary skills in reality has proven to be a complex challenge. Although several companies have developed prototype robot chefs, commercial availability remains limited, with these machines lagging behind human chefs in terms of expertise.

Unlike robots, human cooks can acquire new recipes through observation, whether by watching others cook or browsing instructional videos on platforms like YouTube. However, programming a robot to master a variety of dishes has traditionally been a costly and time-consuming endeavor.

The research team, led by Grzegorz Sochacki from Cambridge's Department of Engineering, aimed to enable a robot chef to learn recipes incrementally, similar to human learning. They devised eight straightforward salad recipes and filmed themselves preparing them. Leveraging a publicly available neural network that could identify various objects, including the fruits and vegetables featured in the salad recipes, the researchers trained their robotic chef.

Employing computer vision techniques, the robot meticulously analyzed each frame of the instructional videos, identifying objects, actions, and even human features such as arms, hands, and faces. By converting both the recipes and videos into vectors and conducting mathematical operations on them, the robot determined the similarity between a demonstration and a vector.

By accurately recognizing the ingredients and actions of the human chef, the robot could deduce which recipe was being prepared. For instance, if the human demonstrator held a knife in one hand and a carrot in the other, the robot inferred that the carrot would be chopped. Impressively, the robot correctly identified the intended recipe in 93% of the 16 videos it watched, even though it only detected 83% of the human chef's actions.

Moreover, the robot exhibited the ability to discern slight variations within a recipe, such as adjustments in portion size or common human errors, accurately recognizing them as variations rather than entirely new recipes. Fascinatingly, the robot even created a ninth recipe entirely on its own, which it then added to its repertoire and successfully prepared.

"We were amazed by the level of nuance the robot could detect," said Sochacki. "Though these recipes were not complex, consisting mainly of chopped fruits and vegetables, the robot excelled at recognizing, for example, that two chopped apples and two chopped carrots constituted the same recipe as three chopped apples and three chopped carrots."

The videos used to train the robot differed from the fast-paced, visually edited food videos popularized by social media influencers. Instead, the robot required clear visuals and unobstructed views of the ingredients. For instance, the human demonstrator had to hold up a carrot for the robot to identify it accurately, as opposed to having their hand wrapped around it.

Sochacki clarified, "Our robot isn't focused on the kind of food videos that go viral on social media—they are often too challenging to follow. However, as robot chefs improve their ability to identify ingredients in food videos, they might be able to leverage platforms like YouTube to learn an extensive range of recipes."

The research received support from Beko plc, as well as the Engineering and Physical Sciences Research Council (EPSRC), a part of UK Research and Innovation (UKRI).

Source: University of Cambridge

Cite this article:

Hana M (2023), Meet the Robot 'Chef' That Masters Cooking by Watching Food Videos, AnaTechmaz, pp.296