Open AI researchers created a new technique called "Video PreTraining (VPT)" to train a neural network on how to play Minecraft. This involved gathering 2,000 hours of sample dataset from actual humans playing Minecraft to include not just the raw video, but also exact keypresses and mouse movements..

Figure 1: AI trained neural network to play like human.

Figure 1 shows that the researchers trained an inverse dynamics model (IDM) to predict the future action being taken at each step in the videos. [1]

During the first stage, the AI "watched" 2 thousand hours of labeled gameplay videos. The labeled data was keypresses and mouse movements, and the AI used emulation of a standard mouse and keyboard. As a result, the neural network learnt how to process videos, guess keypresses, and record them.

During the second stage, the neural network watched 70 thousand hours of unlabeled gameplay videos (without data about the keypresses) taken from open sources. As a result, the system learned not only how to walk in the game world, but also how to mine resources and create objects, search for food and hunt, run, swim, bypass obstacles, etc. The AI also learned to pillar jump – to elevate oneself by repeatedly jumping and placing a block underneath oneself.

During the next stage, the researchers involved users who were asked to create a new world in the game, collect the necessary resources and make basic necessities from them. This data was recorded on video and shown to the neural network. The researchers also used a reinforcement learning method, which allowed the AI to eventually create a diamond pickaxe. [2]

Researchers concluded in their study, “VPT paves the path toward allowing agents to learn to act by watching the vast numbers of videos on the internet. Compared to generative video modelling or contrastive methods that would only yield representational priors, VPT offers the exciting possibility of directly learning large scale behavioral priors in more domains than just language.”

They added, “While we only experiment in Minecraft, the game is very open-ended and the native human interface (mouse and keyboard) is very generic, so we believe our results bode well for other similar domains, e.g., computer usage.” [3]

References:

1. https://in.ign.com/minecraft/173208/news/an-ai-was-trained-to-play-minecraft-with-70000-hours-of-youtube-videos
2. https://internetprotocol.co/hype-news/2022/06/27/openai-trained-a-neural-network-to-play-minecraft/
3. https://www.indiatimes.com/technology/science-and-future/scientists-trained-a-neural-network-to-competently-play-minecraft-like-humans-573055.html

Cite this article:

Sri Vasagi K (2022), New VPT Technique to Train a Neural Network to Play Minecraft, AnaTechMaz, pp.72