shutterstock
Google taught DeepMind to play Atari games all on its own, but letting humans help may be faster, according to researchers from Microsoft and Germany. They invited folks of varying skills to play five Atari 2600 titles: Ms. Pac-Man, Space Invaders, Video Pinball, Q*Bert and Montezuma's Revenge. After watching 45 hours of human gameplay, the algorithm could beat its mentors at pinball, though it struggled at Montezuma's revenge -- just as Deepmind did.
Unlike with DeepMind's trial-and-error methods (below), however, the human-aided AI learned to play the games in less time than other AI systems. "Current state-of-the-art approaches require millions of training samples," the paper states. "A promising way to tackle this problem is to augment [deep learning] from human demonstrations."
The researchers trained the AI using "Javatari," an Atari 2600 emulator coded in Javascript that works on any browser. They then opened it up to anyone to try, recording actions taken at each time step along with the reward earned. To further motivate players, they showed them how their average scores compared against other humans as well as Google's DeepMind system from 2015.
In the end, the study collected 45 hours of 60 fps gameplay, around 9.7 million frames in all. After crunching that data, the machine could beat humans at pinball and match them at Q*Bert and Space Invaders, though it was overmatched in both Ms. Pac-Man and Montezuma's Revenge.
The team acknowledges that the latter, more strategy-oriented titles are still giving the machine fits. However, it found that recording the feats of better players gives better results, so it's using that data to tweak the next round of research. "The most important development of the dataset is to collect more data from professional players who achieve higher scores," the researchers explain. If you want to do your own study, the massive 12.5 GB dataset is available for anyone to use.