AI masters flag-capturing game by learning to collaborate with each other, Science
In the past few years, AI has developed to the stage where it can beat the best Go and chess player one on one. Now, computer scientists give them another superpower-the ability to collaborate. Though working with each other might seem intuitive to us, predicting how others will behave—a crucial component of working on a team—adds a whole new level of complexity and uncertainty for AI to deal with.
In this experiment, scientists chose to let AI play the first-person flag-capturing and shooting game Quake. In this game, two teams navigate around a 3D map to retrieve a flag from their opponent’s base and return it to theirs. Players could also defend their flags by firing a laser to tag enemies who are carrying their flag and recover the flag back to home base. The team with the most captures wins. In this setting, the only data the bots had to learn from was the first-person visual perspective of their character and game points, awarded for things like picking up flags or tagging opponents. Scientists created 30 different bots with brain-inspired algorithms called neural networks, which learn from data by altering the strength of connections between artificial neurons, and pitted them against each other in a series of matches on randomly generated maps.
Initially, the bots acted randomly. But when their actions scored points, the connections that led to the behavior were strengthened through a process called reinforcement learning. Researchers also culled the bots that tended to lose and replaced them with mutated copies of top performers inspired by the way genetic variation and natural selection help animals evolve. After 450,000 games, the researchers arrived at the best bot, which they named For The Win (FTW). They further tested the ability of FTW by putting them in games against an FTW bot missing a crucial learning element, the game’s in-built bots, and humans. The FTW team consistently out-performed their opponents, and only the combination of human and FTW could beat them occasionally (5% of the time).
FTW not only grew to be almost invincible but also showed very sophisticated behaviors during the games. For example, they developed strategies such us following teammates in order to outnumber opponents in laser shooting and loitering near the enemy base when their teammate has the flag to immediately grab it when it reappears. Moreover, in one test, the bots invented a completely novel strategy, exploiting a bug that let teammates give each other a speed boost by shooting them in the back.
Though this approach still has a long way to go before they could work in the real world, the scale of the experiments was remarkable and the high-level behavior those bots developed was absolutely amazing.