…and git gud.
I’ve been noticing more and more, over my many years wasted spent playing the vidya games, that game AI is really dumb. The typical, as in universal, fix to enable difficulty is for the AI to just straight up cheat. But you don’t find a suitable chess opponent by playing someone a lot worse and giving them extra pieces. Wouldn’t it be nice if AI provided more difficulty by being smarter instead?
The dumbness problem is one I first acutely noticed in Total War: Empire. The AI, or, more accurately, the computer opponents controlled by scripts, were in a state of complete failure.
For strangers to the 4X genre, this is a game where you rule a nation, trying to build all the things, recruit all the armies, conquer all the cities, and eventually accumulate enough bits and bobbins to declare yourself the bestest of all nations, thereby winning the game.
To call the other, computer-controlled nations AI is a bit misleading, because they have no actual intelligence, and don’t really seek to emulate it in any meaningful way. But in any case, this is the term that’s stuck around for referring to your computer-controlled opponents, so I’ll keep using it. Also, the problem of dumb AI is no way unique to 4X games, but I’ll be focusing on them because they’re the best example of it in my opinion.
So, these AIs were having some issues. One nation, I believe it was France, in this game loosely based on the real-world colonial era (a lot less fun in real life than in game form, or so I hear) was marching an army to my border, then to its capital, then back, stuck in an endless loop of doing absolutely nothing. To be fair, the other AIs weren’t faring much better, mostly treading water in various unproductive ways while patiently waiting to be stomped by the smart-brained human player.
A notable engagement was my war with Spain, which had been stuck in limbo for a few dozen turns. In that time, nothing had really happened. I had a strong navy and had blockaded all their ports for several decades, starving their economy. I couldn’t be bothered with a land invasion to conquer their cities, so I figured I would blockade all their ports and take a diplomatic approach, essentially trying to extract concessions for a peace treaty.
The AI was not onboard with this plan. Instead, they decided that the proper cost of a peace treaty for the war they were disastrously losing, which they offered me every turn, should be that I pay them about 5 times more than everything in my treasury. While they made this demand over and over, they also marched their armies back and forth in their territories, seemingly stuck in their own loop.
The particular, highly visible AI failures I noticed in this game really made me pay attention to what the opponents were doing, and the more attention I payed, the more I realized that at no point in this game were any of the AI opponents capable of a remotely robust understanding of the state of the game or of responding to it appropriately. The 4X genre is especially interesting to me when it comes to thinking about game AI, because it’s a genre where there’s a clear need for really good AI, met by an absolute lack of anything that meets this challenge.
4X games are inherently very complicated. There’s a lot of moving parts to fiddle with, and you proceed to fiddle with them for a very long period of time, min-maxing your civilization, constantly angling towards victory. I have to imagine this makes it very hard to make a good AI, and indeed I’ve yet to encounter one.
In theory, human opponents would be ideal for anyone seeking a real challenge in these games. But not many of us can commit to playing campaigns that take 10s of hours with a group of friends, beholden to the whims of when everyone happens to be able to play. There are some options for more flexible multiplayer, but none of these have ever appealed to me much either. Imagine if we had an AI in these games that could actually compete.
To be fair, it is possible to achieve a reasonable amount of challenge. Setting the difficulty to max in Civilization, for example, is mostly going to provide a satisfactory challenge, at least for some amount of time. But in order to achieve this, the AI turns into a bunch of dirty cheaters.
To allow the rudimentary AI to compete, they get things like a tremendous head start at the beginning of the game, coupled with passive boosts to pretty much everything they do to the tune of 50-100% for the entire game. As a consequence of how these games work in practice, with a strong snowball effect, we should really consider these bonuses to be multiplicative, stacking on top of each other.
Starting with two cities means the AI can immediately build twice as much stuff. A 75% bonus to production on top of this means they can immediately build three times as much stuff. If they push this productive power into expansion, they ought to soon be able to produce 10 times as much stuff, or more, quickly leaving you in the dust. But eventually, using your human brain, with intelligent play, you’ll overtake them. It would be easy to think this makes you some sort of galaxy brained genius, but it’s more to do with their incompetence. It’s actually kind of insane how much they cheat, considering they’re still ultimately designed to lose.
For nongamers, please permit me a Sportsball metaphor. Imagine you’re playing a game of the Basketballs. You’re alone, and on the other side of the court is 10 NBA pros. They all have a couple extra arms, cuz that seems like it’d be helpful. Also, your basket is at a height of 5 feet instead of 10, every goal they score counts for double points, and they start at 20. But ultimately, you know you’re going to win because 3 of them are just spinning in circles in place, 2 others will endlessly pass the ball back and forth between them whenever they lay hands on it, another is glitched out with that little loading wheel over his face, and the rest seem unclear on what a basket is, what a ball is, and how can Basket Ball?
You know what would really push the envelope for 4X games, and perhaps all games? An AI that plays the game even slightly well instead of cheating. I would love to see machine learning in this context. Many games are clearly too complex for a team of humans to sit there and write good, scripted AI for. Or, at the very least, it’s not enough of a priority for the developers to be willing to spend the time and resources it would take. But machine learning could theoretically train smart, capable AIs while using an acceptable amount of development resources.
Also built in to the machine learning model is how the neural networks need to be trained. You’ll want multiple difficulty settings, of course, which could come in the form of the same AI at various stages in its training, with the most difficult being the most up-to-date. Some finessing would be required, sure, but it seems like a natural fit, at least from a high-level view.
Of course, there are obstacles as well. For instance, if the game is updated sufficiently, does the AI have to be retrained from scratch? Will even small updates degrade AI performance, as the AI is trained on an older version of the game? Is it feasible to train the AI in the actual game itself, or will that be too slow to institute the required number of iterations? And, how smart do we actually want the AI to be?
Still, for me, the potential for the application of machine learning to make game AI smarter is much more exciting than the next generation of graphics, VR, or any of the other technologies the industry seems the most interested in pushing and pursuing (cough, metaverse, cough, NFTs, cough). I just hope that someday soon I have the chance to write another post complaining that game AI is cheating by being too damn smart for my squishy human brain.
Song of the week: Civilian – Wye Oak
