Intelligent Artificial Intelligence: How Do NPCs Operate In An Effective, Flavorful, And Varied Way?

And Why Turn Based Strategy Games Are The Proper Medium For Sophisticated Sims

Introduction

I’ve written a lot here and elsewhere on how turn based games allow for superior AI, especially in games with lots of agents who we want to do actually complex things. I’m going to try to go into more detail here about that topic as well as provide a high level explanation of how I plan to make an interesting AI.

There are several long running debates about video games with computer controlled agents and how to make them work in addition to serious debates about players and what they want from an AI. Should they be fun? Flavorful/historical(if the game is about history)? Should they be tough? Is cheating fair? What about copying human meta-gaming? Or should they be difficult without those two shortcuts?

Additionally I’ll be discussing some potential game rules for people who don’t want the most min-maxed genius enemy, even a very flavorful one. Finally I’ll be talking about how to use semi-asynchronous computer agents. They have all the same tools as you and many of the same goals but sometimes they have “personal” goals and personal “preferences for achieving their goals”.

Turn Based Vs Real Time

I’m a big advocate of turn based games for complex strategy. I think it provides a lot of major advantages. Firstly with a two phase plan/act system you can heavily utilize multithreading for the first phase and even execute it during the player’s turn. I actually tried to structure the game so the player would spend at least a couple minutes per turn so as to ameliorate the turn time cost of an high quality AI.

I tried to make the UI as effective and low-click as possible but the moment to moment gameplay and per turn potential action count should make 2-5 minute turns both interesting and common. Additionally while you could theoretically multi-thread real time or “tick-based” games it is hard to do so when the target is constantly changing as ticks pass.

Turn based games also allow for more time for complex battle simulations. Although you can’t watch or control battles in Axioms you can plan them to some degree and they still have a detailed simulation, as I’ve discussed in recent posts. All battles are run immediately after the player’s turn using multithreading. Battles don’t interact in anyway so this allows for relatively simple multithreading which I as a non-PhD/professional can implement effectively. Well I hope so, I haven’t gone deep into the battle multi-threading like I have for AI multi-threading.

Additionally being turn based allows the “Social Occasions”, as described in the eponymous post on this blog, to be much more interactive and interesting and impactful than they are in other games. That includes, in my admittedly biased opinion, the new “Grand Events” announced for CK3. They actually have a pretty strong resemblance to my high level description in last year’s post of “Social Occasions”, as CK3 Secrets did to my 2014 blog post on the Secrets system for Axioms, but this is probably a coincidence. There is just time for much more complex decision making and better planning and more detailed interactions for Social Occasions because Axioms is turn based.

What the Characters in Axioms do is far more detailed, integrated, and flavorful than anything agents do in other strategy or social simulation games. As I’ll detail in this post they are very dissimilar from the bland, forgettable, and interchangeable agents in any non linear narrative video game. They have a full consciousness simulation including, as noted in previous posts discussing Characters: Ideology, Personality, Interests, Desires, Skills, and Goals. They also have Language, Religion, and other more strategy oriented aspects and also these systems are uniquely deep in Axioms.

Difficulty Or Variety Or Verisimilitude?

Should computer agents in strategy games be interesting, competent, or flavorful? Why choose I say. While games that operate at very high abstraction levels or with very unrealistic control schemes can really only choose difficulty as an AI goal, despite devs claiming otherwise, in a game with a detailed and integrated simulation we can make our characters interesting, memorable, distinct, and competent. Most strategy games operate with you as an immortal hivemind command node. RTS, tactical strategy, strategt rpg, etc. Doesn’t matter. They can’t operate any other way.

In Axioms each agent, mostly Characters but sometimes other kinds as noted below, should be competent(unless they specifically aren’t Consciousness wise), varied/distinct from other agents, and operate in at least a semi-understandable way based on their Consciousness.

What Characters Want

Agents in Axioms can be hostile, neutral, or friendly to the player’s Character. There are a wide variety of agents including Characters, Populations, Deities, and Spirits/Demons/Creatures/Monsters. Within these agent types there are a variety of subtypes and such.

Characters can be leaders of various types like rulers, generals, statesment, manipulators, and so forth. They can also be merchants, mercenaries, adventurers, crafters, researchers, and more. I’m going to focus primarily on those playing the “great game” here. Political leaders, who may dabble in one or more other leadership roles, are quite varied in Axioms both in their nature and actions.

Two leaders with the same problem will solve it in distinct ways based on their Consciousness. Consciousness is the class type, a subclass compositionally in the code, that contains the Ideology, Personality, Interests, Desires, Skills, and Goals that Characters will have. Populations also have Ideology but not the others.

Other genres including character based strategy, social simulation, and RPGs might give a character a simplistic ethical system like in a narrative RPG, or a simple set of personality traits like a social sim or a character based strategy game and that is really it. They don’t have a full Personality, they’ll rarely have a multi-polar ideology, their Culture/Background will be more like a minor set of modifiers, and any “Skill” system used for skill-checks and such will be quite simple.

Characters might have predefined quests in a narrative rpg but in none of these genres will they have meaningful interests/hobbies, personal desires, or goals you can help them with. If they want to be an archmage or a successful merchant you’ll at most complete a linear narrative adventure. You won’t train with them or stake them with resources, you and they won’t build up actual contacts with other interesing characters.

Characters will have Interests and Desires. They’ll want to engage with their Interests and act upon their Desires. Desires can be generated by their Personality, Station, Interests, Ideology, Religion, Relationships, and so forth. A Desire can be anything from finding a good husband for their beloved daughters to conquering a continent to gathering estoric magical knowledge to getting revenge on a hated enemy and more. The post on the blog specifically about Desires expands on the potential options.

Desires will impact your relationships both passively and actively. In a polity where Characters can fulfill their own desires autonomously they’ll be happier and more supporter. You can also actively assist other Characters in which case the boost to their happiness and their relationship with you will be more intense.

Interests create another aspect of the distinctness of Characters and Relationships. When you want to make a Character like you for some reason whether romance or political support or pure friendship the optimal way to achieve your goal depends on their Consciousness and especially their Interests. A Desire can be fulfilled by one person generally and is a one off boost. Any number of Characters can engage a single other Character in their Interests.

A Character who is an equestrian and naturalist will generally be very happy to ride horses on wooded trails in the hills. A different Character would get only the base interaction boost to their Opinion of the other Character. They might instead prefer to debate scholarly issues of the day or go to the taverns. If you want to court some noble girl maybe she wants to dance at balls. Or maybe she hates balls and wants to sneak away to watch a magician or train with a spear or sword.

Additionally *your* Character will react differently to these different activities. You might hate dancing in public but suffer it for your childhood crush and then need to decompress with some other activity. The same applies to NPC to NPC interactions. Some Personality types will reduce or remove such friction. A Character with such a trait might make a fantastic spy or diplomat.

How Characters Achieve Goals

Suppose that a Character needs to achieve some Goal. Perhaps bringing low a rival nation. Different Characters will consistently achieve the same goal in different ways based on their Consciousness. One Character with strong physical traits who is militaristic and has an interest in combat and strategy would be predisposed to attempt a war based strategy. An Ambitious Character might crave conquest. A Greedy Character might engage in a campaign of raiding and pillaging and not want to bother ruling conquered territory.

However a Character with strong diplomatic or intrigue skills might attempt to socially isolate their enemy causing allies to break alliances or betrothals to be cancelled. Perhaps a Character skill at Propaganda might create nasty rumors to lower the prestige of the opposing nation. One with Political and Espionage skills might create a revolt among vassals. Depending on the degree of the grudge simply causing them to lose face or have vassals demand lower taxes or something migt work.

When a Character picks a primary goal and then evaluates potential solutions they will be more likely to pick strategies in line with their Consciousness. If the old king was militaristic and built up a large army, the new economics focused king will still consider their military power but they’ll be much more likely to prefer sanctions or tariffs or economic sabotage than the previous king. So they don’t blindly ignore their actual situation. Additionally unless a Character is dull or foolhardy or some such thing they’ll generally try to come up with a worthwhile method that seems likely to succeed. A king focused on administration or trade might very well launch a big war if he has to as part of a primary goal, even if he dislikes conflict or considers himself a poor strategist, while also building up their messenger network or trade fleet over time.

Characters in Axioms need to manage their mental state, family situation, and consider their popular support and the AI knows this. We’ll see how effective they are at balancing things once I get farther into fleshing out the AI and doing playtesting but they will evaluate these factors every turn. Even the player won’t be able to blithly rampage along ignoring their Consciousness minmaxing for a single specific goal at all times.

Tired, depressed, ill, or otherwise impacted rulers will have various penalties to their actions. Some Characters even have factors causing them to focus more on their internal state than their external situation. Cruel Characters for instance will need to do bad stuff to others or face happiness penalties. They’ll also upweight solutions like war or torture that are tagged for them. The same with other personality types.

Characters will attempt to consider the intersections of their goals as well. I want to gain glory and increase revenues. Conquest or raiding or enforcing tribute is good. I also want to bring down a hated rival. I hate him X amount and his nation will be Y harder to defeat, is it worth it to go for him over an easier country? I need to import Cinnamon for cultural reasons. My hated rival has Cinnamon. Should I try to break his relationship with his supplier or should I take an easier option?

Competent, Interesting, And Realistic AI Takes Time

Now we get back to where we were at the start. The computer controlled agents in Axioms are operating at a much more complex level than those in other strategy or social simulation games. They are considering more interrelated variables and more variables overall. This requires a significant amount of processing time even with clever multi-threading and efficient processing strategies.

Axioms tries to never deal with string comparison for instance. You could get simpler and easier to write code if you were willing to take the string comparison performance hit but Axioms couldn’t function that way. Even with the most efficient algorithms you simply can’t do what Axioms does in real time on consumer hardware.

Similarly Axioms has significant RAM requirements. Why? Well we gain a lot of value from simulating very complex relationships between agents but we also have to store that data. At a significant cost in code complexity for me I got the RAM usage from “opinion modifiers” down from 57 to 4 bytes. When you have 100s of 1000s of agents who each have relationships with 1000s of other agents the number of Opinions you have spikes very fast. You’ll probably need a minimum of 32GB of RAM for small to medium maps. I won’t have final numbers until I get much farther into playtesting and performance testing.

I’d expect medium maps to require maybe a minute of turn time processing in the late game, though we can’t know for sure yet, and this is only because the “planning phase” is done while it is the player’s turn. Thus only the “implementation phase” is done between when you press end turn and the new turn. The “planning phase” would probably take 5 minutes in the late game, even though there is significantly multithreading. Single core processing would arguably take 40+ minutes a turn.

Of course Axioms provides, for me anyways, significant value in exchange for the requirement to be turn based, for the high RAM usage, and for using a ton of CPU time. Smart agents, with varied goals, who respond in an understandable and somewhat predictable way and who require unique interaction to deal with.

How The AI Is Designed

We are getting pretty high on word count now but luckily for you, and my fingers, this is the last section. I want to explain at a moderately low level how I’m writing the AI to handle the complexity of the game.

At a basic level I’m working through each interaction an agent can have with an entity in the game, so Province, Population, Character, Character-subtypes like Deities, Resources, Buildings, Religion, Armies, and so forth and looking at how I as a human player would engage with that entity depending on my goal. Then I’m filtering things using the tag system. Then applying weights based on the totality of the agent’s Consciousness. I’ll probably be doing quite a bit of playtesting with this, even during early access.

My current implementation is generating a file for each Action taken by each Character during each Turn. So I can read through the file and see what Goal an Agent had and what they did to achieve that. Agent 1 wanted to conquer Province A so he checked his power vs the owner and then generated a Plan to break off vassals of the Owner, upset their Populations and thus military units, etc.

With more detail the strategy might be:

Goal: Want Province

Action: Declare War

Problem: Owner Too Strong

Plan Spend Half Attention Budget For Primary Goal To Damage Vassal Relationships

Plan Spend Half Attention Budget For Primary Goal To Build Up Own Military

Then I’d break down the two Plan Steps recursively.

Conclusion

My general expectation is that making the NPCs Competent and Interesting will be the most difficult part of developing Axioms. After all I systematically removed as many non-gameplay difficulties as possible, like real time or multiplayer or fancy graphics to save time and effort.

Making the NPCs Varied and Flavorful on the other hand is relatively easy. I designed the game to allow the Player to do lots of interesting things for interesting reasons. And anything I can think of as a player I can teach the AI, in a general sense, and part of the design goal was to create simulation data for the AI to interact with.

I’m pretty confident in creating a game that allows unique and cool stuff and that the AI can play at a level similar to most existing games as far as the 4 design goals. The real issue is making the AI *much better* at the four design goals than existing games. Again these are Competent, Interesting, Varied, and Flavorful.

My argument/complaint about existing strategy games and their AIs is that while having the proper level of simulation fidelity doesn’t automatically make the computer controlled agents able to meet the 4 key design goals, you can’t do it without that. It is not sufficient, I could fail at properly utilizing that fidelity, but it is *necessary*.

Of course many players have low expectations for computer controlled agents and many also can sort of hallucinate interesting CCAs with high graphical fidelity. And many players might fail to engage with mechanically effective CCAs in a game *without* high graphical fidelity. But I set my target audience as mechanics focused players. I guess I won’t be sure if I can/did achievemy goal until Early Access at a minimum.

Somewhat related: I originally planned for a design where players could easily make fully custom AIs. I still want to do that but it will have to be in the second expansion or perhaps the sequel. The main issue is technical problems regarding turn times.