Why use Utility AI instead of Behavior Trees and Finite State Machines

After years of working with Behavior Trees and Finite State Machines, I realized that they have many drawbacks and are not a good choice for creating game AIs. Then I explored Utility AI and discovered that it can address many of the disadvantages of Behavior Trees and Finite State Machines. That’s why I like Utility AI and created Utility Intelligence.

Below are the disadvantages of Behavior Trees and Finite State Machines and how Utility AI addresses them.

Temporal coupling between decisions¶

In Behavior Trees and Finite State Machines, an agent makes decisions by answering one Yes-No Question at a time and in a fixed order:

• Should I move towards the enemy?
• Should I attack the enemy?
• Should I flee from the enemy?
  

  if(ShouldMoveTowardsEnemy())
      MoveToEnemy();
  else if(ShouldAttackEnemy())
      AttackEnemy();
  else if(ShouldFleeFromEnemy())
      FleeFromEnemy();
  else
      Idle();
  
  bool ShouldMoveTowardsEnemy()
  {
      return DistanceToEnemy > 10 && MyHealth > 50;
  }
  
  bool ShouldAttackEnemy()
  {
      return DistanceToEnemy < 10;
  }
  
  bool ShouldFleeFromEnemy()
  {
      return DistanceToEnemy < 20 && MyHealth < 20;
  }
  

For each decision, we must define the conditions that determine whether the agent will make this decision or not. And if we want to prioritize one decision over another, we have to change the order of these decisions. In the example above, if DistanceToEnemy = 5 and MyHealth = 10, the agent will decide to attack the enemy rather than flee from the enemy. But if we change the order of these two decisions, the agent will choose to flee from the enemy:

if(ShouldFleeFromEnemy())
    FleeFromEnemy()
else if(ShouldAttackEnemy())
    AttackEnemy()

bool ShouldAttackEnemy()
{
    return DistanceToEnemy < 10;
}

bool ShouldFleeFromEnemy()
{
    return DistanceToEnemy < 20 && MyHealth < 20;
}

For simple AIs, everything is okay, there aren’t any problems. However, as your AIs become more complex, they will have more decisions and their decision-making conditions will be related to a lot of factors like health, energy, distance to enemy, distance to cover, attack cooldown, etc.

Therefore, if your game design changes, it will be very time-consuming to apply these changes to your game because they might cause significant changes to the structure of Behavior Trees and Finite State Machines, and you will have to redesign them from scratch by adding, removing, and reordering the conditions and decisions.

Additionally, testers will have to recheck every behavior of these AIs from the beginning to ensure that they behave as intended. As a result, the cost of syncing your AIs with the game design will increase over time. Eventually, you may lose the ability to change your AIs because the cost of making changes becomes too high.

This is the behavior tree of an agent in one of my previous games:

As you can see, it’s quite complex, isn’t it? I still remember that when it became complex like this, it took a lot of my time to apply changes whenever designers altered the game design. I had to redesign my behavior trees again and again. It was a nightmare, and that’s one of the main reasons I created this plugin.

How Utility AI addresses this¶

Unlike Behavior Trees and Finite State Machines, the question a Utility-Based Agent need to answer is: What do I want to do the most right now? So for each decision, the agent needs to ask itself: How much do I want to take this decision at the moment? And depending on the answers, it assigns a score to each decision, compares all of those decisions to each other and select the best one with the highest score.

As a result, the order of decisions is no longer important, you don’t need to worry about the conditions and order of decisions anymore. What matters to you is simply: What is the most important thing to do at the moment? For example, if the agent health is 30, the energy is 50, the distance to the enemy is 40, what does the agent want to do the most?

• Move towards the enemy?
• Flee from the enemy?
• Attack the enemy?

Decision-making conditions are usually based on a threshold¶

Another drawback of Behavior Trees and Finite State Machines is that the decision-making conditions are usually based on a threshold. Consider this decision-making logic of an Enemy AI:

if(IsPlayerInAttackRange())
    AttackPlayer()
else
    Idle()

bool IsPlayerInAttackRange()
{
    return DistanceToPlayer < 10;
}

With this decision-making logic, the enemy will suddenly attack the player when the player enters its attack range (10 m). And if the player is outside of 10 m, it won’t do anything, even though the distance from it to the player is 11 m. So if players know the attack range of each enemy, they can kill any enemy easily without losing a drop of health.

How Utility AI addresses this¶

In Utility AI, this situation is very unlikely to happen unless you intentionally do so because Utility AI measures How much I want to take this decision at the moment. So the distance to the player is 11 m just means the desire to attack the player is lower than when it is 10 m.

For example, if the player is within 10 m, the score of AttackPlayer is 1.0. Then if the distance to player is 11 m, the score of AttackPlayer will be 0.9. Therefore, regardless of whether the distance is 11 m or 10 m, if the score of AttackPlayer is greater than Idle, then AttackPlayer is still chosen.

This is the reason why agents made by Utility-Based AI are far more natural than the predictably robotic If/Else-Based AI such as Behavior Trees and Finite State Machines.

Any target that meets the conditions can be selected for the chosen decision¶

One more drawback of Behavior Trees and Finite State Machines is that any target that meets the conditions can be selected for the chosen decision.

Suppose we set up the attack logic of our agent as follows:

if(IsEnemyInAttackRange() && IsEnemyHealthLow())
    AttackEnemy()
else
    Idle()

bool IsEnemyInAttackRange()
{
    return DistanceToEnemy <= 10;
}

bool IsEnemyHealthLow()
{
    return EnemyHealth <= 20;
}

What if the agent detects multiple enemies with Health <= 10 and within 10m? Assume that there are 3 enemies that meets the conditions:

1. Enemy 1: Health = 20 and DistanceToEnemy1 = 10.
2. Enemy 1: Health = 10 and DistanceToEnemy1 = 4.
3. Enemy 3: Health = 5 and DistanceToEnemy1 = 8.

Which enemy would the agent attack? The result is that it will attack the Enemy 1 because it is the first one that meets the conditions, but it’s not the best choice in this case.

Surely, you can add more conditions to ensure the best enemy is always selected for the attack decision. However, you have to do this manually, and it takes a lot of your time to think about which conditions to add to ensure the best target is selected for every decision the agent makes.

How Utility AI addresses this¶

For each decision, utility-based agents evaluate all possible targets and select the one with the highest score.

With this approach, you no longer need to worry about whether the selected target for the chosen decision is the best one. You only need to focus on scoring decision-target pairs, and the agent will automatically select the target with the highest score for its chosen decision. This ensures that the target selected for the chosen decision is always the best one.

Decision-making and decision-execution are part of the same process¶

Decision-making is forced to run at the same frequency as Decision-execution¶

A major drawback of Behavior Trees and Finite State Machines is that decision-making and decision-execution are part of the same process, which means decision-making is forced to run at the same frequency as decision-execution.

As a result, even though an agent already has the best decision for the current situation, it still has to think about What decision should I make? in order to execute it.

And if agents need to execute decisions every frame, they will have to make decisions every frame. Why do they have to make decisions when they already have the best one? It’s a waste of resources.

Imagine you’ve decided to go running, but while you’re already running, you have to keep asking yourself: Should I go running? How would you feel about that? It would be a nightmare, right?

How Utility AI addresses this¶

Utility AI is essentially a decision-making technique and is not involved in decision-execution. Therefore, we can easily turn decision-making and decision-execution into two separate processes and run each process at a different frequency.

For example, we can run the decision-execution process every frame while running the decision-making process only every 0.1s or every 0.5s by adjusting the decision-making interval to suit your game’s needs.

Moreover, you can even distribute the decision-making process across multiple frames to balance the workload, or manually run the decision-making process when necessary. This approach significantly improves your game’s performance.

This is difficult to achieve if you use Behavior Trees (Finite State Machines) because decision-making is closely tied to decision-execution by nature in these systems and it’s hard to separate.

Behavior Trees and Finite State Machines are essentially a series of If/Else statements run sequentially.

if(ShouldMoveTowardsEnemy())
    MoveToEnemy();
else if(ShouldAttackEnemy())
    AttackEnemy();
else if(ShouldFleeFromEnemy())
    FleeFromEnemy();
else
    Idle();

How can we separate decision-making from decision-execution in the code above?

Hard to debug¶

If you use Behavior Trees or Finite State Machines as your AI solution, you might find it hard to debug why your agents make the wrong decisions as complexity increases.

Figuring out why a specific decision was not selected as expected requires understanding all the conditions and parameters involved in the decision-making process, such as health, energy, distance to target, armor, damage, etc. And it becomes harder and harder to debug as more parameters are involved.

Sure, you can easily identify the issue within seconds when only a few parameters and conditions are involved in the decision-making process. But what if there are dozens or even hundreds of parameters and conditions?

Imagine your AI has to consider 100 different parameters during decision-making time, for example:

• My health = 70
• My energy = 50
• My armor = 20
• The nearest enemy’s health = 30
• The nearest enemy’s energy = 70
• The nearest enemy’s armor = 50
• Danger = 0.7
• Slash damage = 20
• Slash force = 10
• Slash cooldown = 5
• Shoot damage = 5
• Shoot cooldown = 1
• Distance to the nearest enemy = 20
• Distance to the nearest cover = 10
• Distance to the nearest health station = 30
• …

It also has to evaluate 100 different conditions, such as:

• Am I in danger?
• Am I being attacked?
• Am I being healed?
• Is my health low or high?
• Is my energy low or high?
• Is my ammo low or high?
• Is the nearest enemy’s health low or high?
• Is the nearest enemy’s energy low or high?
• Is the nearest enemy’s ammo low or high?
• Are there any enemies nearby?
• Are there any cover nearby?
• Are there any health stations nearby?
• Are there any enemies within my attack range?
• Are there any allies within my healing range?
• Is my distance to the nearest enemy greater than or less than my distance to the nearest ally?
• Is my distance to the nearest enemy greater than or less than my distance to the nearest cover?
• Is my distance to the nearest enemy greater than or less than my distance to the nearest health station?
• ....

Finally, there are 100 possible decisions it can choose from, including:

• Flee from the enemy
• Slash enemy
• Shoot enemy
• Reload ammo
• Use special ability
• Defend
• Move to the nearest enemy
• Move to the nearest cover
• Move to the nearest health station
• Heal allies
• Heal myself
• …

Can you quickly figure out which decision is the best one in just a few seconds or minutes? If you can, congratulations, you’re among the top 1% of the smartest people in the world. Perhaps you don’t need Utility Intelligence, but if you’re feeling overwhelmed by too many issues around you, please consider using it. It would help you handle part of the workload, easing the burden on your brain.

For normal people like us, we have to replay our games again and again to debug why agents make wrong decisions, and it’s very time-consuming and boring. It could take us several hours, or even days to figure out why. I experienced it many times in the past. It’s tedious and I don’t want to experience it again.

How Utility AI addresses this¶

As I mentioned earlier, Utility AI allows us separate decision-making from decision-execution and turn them into two distinct processes. Therefore, we can run the decision-making process independently at editor time to preview which decision is chosen right in the Editor, without having to play your game.

So if the agent makes wrong decisions, you just need to log all the parameters involved in the decision-making process into the Console Window, then enter the values of those parameters into the Intelligence Editor to figure out the cause of the issue.

If decision-making and decision-execution were part of the same process, this wouldn’t be possible because decision-execution depends on the real-time state of the game world. For example, we cannot move the agent to its enemies at editor time, as the enemies are instantiated at runtime and do not exist at editor time.

Although Utility AI allows us to preview which decision is chosen at editor time, not all Utility AI frameworks support this feature. Fortunately, Utility Intelligence supports this feature, so you can preview which decision is chosen by modifying input values in the Intelligence Editor. I believe this feature will save you a lot of time. You can learn more about it here:

• Status Preview
• Input Tab
• Blackboard Tab

Conclusion¶

Above are the disadvantages of Behavior Trees and Finite State Machines that I found after years of working with them and why we should use Utility AI for creating game AIs instead. Utility AI is easier to debug, offers higher performance, and it helps your AI system adapt to constant changes in game design more easily and quickly, while remaining manageable and scalable as its complexity increases.

References¶

4. Building a Better Centaur: AI at Massive Scale
5. Improving AI Decision Modeling Through Utility Theory
6. Designing Utility AI with Curvature

If you enjoy my articles, please consider buying me a coffee on Ko-fi. Your support will give me more time to write new articles.
Thank you so much for your support. I love you all! 🥰