This document is about: QUANTUM 2

SWITCH TO

QUANTUM 1 QUANTUM 2 QUANTUM 3

Goal Oriented Action Planning (Beta)

Available in the Gaming Circle and Industries Circle

1. Introduction

The GOAP quantum code and visual editor are both in Beta stage.

1.1 What is GOAP

Goal-oriented action planning (GOAP) is an AI technique that consists of two parts:

Selection of most suitable goal from a specified set of goals given current situation.
Goals (also known as desires) could be: Defeat a target, Heal, Steal enemy flag, Flee
from enemy
Finding a set of actions (plan) from a specified actions pool that fulfill this goal.
Actions could be: Find object, Find target, Pickup object (e.g. gun, enemy flag, health
pack), Go to position, Go to combat range, Shoot

1.2 Comparison to other BotSDK AI approaches

As opposed to more standard AI approaches like FSM, HFSM and Behavior Trees, there is no
exact AI behavior being created by the designer. An AI agent is told what desires it has and
what it can do and the exact steps are put together (planned) in runtime based on the current
situation (world state).

Strengths

Handles well complexity
Adding new goals and actions that would automatically work with previous AI behavior is
straightforward
Can speed up development time for larger projects
When a large enough set of actions and goals is created it can be reused for many AI
agents with different behaviors
Natural (fuzzy) behavior
When AI behavior is nontrivial it is more difficult for players to spot some patterns if not
desired. As with any AI that calculates some utilities/relevancies/costs, there are many
values being considered that AI will likely act a little differently in similar situations.
Unique solutions

With a large number of possible actions, AI has many possibilities on how to solve some
problems that it can find a unique solution - this behaviour could be both good and bad.
On the other hand, having too many actions can impact planning performance, so it
needs to be approached carefully.

Weaknesses

Planning part (process of finding a suitable set of actions) needs to evaluate a lot of
possibilities thus GOAP can be slower in general than more lightweight AI solutions (e.g.
FSM)
More complex to master
When designing GOAP AI, users need to figure out how to simplify a complex world into
WorldState values and then always think about how GOAP actions will affect this
WorldState rather than specifying the behaviour directly.
As AI is not told specifically what to do it is cumbersome to do “scripted” AI behavior
such as boss fights

In order to best use GOAP strengths, the actions and goals need to be carefully designed.
GOAP actions should be atomic - meaning more complex actions should be split into multiple
actions if and only if such newly created actions could act in different situations with different
actions around. An example could be some complex KillEnemyNest action that could be split
into FindNest , GoToCombatRange and Attack actions.If such an approach is not possible or
desirable, it is probably better to use a more standard AI solution right away.

2. Quantum GOAP Coding

2.1 Implementation

Let’s first analyze the GOAP from the code perspective, mainly to understand how it works
internally, with just a few image examples. Then, we will follow up by analyzing it directly on the
Unity Editor.

The core of GOAP is located in the GOAPManager. Manager(in GOAP’s context referred to as
planner ) chooses the best goal, builds a plan (sequence of actions) that fulfill that selected goal
and executes all actions from the plan if possible.

Search for possible sequence of actions is done through backward A* with heuristic being world
state difference (how many world state flags are different from the target state).

BotSDK GOAP is very fast and lightweight but can also be further expanded based on specific
game needs.

2.2 Creating and Updating a GOAP Agent

Here is a step by step for what you need to do to create and update your own agents:

Add the GOAPAgent component to your entity. This can be done directly on Entity
Prototypes on Unity, but it can also be done directly from within the quantum code by
using the frame API;

Find the desired GOAPRoot asset and use it on the initialization method:

var goapRoot = frame.FindAsset<GOAPRoot>(playerData.GOAPRoot.Id);
GOAPManager.Initialize(frame, entity, goapRoot);

Call the Update method, passing the agent entity as a parameter:

C#
```
GOAPManager.Update(frame, filter.Entity, frame.DeltaTime);
```
When needed, manually set the Agent’s current state with:

C#
```
goapAgent->CurrentState.SetFlag();
```

2.3 GOAP State

GOAP state represents the state of the game that is relevant for the planner. Each AI agent
has its own GOAP state. In our implementation the state is represented as Positive and
Negative bitmasks. That means that each world state flag can have three values: true, false and
not defined.

Choosing a correct set of flags is very important for good GOAP AI design. Universal rule is that
a flag should be present in the world state if and only if the flag can change during
planning (= is used as a Condition or an Effect of some GOAP Action).

E.g.: There is no point having the IsHungry flag in the world state if there is no GOAP action that
would have IsHungry as a condition or an effect even though other game code uses it. In that
case it is better to place the IsHungry flag in your custom agent’s data.

To add flags to the world state you have to edit EWorldState enum in the GOAPState.qtn file.

It is possible to expand the world state with custom data.

2.4 GOAP Goal

Target State
State that the planner tries to get to during the planning process.

Start State
Usually left blank. When the planner is starting the planning process it takes the current agent’s
state and merges it with the goal’s Start State. This merged state then acts as FROM state and
Target State acts as TO state.

Note: Usually used to clear some flag from the agent's state to ensure some specific action will
be planned.

Validation
If a goal is not valid, it will not be considered during the goal selection process.

Relevancy
Says how relevant this goal is. Goal with the largest relevancy will be selected by the planner.
When two valid goals have the same relevancy, the one that is first in the list of goals will be
selected.

Disable Time
How long it takes before the goal could be reused.

Note: Disable time is often used when we do not want an AI agent to execute the same goal
multiple times in a row even though it is considered as the best goal for the situation (e.g. we
might not want an enemy to throw five grenades in a row). Also it is important in situations when
a plan cannot be found (some actions are not valid in the current situation) and different goals
need to be selected.

IsFinished
When the goal is satisfied, the planner deactivates this goal and starts searching for a new goal.
Goals are satisfied when either IsFinished is true or when the agent's GOAP state contains the
goal's Target State.

Interruption Behaviour
The execution of an active goal (plan) can be interrupted. If interruption is enabled, the planner
periodically checks if some more relevant goal exists. If so, then execution of the current plan is
stopped and a new goal gets planned.

● Always - Always check whether more relevant goal exists
● Never - Plan is never interrupted
● Based On Actions - Active action controls if the plancan be interrupted

OnInitPlanning
Actions that are executed before the planning process begins.

OnActivate
Actions that are executed when a plan is successfully found and the plan execution starts.

OnDeactivate
Actions that are executed when the goal gets deactivated.

2.5 GOAP Action

Conditions
What state this action needs in order to be considered in the plan.

Effects
What effects does this action have on the agent’s state.

Validation
Actions that are not valid are not considered during the planning process.

Note: This way you could “disable” certain actions in certain situations without actually
complicating world state with unnecessary flags. As mentioned in the world state section, world
state should contain only those properties that can change during the planning process itself.

Cost
Determines the cost of the action. The planner tries to find a sequence of actions that has the
lowest total cost. Cost of an action must be greater than zero. Usually it is best to think about
the cost as a time in seconds the action will take but keep some reasonable range (e.g. 0.5 to 5
with most actions being around 1). Too large cost differences can hurt GOAP performance. See
Optimization section for more details.

IsDone
If an action is completed, the planner continues with the next action in the plan. Action is
completed if IsDone is true or automatically when all Effects are present in the agent’s state.

IsFailed
When action fails, the whole plan fails and the search for the new best goal starts again.

Interruptible
Whether the action can be interrupted. Current active goal needs to have Interruption Behaviour
set to Based On Actions in order to this setting be considered.

OnActivate
AI actions executed when this GOAP action gets activated.

OnUpdate
AI actions executed when this GOAP action is updated.

OnDeactivate
AI actions executed when this GOAP action is deactivated.

Plan State Validation
Plan state validation node is used for advanced GOAP capabilities. It basically means that it will
validate the action during the planning based on the current planning state. It also has the
possibility to branch the plan.

3. Editor

3.1 Creating a new GOAP

On the editor's top bar, click on the (+) button, then select the alternative GOAP.

You will then be prompted to save the HFSM file. Save it wherever you prefer. This will create a
data asset used to persist the work that you have done on the visual editor:

Note: The name that you choose now will be the name of another data asset generated further
when you compile what you have done on the visual editor. That will be the data asset used to
actually drive your Bots, so you can already choose a suggestive name.

When you save the file, the main Bot SDK window will show the most basic format of a GOAP
document, in which you will see two containers: one named GOAP Goals and the other named
GOAP Actions.

3.2 Creating new GOAP Goals

To create a new Goal, right click on any empty space on the Bot SDK window and select a
GOAPDefaultGoal. In more advanced settings, it is possible for you to create your own goals,
which will also be listed on the right click menu.

On the root level, a non-modified goal node looks like this:

It is also possible to double click on any goal node in order to see it’s details. The default goal
provides you lots of fields that you will have to fulfill in order to express what you want for your
AI.

Two core concepts of every goal are the “Start State” and the “Target State”, which are Enums
of the type EWorldState that represents the current world state, given a specific GOAP Agent
(so this is not a shared/global state). Those states are used in the planning process in order to
select the goals that will change the game state in a way that fulfills the agent’s objective.

It is possible to express a world state flag as Positive or Negative. During the planning process,
the planner will filter the Goals that properly fulfills the desired world state. Let’s exemplify this a
little bit with an example.

Note: Find more details on the definition of the World State on the Quantum GOAP Coding
section.

3.2.1 Setting the Start/Target States:

Click on the Start State or Target State buttons to define what are the flags that must
be contained by the world state during the planning process, either as negative or
positive;
After selecting the desired states, they will now be contained within the goal node:
By default, the world state will show up with a small green circle. This means that it is
expecting the world state flags to contain that world state flag and that it should be
Positive. It is then possible to click on the small green circle in order to change it to
Negative. The circle will then be colored in red:
In the sample above, it is expected that the world state flags inform that the agent is in
melee range , but is not on recovery , in order for this Goal to be included in the
planning process (alongside with other goals). But it will only be included if the Target
State’s result will contribute to the Agent’s current objective. In the example above, the
result of choosing that Goal is that the target shall be defeated;
It is also possible to see and change the world state flags from the root level. You can
go back to the root either by pressing “ESC”, or by clicking on the button “Root”, on the
breadcrumbs at the top bar. The root level goals are always shown in a summarized:

3.2.2 Including Goals and organizing them

For the Goal nodes to be included in the planning process, it is necessary to include them on
the GOAP Goals container (the gray area).
In order to achieve this, just drag and drop your goal nodes inside the container:

It is also possible to rename goal nodes. To do that, select the node and press F2, or right-click
on the node and choose the alternative “Rename”:

One important thing to notice here is that the node's order matters. Topmost nodes will have
higher priority in comparison to the others. The node positioning is the tiebreaker if the planner
decides that more than one goal with the same relevancy can be used in the same way to
achieve the agent’s goal. It is a good practice to rename the goal nodes and to reorder them
properly:

3.3 Creating new GOAP Actions

In terms of creating, editing and organizing nodes, the same concepts above apply to the GOAP
Actions. Here is a simple example of some actions included on the Actions container:

3.4 Ignoring and Muting Goals/Actions

It is possible to take goals and actions away from the planning process without the need to
delete the nodes. This can be useful when debugging and trying alternative routes, without
losing your nodes data, in case you decide to add them back again.
There are two main ways for doing that:

To ignore a node , just remove it from the container and add it back if it should not be
ignored anymore;
To mute a node , right click on it and select the alternative “Mute”. The node will then be
a little bit transparent, to express that it is muted. It is useful to mute nodes because you
do not lose it’s position/order on the container.

Ignored and muted nodes are skipped during the compilation process, so don’t forget to
re-compile when doing that.

3.5 Setting slot values

When editing GOAP Goals and GOAP Actions (after double clicking on them), you will be able
to edit many slots values. Some of them can be hardcoded values, or they can be from other
nodes, and some other slots, such as the Action slots, can only be linked to one or more Action
nodes.

In this section, we explore a few alternatives on how you can define such values.

3.5.1 Primitive Types

To start exemplifying, the GOAP Goals have a few slots with type FP and Boolean. You can
define these slots values directly in the editor by clicking on them:

But besides hard coding values, it is also possible to link the slots with nodes which might either
contain a predefined value, or a predefined logic that will run upon polling that slot value. There
are many ways for doing so and all of them are explained on this topic:

Shared Concepts.

3.5.2 Action slots

Both GOAP Goals and GOAP Actions come with some Action slots on it’s right side. These are
the micro actions which will be implemented by the users, that will actually change the game

state. These actions can be used to chase a target, to perform an attack, to collect an item and
so on.
Important: please, do not confound “GOAP Action” with“Action”. These are two different
concepts.

The Actions are a common topic as they are also re-used by Bot SDK’s HFSM and UT. Find
more information on how to use actions in this topic:

Coding Actions

3.5 Compiling Your GOAP

In order to actually use the GOAP that you created, you have to compile what you have done so
you can get the Quantum asset which is used by the Quantum simulation.
To compile, you have two options:

The left side button is used to compile only the currently opened document;
The right side button is used to compile every AI document that you have on your
project.

Your GOAP files will be located at: Assets/Resources/DB/CircuitExport/GOAP_Assets.

3.6 Setting the AI to be used by your Bots

To use the compiled asset, you need to reference it from within the Quantum code. You can do it
by loading the asset based on the GUID, or you can just create AssetRefGOAPRoot to point to

the desired AI asset. It can be contained within some Component, some data asset,
RuntimeConfig, RuntimePlayer, whatever better fits your code architecture:

PS: find more details on the Quantum GOAP Coding section.

4. Optimizations and debugging

4.1 Debugging

BotSDK comes with its own debugging tool, but the GOAP editor currently does not make use
of it. Adding the debugger is one of the next upcoming features for the GOAP editor.

For the time being, you can use GOAP logging functionality. Simply assign an entity into
GameManager.DebugEntity and debug messages will be printed into the console.

4.2 Optimizations

GOAP optimization usually comes down to limiting number of nodes the A* needs to process.
To check the numbers and time it took to process, enable entity debugging (see above) and
check the log messages:

Note: The search time values are usually much larger in Unity editor than in actual build of the
game. But it is a good reference when trying to improve performance.

Generally speaking you want to keep the number of visited nodes low, but exact number is
dependent on the complexity of the AI. If the number of visited nodes is too high (e.g. over 100
for simple AI), chances are that the actions are incorrectly designed. Usual issue is that
conditions and effects of action’s are too vague so the planner is considering a lot of options but
most of them do not end up valid. Another thing to keep an eye on is actions Cost. A* uses cost
to determine what is the best action sequence during the planning process. So having
incorrectly set up costs of actions could cause planner to hit a lot of dead ends before
returning to the correct branch. Best is to imagine cost as a time in seconds the action will take
but keep it in some range (e.g. 0.5 - 5, with most actions being around 1) even though action will
realistically take less or more just to not have vast differences of costs between actions.

5 Extra Details

5.1 AIParam

Find here more information on using the AIParam, which is useful if you want to have more
flexible fields that can be defined in different ways: settings by hand or from
Blackboard/Constant/Config Nodes: AIParam.

5.2 AIContext

Find here more information on how to pass agent-contextual information as parameter: AIContext.

5.3 BotSDKSystem

There is a class which is used to automate some processes such as deallocating Blackboard
memory. Find here more information about it: BotSDKSystem.

5.4 Visual Editor Comments

Find here more information on how to create comments on the Visual Editor: Visual Editor Comments.

5.5 Changing The Compilation Export Folder

By default, assets generated by Bot SDK's compilation will be placed into the folder
Assets/Resources/DB/CircuitExport. See here how you can change the export folder: Changing the export folder.