Mindflock.com

If you ask anyone with a passing interest in AI and more specifically in AI in games what is the most important aspect, you will probably be answered with something relating to logic. Logic and decision making seems like the most important aspect because it drives behaviour after all.

But in this post, I want to put the case that actually logic is really not a problem for games.

Of course, we rely on some system of decision making to exist, but what specific flavour it is doesn’t much matter. A good number of games have been created using simple finite state machines, although these days you would expect those to be heirarchical. A lot of more action oriented games use Behaviour Trees, which is the system we have implemented for StoryBricks and you can learn more about them here. Other approaches use goal based systems such as Goal Oriented Action Planning and Heirarchical Task Networks.

But no matter what structural system you choose to represent the problem, it essentially boils down to a simple statement:

if <condition> then <action>

Of course, the number of conditions and actions available can be very high and the conditional tests and the execution of actions can be quite complex. But we have this fundamental architecture of the above statement.

Ultimately, the structure we choose doesn’t matter, we are simply executing a program which controls flow of control. With more or less efficiency and ease of authoring.

So if we accept (any not everyone reading this will, granted) that action selection is not the core problem, then what is?

My assertion is that what is key AI especially for “believable agents” work like StoryBricks is animation and movement.

Imagine, if you will, the following scenario. You are walking across a field and come across another fellow. He is looking intently at the ground. He does nothing except look at the ground. So the question is, what is he thinking? Why is he looking at the ground? Is it:

• He  is interested in the composition of the ground
• He is afraid of you and is looking at the ground to avoid your gaze
• He is somehow compelled to look at the ground

Which of those statements is true? Unfortunately given so little context, you simply cannot know why he is looking at the ground.

What if I said “he is the queens geologist and has been tasked with finding gold for a new crown”, would that allow you to choose from the list?

What if I said “he is your sworn enemy and has no means of defending himself”?

The point is, that without context, it is impossible to understand the intent of the action. For all the complexity we could put into the <condition> part of a if <condition> then <action> statement, we have to be able to understand the <action> part in context in order to understand the <condition> part.

Thinking about traditional media

Now if you think about cause and effect in other media like books and film, it is pretty easy to understand the cause, because it can be explicitly expressed. Consider the following narrative:

“The gunfighter put the empty whisky glass down on the bar and stepped out of the saloon. He walked down the creaky wooden steps and out onto the hard dirt. Dust kicked up with each step as he walked, his spurs chiming on the ground with each footfall. When he got within pistol range of his opponent he stopped. For long minutes he stood silent and still, his only movement a slow flex of his hand above the gun in its holster. A bead of sweat drifted down the side of his face, he was at a disadvantage he knew, because the sun was in his face. But the wide brim of his hat sheltered his face from the harsh blinding rays of the noon day sun. He waited and watched, coiled like a spring, a deadly serpent ready to pounce at the slightest movement”.

(apologies for any real authors btw).

So, given that narrative, you can picture a scene. You understand there is a gunfighter and he has an opponent. You can start to evoke a sense of the scene and what it might be like using your own knowledge of that context, which has been learnt over time from other examples of this behaviour. What you can’t get from this narrative is exactly why he is having this gunfight.

If we continue:

“He had thought of this moment for a long time. His had played the scene out many times in his mind, ever since he had heard about how his wife and children had been brutally murdered. He had spent many years tracking the band of outlaws that were involved in the killing. One by one he had silenced them, with only this last foe remaining. It wasn’t “justice” he was seeking, even though he wore a badge that legitemised his handiwork. This was pure sweet revenge, the brutal slaying of his innocents met with equally brutal punishment.”

Adding that context to the narrative makes you understand the motives behind the actions. It is one of the most powerful aspects of books, in that it can go into great detail and really involve the reader in understanding the psychology of the actions.

Films have a different vocabulary, with a more visual way of telling the same tale. Consider this set of storyboards:

protagonist puts down his drink

And steps from the bar into the street

He sees his foe waiting for him

his spurs chime as they hit the hard packed dirt

He stands facing his enemy

The sun beats down hard

A bead of sweat runs down the gunmans face

He stands motionless, his hand ready at his gun

his mind drifts back to the day they came

his wife begging them for mercy

begging for mercy in the face of their guns

and receiving none

The placement of camera, use of lighting to set the mood, adding music and all the many different aspects that cinematography bring to the table to express the same notional scene still allow the same story to be told.

Now consider this scene in a game. By this I mean in an actual interactive scene, not in a cutscene. Hard to imagine isn’t it? When the player can look anywhere, you can’t rely on them seeing anything. If you place the player as the protagonist, you will most likely have a hail of gunfire within a few seconds. If you place them as a bystander watching the scene, the player will most likely stand in front of the protagonist or just generally “get in the way” of the scene.

The fundamental problem, is that scenes like this are really evocative in the traditional linear narrative media like books or film, but fail entirely in games unless they are controlled as they are in cutscenes. Interactivity simply breaks down too many notions that are evoked in scenes like this.

The language of interactivity

So if we accept that some forms of dramatic content are simply not useful for interactive storytelling, what do we put in its place? That is where StoryBricks comes in. The conceptual issue that StoryBricks addresses is the interactivity relationships between NPC’s, players and objects.

In the interactive medium, you are not simply watching a narrative unfold, you are the narrative. Your actions and interactions are the fundamental control scheme for the narrative of the world as you experience it.

Consider this. If you were the lawman/gunfighter from the scene above and had been going around the land taking part in pistol duels with the people who killed your family, what would it look like to an observer who was casually watching your actions? What if you actually roleplayed the gunfighter? Would the observer understand the narrative of the world as it unfolded and what kind of experience would they have had?

Going back to the case of the man looking at the ground for a moment. What I am saying is that there is an issue with the players understanding of the intention of the actions of an agent. So my suggestion is that the important part of game AI, especially for believability, is that. The understanding of the intent is the key, not the actual formulation of the intent.

Let me give you an anecdotal example. In this video Wil Wright talks about one bug they had in “The Sims” where NPC’s would randomly set on fire. The players of the game made up different explainations about why it was happening. Was it spontaneous combustion? Was it something they had done?

We joke in the game AI community that the best initial approach is a random one. But this is actually true, in that we can often convince players that actions have considerably more behind them than is actually the case.

I once saw a fantastic presentation by Baylor Wetzel that highlighted this issue. He took a number of different algorithms for choosing patterns of attack behaviour in a wizard duelling prototype he had written and asked his students to identify them. It turned out that we are actually pretty poor at judging the cause of actions when any reasonably complex pattern is used. In this case, random is almost as good as complex pattern because if the pattern is complex enough, players will have trouble discening the pattern to the point where is essentially *is* random to them.

So when we are trying to make players understand the actions we give our NPC’s, we have to be doubly sure that they are capable of understanding the motivations for them.

How do players understand characters?

The player’s reading of any given action is coloured by many things. One of the strongest being the actual visual look of the NPC. We had a team at the University do some work a few years ago where they looked at the interactions between a group of test players with a number of differently designed NPC’s. The theory was that empathy could be induced by character design and behaviour. It turns out that a large part of the empathy players had for characters was based on their appearance.

In traditional animation, this is actually formalized as a concept called “appeal”. The psychology of this is quite well understood. Most appealing characters have open faces with large eyes and round heads. Often animation will use childish or “cute” proportions. But aspects such as clothing,posture and gender will also contribute to the appeal.

The second most obvious and important factor is movement. When I say movement, I am talking about things such as posture, gaze, gesture etc. The fact is, we communicate a vast amount of our character non verbally. We communicate with our facial expressions, with our gestures, our gaze and our posture far more than we communicate with actual words.

The psychologist Albert Mehrabian suggested that we communicate a huge amount of our emotional intent non verbally. He spoke of a 7%-38%-55% split between verbal, vocal and visual language respectively. The suggestion being that we use more than just words to understand the true intent of an interaction. Which is confusing, given that most games rely on textual or verbal dialogue to communicate intent.

Think about RPG’s and how dialogue heavy they are. Think also about how often you simply click away these dialogue items. Simply “hoovering” the options to set some flag for later progress.

If we accept the notion that players understand characters intention through movement, then how can we improve the players understanding of the man looking at the ground?

Let’s consider each case in turn:

He  is interested in the composition of the ground

This one is relatively easy. You could have him walk around, looking at different areas of ground. Occasionally stooping to pick up a sample of dirt. You can use a prop that he brings out to look more closely at a sample. You could have him lift up stones on the floor and look under them. You could have him holding a book and comparing the sample to a page in the book.

He is afraid of you and is looking at the ground to avoid your gaze

Again, this one is relatively easy. You would have him use occasional glances up at you and quickly back down again if you are looking in his direction. You could alter his posture to be more wary and frightened. Perhaps supplicating himself with gestures and hand wringing. He would seek to avoid eye contact, but would definitely look up often to make sure you weren’t about to attack.

He is somehow compelled to look at the ground

This is the easiest one of all. You would fix the bug that made him look at the ground.

Conclusion

There is a great deal we don’t know about this whole conceptual area of interactive storytelling and drama. We intuitively feel that traditional media approaches to narrative break down fairly quickly, but as yet we haven’t built up a good enough knowledge of the principles we need to apply to the new medium of interactivity. We can definitely learn a lot from psychology and the principles involved in other media, like film, books and animations. But there is still a lot to explore.

StoryBricks as a concept, is a step towards exploring this new space. Of course the bricks themselves are just a small part of that space. We must investigate new methods of understanding the animation, behaviour and psychology of interaction.

Hopefully some of you reading this will come along with us as we explore this space together. As we develop the toolset, we will learn a lot about our understanding of characters, our ability to create performances that are comprehensible to others and perhaps as importantly we will learn how to create deeper and more meaningful experiences because of what we learn.