Mindflock.com

Social greetings like the kind you get when a new person approaches a group is what interests me right now. The whole “social greeting” aspect of groups can be quite varied both within and between cultures. In some cultures, it is quite common to kiss as a form of greeting. In others that would imply a level of intimacy that is quite uncommonly displayed. The anthropologist Dr E.T. Hall (who sadly died in 2009) called the study of this phenomenon “proxemics”. To quote his definition proxemics is: “The study of man’s transactions as he perceives and uses intimate, personal, social and public space in various settings while following out of awareness dictates of cultural paradigms.”

You can read more about Hall here

The relationship of proxemics and other non verbal cue’s such as kinesics, paralanguage and others is where my current focus lies. My theory being that without a deeper understanding of the human methods of non-verbal communication we simply cannot engender our virtual characters with a fully developed “human-like” set of behaviours.

One of the biggest issues, is that we are constrained by so many factors when creating games. It is almost impossible to imagine that fully capable non-verbal communication is ever going to be something that we will achieve with a hand-crafted approach, like many other aspects of characters are today. The industry as a whole has taken to using motion capture data to drive character animation because it is as close to human-like as it can be (although I have spoken to many animators who decry the use of motion capture data). But I feel that only via modelling and run-time generation of aspects like gesture, posture, gaze, proxemics, kinesics and paralanguage are we truly going to make our characters feel believable.

One current technical problem which must be overcome is that of inter-agent touch. Without contact based communication we will never make our agents feel alive. So the big question is how to achieve that. I see some potential for the inverse kinematic approach that was used in Assasins Creed, although I can’t find the video on youtube right now that shows some issues with the crowd pushing IK.

Human touch is a key aspect of communication and yet we hardly ever see it displayed in games outside of cutscenes which denote intimacy. The big problem is weirdly one of physics, in that currently it is very hard to get agents to actually contact each other without getting inter-penetration of limbs. Basically the issue is that humans in games are visually represented as meshes of increasing complexity, however their physical counterparts in the simulation are actually represented commonly as sets of linked bodies. Typically these bodies are simple volumes, often spheres or capsules. Clearly this representation is a vast simplification of the overall form of a human, but it is required in order for the physical simulation to be able to perform at an interactive rate.

So touch is definitely an area that needs considerable work. However right now my main concern is proximity as I feel that it is proximity (and posture, gaze etc) that leads to the relationships required for touch to be applicable. Proximity is the first part of any human interaction, in that in order to have real human communication (not mediated via some other system) we are required to be in proximity to each other. Hall actually determined that there were different aspects of proximity and different affective relationships amongst them. He also noted that there were differences between cultures and between varying relationships (same sex, different sex, different stages of intimacy etc). Issues like how far apart we are, wether we are actually facing each other or not. Unfortunately I don’t feel games are really very good at this yet. Often games have characters looking directly at each other, with an unflinching gaze and with a posture that seems aggressive. It will take some time for the industry to start considering these aspects to be important, but as we see more work developed in this area I feel we will slowly work things out. Much like facial and body animation has improved, I see the other elements of non-verbal communication improving.

The real question is can we actually use this to make more convincing and compelling characters and does that add significantly to the experience of the game?

I was walking through the center of Bolton this morning thinking about a video I’d seen of some interruptible behaviours from a paper at AIIDE. One thing I have seen a lot of while working at the University is the type of social greetings that different cultures use when meeting each other. One thing that struck me when comparing the video (it was a movie of Neverwinter Nights doing some multi-queued behaviours) was that the game completely and utterly fails to capture the social greetings we humans use.

In a typical game, you have an agent approach another agent and what you typically see is the second agent swivel on the spot to face towards its newly percieved interacting agent. But this completely fails to capture even the most basic of human behaviours. The hug!

Of course, there are good reasons why games fail to implement things like hugging, or even hand shaking. Mostly involved with the cost of actually capturing the motions and applying them to multiple agents. But a second issue is that it can actually be very hard to synchronize multiple agents with enough positional accuracy that they can physically touch each other without causing interpenetration of the various limbs. Adding on top the problem of having suitable animations available during any discourse between the characters you generally get a fairly poor representation of discussion. That is when there is even any inter-agent conversation. RPG’s are probably the major force for inter agent discussion scenarios and yet they still fail to capture much of the human behaviour involved in even rudimentary communication acts. First person games? Don’t even ask.

It gets even more complicated when you consider the effect of having a group of agents in this social situation (imagine an agent joining a group of half a dozen other agents, how would you approach this task if you were trying to make them feel more human?). One thing I’ve been observing is that human beings typically never spin on the spot. Why is it that games think that spinning on the spot is acceptable? Another thing is that different agents attitudes to different group members can be quite diverse. I used to work with a guy called Jan Ruud (who lived up to his name quite often) who had a habit of standing between any women and the rest of the group. Clearly this was his attempt at securing the female attention for himself. Now Jan was quite a unique case in that he actually did this quite deliberately and without any qualms. But this sort of behaviour actually happens quite a lot in any social grouping. People react differently based on friendships and current moods. Try looking at any group of your friends and watch what happens when another friend joins the group. Does the original group reform its structure to allow the newcomer to integrate? Do some of them tend to face away from the newcomer? Do some of them face towards?

I’m trying to get permission to do some actual video recording of these situations. Because they are fascinating to try and recreate using AI techniques. Learning how to create agents that interact with each other in a more human manner should offer some insights into the formation and dispersion of social groups that might be useful to games. Certainly it has made me look at social interactions with new eyes.

So, when do you think we will see games with agents in them that actually go up and hug their virtual friends?

Feels like there are a lot of zombie themed games out there right now. But this isn’t a post about that trend, but rather about the issue of having a “train” of agents after a single object.

Let me outline the issue for you. Imagine you have N number of agents, all seeking a particular resource. Now the problem occurs when all of those N agents try and attain the same object at the exact same time. What you get is basically a “train” of agents all moving towards the given object. This gets even worse if the object is moving and is faster than the agents trying to obtain it. You get what old MMO players used to call a “train” of AI characters.

Above is a video of agents with a simple “find object in radius, if object within radius try and get to it” behavior. Notice it definitely looks like a train? In the case of zombies (which you are seeing here) it really isn’t that much of a problem as you expect zombies to be a bit dumb. But this really isn’t acceptable in the general case.

In real life, you would never see this because humans would spot that someone else is likely to get the particular object and would decide to therefore try and attain something else, or they would decide to compete for the resource in some way (perhaps by blocking the progress of the other agents?).

Clearly we need to have some logic to alleviate this problem.

Solition 1 – Use agent proximity

One answer is to do what humans do and simply consider all other agents and their relative proximity to the object and determine who is more likely to get it. But this would need to be computed per-agent for each other agent. These kind of complex calculations are relatively cheap to do on modern machines, but too many of them soon add up to major slowdown. Clearly as the number of agents increases the cost of calculating this sort of data quickly gets out of control.

Solution 2 – Use “smart objects”

Another answer, which seems more logical, is to have all agents express their wish to acquire the resource and have the resource itself choose who actually gets it. This is essentially a simplified version of the Sims “smart object” idea. The advantage here is that if you have less smart objects in a world than you do agents, you should get a significant cost saving on the calculations involved. The only agents allowed to request access to the smart object would be the ones that the object itself has detected within its proximity.

Another advantage of the smart object system is that the objects themselves can have more logic involved above the simple “am I available to be used” logic. It might be that some smart objects require a number of agents to be involved before the interaction is possible. This multi-agent synchronisation can get pretty complex, especially when there are potential failure states. Having a reactive system that can gracefully fail and release any agents tasked on the smart object is a requirement for this kind of system to work.

Solution 3 – Rely on variety

As the problem is simply that agent’s are all trying to acquire the same object, another obvious answer to this is to simply have different objects and try and make sure that agents choose from the available choices rather than all synchronizing on one specific object. The problem with relying on this notion is that eventually unless you artificially ensure that all objects within a world are maintained in equilibrium, you will inevitably achieve starvation of objects of a particular type. Imagine a case where you had a pack of lions who could only eat zebra’s. Eventually if the zebra population was not replaced, the lions would converge on whatever zebra’s were available in greater and greater numbers, eventually leading to complete extinction of the zebra population. So I guess your thinking “why is it that zebra’s don’t die out then?”. That’s a good question. One possible reason that the zebra population doesn’t collapse may be that the population regeneration rate is faster than the predators can kill them off. Another reason might be that lion’s are a bit lazy and will take on any food source available. In essence they have more than zebra on their minds for food sources, so they spread their attention across multiple types of prey.

Using this principle, all we would need to do is to ensure that our predator agents simply choose from a variety of prey and ensure that enough prey types were available so that behaviour would never converge to cause extinction. This suggests that some system is providing a method of ecological overview, adding population to prey types that are being killed to replenish the stocks.

Solution 4 – Enforce varying selection

The issue can also be tackled by taking on the root cause of the potential problem. That being the selection of prey being narrowed to one species. If we artificially enforced varying selection by the predator by simply not allowing it to eat any prey at a higher frequency than any other, we would achieve the equilibrium we require in the system. However there is a downside to this solution, in that the behaviour of the predator is unlikely to feel very natural. In nature predator animals DO have a propensity to prey on a specific subset of other creatures. This might be due to past successes, or geographical availability.

So what is the answer?

Right now, I am considering the smart object route. Because it is a very flexible system in terms of multi-agent tasks. Although we might think of predator/prey situations, a smart object system is just as likely to facilitate a social situation (for example kids on a see-saw) as it is to facilitate a predator/prey scenario.