Mindflock.com
Social Game Artificial Intelligence
Social greetings and proxemics
Social greetings like the kind you get when a new person approaches a group is what interests me right now. The whole “social greeting” aspect of groups can be quite varied both within and between cultures. In some cultures, it is quite common to kiss as a form of greeting. In others that would imply a level of intimacy that is quite uncommonly displayed. The anthropologist Dr E.T. Hall (who sadly died in 2009) called the study of this phenomenon “proxemics”. To quote his definition proxemics is: “The study of man’s transactions as he perceives and uses intimate, personal, social and public space in various settings while following out of awareness dictates of cultural paradigms.”
You can read more about Hall here
The relationship of proxemics and other non verbal cue’s such as kinesics, paralanguage and others is where my current focus lies. My theory being that without a deeper understanding of the human methods of non-verbal communication we simply cannot engender our virtual characters with a fully developed “human-like” set of behaviours.
One of the biggest issues, is that we are constrained by so many factors when creating games. It is almost impossible to imagine that fully capable non-verbal communication is ever going to be something that we will achieve with a hand-crafted approach, like many other aspects of characters are today. The industry as a whole has taken to using motion capture data to drive character animation because it is as close to human-like as it can be (although I have spoken to many animators who decry the use of motion capture data). But I feel that only via modelling and run-time generation of aspects like gesture, posture, gaze, proxemics, kinesics and paralanguage are we truly going to make our characters feel believable.
One current technical problem which must be overcome is that of inter-agent touch. Without contact based communication we will never make our agents feel alive. So the big question is how to achieve that. I see some potential for the inverse kinematic approach that was used in Assasins Creed, although I can’t find the video on youtube right now that shows some issues with the crowd pushing IK.
Human touch is a key aspect of communication and yet we hardly ever see it displayed in games outside of cutscenes which denote intimacy. The big problem is weirdly one of physics, in that currently it is very hard to get agents to actually contact each other without getting inter-penetration of limbs. Basically the issue is that humans in games are visually represented as meshes of increasing complexity, however their physical counterparts in the simulation are actually represented commonly as sets of linked bodies. Typically these bodies are simple volumes, often spheres or capsules. Clearly this representation is a vast simplification of the overall form of a human, but it is required in order for the physical simulation to be able to perform at an interactive rate.
So touch is definitely an area that needs considerable work. However right now my main concern is proximity as I feel that it is proximity (and posture, gaze etc) that leads to the relationships required for touch to be applicable. Proximity is the first part of any human interaction, in that in order to have real human communication (not mediated via some other system) we are required to be in proximity to each other. Hall actually determined that there were different aspects of proximity and different affective relationships amongst them. He also noted that there were differences between cultures and between varying relationships (same sex, different sex, different stages of intimacy etc). Issues like how far apart we are, wether we are actually facing each other or not. Unfortunately I don’t feel games are really very good at this yet. Often games have characters looking directly at each other, with an unflinching gaze and with a posture that seems aggressive. It will take some time for the industry to start considering these aspects to be important, but as we see more work developed in this area I feel we will slowly work things out. Much like facial and body animation has improved, I see the other elements of non-verbal communication improving.
The real question is can we actually use this to make more convincing and compelling characters and does that add significantly to the experience of the game?
January 6, 2010 - 9:42 am
You bring up many good points on how, as of right now, in the gaming industry, physical contact animations are something that is definitely not the easiest to do, and as well as the fact that we may not be able to create more 'human-like' characters without a deeper understanding of the human psyche. Every little thing, be it facial expressions, hand gestures, etc. has an impact on the ways we interpret reactions and ant the motives of other people. For example, if someone is laughing and the facial expression is there, but, the movement of the shoulders is not, it would seem incorrect and be labeled very poor animation and possibly inhuman. This takes into consideration posture and gaze as well. If something doesn't quite match what we consider it to be, it won't give the proper feel.
As you stated in this post, gaze and posture would be the most reasonable, and logical place to start first. Posture would give added insight into the agents underlying feelings, and a gaze with occasional glances directed away from another agent or character would portray nervousness, and very possibly worry and distress.
All of these factors, would indeed, in my opinion, make much more convincing and compelling characters would make a much more significant experience to the game, and potentially make much more personal experiences with the player.
January 9, 2010 - 6:30 am
It's funny that you mention that – the thing that surprises me constantly in France is “la bisse”. In the South it's 3 times, alternating cheeks, and is done to women (even if it's the first time you've met), family, and really good friends. If not you *always* shake hands with men. On the other hand in Australia you'd probably just say “hi”, with hand shaking much more of a formal, business-type gesture – it's a much less physical civilisation!
As for what you might call “procedural emotion”, I'm not convinced that most game programmer understand *real* social interaction anyway – it's not like you study sociology when you're doing IT, especially as the emphasis at school and universities is becoming more and more about *training* for a specific job, and less and less about *education*.
January 9, 2010 - 6:37 am
I agree that most programmers (and indeed some artists and animators) do not understand these things. I'm a relative newcomer to this as well (only been looking at it for a few years). It is definitely a complex thing to understand. But I think that there are enough patterns that we can begin to generate models of such behaviour for use in games.
Reading some fascinating books/papers right now that I'll try and summarize soon.
February 6, 2010 - 2:23 am
An important point that we have to remember when modeling this kind of interaction is finding a way to add these kinds of interaction into the unevenly paced dialogue sequencing of video games. In my opinion, the greatest loss of immersion comes from the fact that while the player is trying to decide what response to give, the characters are simply standing, continuing the same action throughout.
Even if touch, proximity, and gaze create a much more life-like scene, we need to conquer this boundary to make interaction truly realistic. In real life, people constantly switch gazes, shift their weight, and make body gestures through a conversation. But problems begin to arise when the conversation is stopped, as there is no longer any stimulation to dictate what actions the character must take, resulting in a very stale experience.
I'm mainly referring to games such as Mass Effect, Fallout 3, and other RPG's where NPC/Player interaction is such a large focus of the game. For action games, where conversations are created to liven up the scenery, and don't require player input, the above isn't a concern.
Hopefully that makes sense, it's 2:30 AM over here and my brain is starting to lapse.
February 6, 2010 - 6:46 am
You're completely right. The level of subtle non verbal communication during conversation and other similar interactions is huge. But we have to start taking these things into consideration. I think you hit on a key problem, in that for the most part we expect to have a two way discourse with language and NVC elements, yet we simply cannot model the verbal let alone the non verbal accurately.
My own viewpoint is that we would be better served trying to frame non verbal communication for non verbal interactions as the first step. So finding interactions that do not involve language and trying to simulate those as a means towards a fully verbal and non-verbal interaction in the longer term.
February 6, 2010 - 7:23 am
An important point that we have to remember when modeling this kind of interaction is finding a way to add these kinds of interaction into the unevenly paced dialogue sequencing of video games. In my opinion, the greatest loss of immersion comes from the fact that while the player is trying to decide what response to give, the characters are simply standing, continuing the same action throughout.
Even if touch, proximity, and gaze create a much more life-like scene, we need to conquer this boundary to make interaction truly realistic. In real life, people constantly switch gazes, shift their weight, and make body gestures through a conversation. But problems begin to arise when the conversation is stopped, as there is no longer any stimulation to dictate what actions the character must take, resulting in a very stale experience.
I'm mainly referring to games such as Mass Effect, Fallout 3, and other RPG's where NPC/Player interaction is such a large focus of the game. For action games, where conversations are created to liven up the scenery, and don't require player input, the above isn't a concern.
Hopefully that makes sense, it's 2:30 AM over here and my brain is starting to lapse.
February 6, 2010 - 11:46 am
You're completely right. The level of subtle non verbal communication during conversation and other similar interactions is huge. But we have to start taking these things into consideration. I think you hit on a key problem, in that for the most part we expect to have a two way discourse with language and NVC elements, yet we simply cannot model the verbal let alone the non verbal accurately.
My own viewpoint is that we would be better served trying to frame non verbal communication for non verbal interactions as the first step. So finding interactions that do not involve language and trying to simulate those as a means towards a fully verbal and non-verbal interaction in the longer term.