In-game speech animation is about to get a whole lot better

Brian Ambrozy 9 Feb, 2012 at 4:54pm ET Article published in Tech

Today a company named Speech Graphics sent out a press release touting their new lip-sync software for rending facial animations with audio. The goal is more realistic rendering of mouths while speaking. The video speaks for itself:

The goal here is significantly reduced cost for rendering of realistic speech in video game development. Speech Graphics used the recently released massive-budget title Star Wars: The Old Republic as an example:

The Speech Graphics dudes

In response to the rising quantity and cost of speech and dialogue in today’s story-driven games, Speech Graphics’ technology uses just audio input to automatically drive a 3D character to speak. Put into context, Bioware’s recently released Star Wars: The Old Republic, contains 260,000 lines of dialogue, all of which had to be animated—not just once but in three languages.

The Speech Graphics technology, as you can see from the video, is capable of rendering speech in any language. They have more samples on their website.

They will be showcasing the technology at GDC next month.

Comments

9 Feb 2012 ~ 5:17pm Canti Wow
9 Feb 2012 ~ 6:27pm csimon Amazing advancements since Bell Labs of the '80's.
9 Feb 2012 ~ 6:57pm Annes Oh boy.
9 Feb 2012 ~ 7:58pm BHHammy Now THAT is impressive.

If we can make L.A.-Noire-calibur facial animations far easier and cheaper to accomplish, it'll do wonders for the medium (at least as far as folks who take advantage of it properly).
10 Feb 2012 ~ 12:54am UPSWeezer
10 Feb 2012 ~ 8:24am MiracleManS I did notice that the faces are mostly stationary outside of the cheek/mouth. I wonder what kind of cost is incurred when you incorporate full facial expressions?

Needless to say, this is still amazing.
10 Feb 2012 ~ 11:14am Thrax This video is awesome without sound. It looks like a bunch of babbling, emotionless mannequins.
10 Feb 2012 ~ 11:24am Snarkasm The tech is awesome, but at the end of the article, what stuck with me was how much one of the guys from Speech Graphics looks like @mas0n, heh.
10 Feb 2012 ~ 12:40pm PirateNinja I wonder if they could work with linguistic experts and statisticians to develop databases that essentially mapped common facial expressions to common verbal expressions. For example .. you see something cute and say "awwww" .. your eyebrows lift and you make the "isn't that cute face"

If they could automate that and create an API for it along with their lower jaw/mouth/tongue animations then I think they would be in serious business here.
10 Feb 2012 ~ 1:12pm Tushon
I wonder if they could work with linguistic experts and statisticians to develop databases that essentially mapped common facial expressions to common verbal expressions. For example .. you see something cute and say "awwww" .. your eyebrows lift and you make the "isn't that cute face"

If they could automate that and create an API for it along with their lower jaw/mouth/tongue animations then I think they would be in serious business here.
The man you would be looking for is Dr. Paul Eckman and his creation, FACS. TL;DR: he figured out the emotional significance of every facial expression and mapped it to the specific muscles involved.
10 Feb 2012 ~ 1:49pm PirateNinja Great, I contacted them re: Dr. Eckman and our suggestion. Perhaps something comes of it.
10 Feb 2012 ~ 2:50pm Tushon We could say, "I did that" and get a mention in the credits!
10 Feb 2012 ~ 8:28pm Bandrik Mind. Blown. It's not Pixar, but it sure as hell would be better than 90% of the games I've seen so far that still have "sock puppet" mouths.
11 Feb 2012 ~ 12:58am Sledgehammer70 Just saying... EA has had tech like this for over 7 years.
11 Feb 2012 ~ 12:31pm chrisWhite Sledge is right, while perhaps one of the more impressive contenders this is nothing new.
12 Feb 2012 ~ 4:12am test test
13 Feb 2012 ~ 12:09pm PirateNinja Response:
http://www.speech-graphics.com/automating-nonverbal-facial-animation/
13 Feb 2012 ~ 12:27pm primesuspect If EA has had that tech for seven years, how come this looks way, way better than any game I've ever seen?
13 Feb 2012 ~ 1:01pm PirateNinja I also think there may be confusion about the key factor in this technology. It isn't just a "it looks so good" thing, it's the fact that the animation is generated by the audio recording itself and it still looks that good. There was no manual tuning or matching. I also question EA having anything like that.