Cinema Crunch EP 133: Motion Capture Stunts for Film and Video Games

Cinema Crunch EP 133: Motion Capture Stunts for Film and Video Games


(upbeat music) – Hey there and welcome to this
episode of “Cinema Crunch”. My name is Rose Donahue and we are filming at the Quantum Arc Media
studio in Las Vegas, Nevada. Technology has done so
much to impact filmmaking. Today we’ll be talking
about motion capture for both film and video
games with stuntman and cofounder of SuperAlloy
Interactive, Eric Jacobus. – Thanks for having me. – Thanks for being here, Eric. Super excited about it. So I’m gonna admit, and I kind
of said this to you already, you’re doing very technical
work with the motion capture, so this is new to me. So I’m excited, I’ve done some research, but I’m really looking
forward to hearing bit about your experience actually doing it. – [Eric] Okay. – Why don’t we start with a
little bit of history on you. – Sure, yeah.
– You’ve been doing stunts for a long time, you’ve been
studying stunts and violence. Can you tell me, tell
me how you got started. – I’ve been studying
violence, that is the truth. You know it started out
back when I was a kid in a small town in northern California and really wanted to make action films. I was a big fan of Hong Kong movies. Started out actually as a techie. I was a programmer, I did PHP
and Visual Basic programming when I was fifteen, and I made a website where I reviewed Hong Kong movies. Jackie Chan movies and things like that. Which were kind of fresh at the time, this was in 1997, 1998. And reviewed so many of
these movies I decided well why don’t I just
go try and make my own. ‘Cause I started seeing
how these action scenes were put together and I didn’t have much martial art experience
but I figured, you know, I’ll give it a shot and
I’ll start training. And, you know, if I ever
wanted to actually do this in real life, I knew I actually had to put content out there. – For sure, yeah. – Because, you know, we were
so far away from Hollywood and I was so inexperienced
that if I wanted a shot, I had to just make my own stuff. I wasn’t gonna go down to L.A.
and ace an audition, right? I was gonna be competing against people that have done martial
arts their entire lives. But I found out that I
could fall down pretty well. – Nice, that’s a good one.
– I could hit the pavement without pads pretty well. And so over the years we
did a bunch of short films. And I did some feature films. And then I did end up
working in Hollywood. Did some stunts in Hollywood. They don’t really like, and by the way when I was doing these
short films, as you know, when you’re making films
you learn to do everything. – Sure, oh yeah.
– You understand the writing. You might just wanna do acting, but you’re gonna learn
writing, producing, directing, you’re gonna learn the whole thing. And I learned camera and editing also. And how camera and editing affect action. And it was so critical that
camera and editing were right. You know, and that was
why I was such a fan of the Hong Kong films, because
they really nailed that. And I took that model and that’s what we did or short films with. But Hollywood has a different model. They have a very kind of,
almost like an assembly line. Where as a performer you go
in, they say do your stunt and they don’t really want you
to tell them how to shoot it. They’ve got that figured out. They’ve already storyboarded
it, it’s already done, right? So then I did some jobs doing
pre-vis and things like that where I could actually
use my filmmaking skills. But then I did this short
goofy YouTube series called “Tekken in Real Life”
where I copied video game moves and then Sony found it
and they hired me to do Kratos’s motion capture. – [Rose] Yes, mm-hmm.
– For “God of War”. – And I did the combat motion
capture for “God of War” as well as for a bunch of
other games after that. – What a wild ride for you. How, I mean it makes so much sense. That’s really the way to get into, the way to find opportunities today is to make your own content and just– – [Eric] Exactly.
– If you’re not doing it– – [Eric] Exactly.
– You’re not getting practice. – Exactly, it’s exactly the same with, like the open source
movement wouldn’t exist if people weren’t in their
basements just programming away and making stuff, right?
– [Rose] Yeah, yeah. – Not that education is bad,
but it didn’t start there. It starts with people wanting
to just make something. – Mm-hmm, mm-hmm. So it’s, and I’m interested
hearing that you started doing programming and tech stuff
’cause that’s so relevant to what you’re doing now. – Yeah, I didn’t see it comin’. – Yeah, right. (laughs) It all adds up, it’s
kind of a domino effect. – I mean it helped in
terms of making websites and actually getting our name out there. And, you know, your talking
about before YouTube back when you had 20
megabytes on a GeoCities page and you’re, and you know,
we managed, and I managed to get the software and edit
and make a website out of PHP and it was very easily searchable. And that helped us for a while. And then for a long time that
tech stuff kinda laid dormant. – Sure. – Then when we actually
made a company, SuperAlloy, – Yes. – We started actually utilizing
that technology again. And now, it kinda comes full circle now. – Totally.
– Almost 20 years later. – Yeah, so you’re using the Xsen? – We use Xsens. – [Rose] Xsens, okay.
– Xsens, yeah. – Okay, that’s the way to say it. – Yeah.
– Okay, awesome. – Yeah, and so, with, motion
capture is pretty simple. – Okay. – You put a suit on. – Which is the Xsens. For anyone who doesn’t know.
– Which is the Xsens. Right, that’s our suit.
– That’s the suit, okay. – And then other people use Vicon. Sometimes you’ll see behind
the scenes of Andy Serkis and these guys wearing
motion capture suits and they’ll have these
reflective balls on it. Those are called passive optical systems where there are a bunch of
cameras and they literally just look at you and they figure out what you look like in 3-D and then they record it into a computer. And then with Xsens it’s inertial. So there’s no camera. And so it’s just these
accelerometers that are in the suit. Moves around but it works the same way. So you move around, it
records it into computer and now you have computer
data on movement. – That’s awesome. Yeah. So with SuperAlloy Interactive, you are, you’re recording those things and then just sending that content off to– – [Eric] Pretty much.
– The editors, or? – Pretty much, well, we will
have a game company hire us to do any number of things
in a game or movie sometimes. But let’s say for
example a game, they say, “We want you to do a motion
capture for our in-game “characters and also we
want you to do previs “for our cinematics,” for example. And so I might help create the shot list, you know if we’re building
a navigation system for a character, or a combat system. Or they will have their idea. Whatever it is, we shoot with them. They do it over Skype a lot of the time. Sometimes they come here. Sometimes we take the system
to them ’cause it’s portable. So it’s very modular.
– [Rose] Yeah. – And the nice thing
about the Xsens system is that it can kinda fit
into a production workflow anywhere they need it to fit. So we’re very agile. – That’s great. – Yeah. And so we find that we’re able to get things done very quickly. – That’s great, yeah, that’s always good. – Plus, since we have,
since I’m a stuntman I can do almost any motion
capture that they need. – [Rose] Mm-hmm.
– I don’t need to go and cast stunt talent. – That helps, yeah. – But, you know, we’re
based in Vegas and I can get any physical talent that
exists under the sun. – (laughs) There’s a lot
of opportunities here. Yeah, absolutely.
– Yeah, so if I can’t do it, I mean, I’m not gonna learn
how to be a contortionist for a movie right now.
– Sure, sure. – I don’t have enough time. So I’ll just hire one. – Perfect.
– Yeah. – I was wondering, what’s the benefit of having someone in one of
these motion capture suits, whatever it may be, what’s
the benefit of doing that rather than having an
animator just animate someone? – Sure. There are costs and benefits. – Mm-hmm. – The benefit is that you
get very human-like movement. Takes less time. Of course there’s a
style to hand animation, and it’ll always be around, right? It’s like, claymation
will always be around. Painting will always be around. And then photography
made realistic painting in a sense, right, but people still paint. And so it’s a stylistic choice a lot of the time with game companies. – [Rose] Very cool.
– What they’re trying to do, typically when somebody
wants to do motion capture in a game it’s because
they’re trying to create a level of immersion that
looks realistic, right? And not fantastical, right. Even for sometimes with
fantasy they’ll still capture human movement, but the
thing is that in 3-D, when you capture human
movement you can kind of stretch things out and you
can do a standard punch but then they can kinda
stretch it out and make the arm really long and then and
then, ah, they can move the spine around, they can
really kind of mess with it. – Yeah. – And make it look a little
bit creature-like for example. – But if they’re trying to get
humanoid realistic movement they wanna touch it as little as possible. They don’t really wanna mess with it because the more that they touch it the more it starts getting
into that uncanny valley kinda area and so that’s typically why they go with motion capture. – Okay, very cool.
– You could do it with animals too, you can
do it with horses and dogs. It’s been done. – It’s pretty cool, yeah.
– Aww, a little dog suit! – Yeah, a little dog suit.
(Rose laughs) We’ll put the little markers on it. – That’s awesome. I think I did see a pit photo of that and it did look adorable. – That’s funny.
– Yeah. – Very cool. So yeah, so you guys pass it off, you let the editors do their thing and then you’re on to the next one. – That’s it, yeah. – Awesome.
– That’s it. And part of that process too,
is helping them figure out, you know they give us the moves,
but part of what we do also is help them understand,
or help them to decide on kind of like a style
of action for their game. And so, yeah, I’ll do the
movement, but there’s also going to be a style when
there’s interaction, right? Like when the main character
fights a villain in the game should it be hyper violent? Should it be funny?
– Mm. And so that’s what
we’ll also do is consult with the game developers and say, “What kind of game are you making? “Are you making a hack and slash? “Well maybe we should, “maybe the fights should
be short and sweet.” Right?
– [Rose] Sure. – It just depends on, and
that’s kind of the action code that we help them develop
and we figure that out. We figure out how to make
an action code based on a bunch of different criteria. – Okay, when you say action
code, that’s the style of fight? – It’s the style of movement. It’s the choreography. Sometimes it’s where they put the camera. For example in a cinematic
scene in the game, or for a movie, obviously. In editing too. Editing affects that code. – [Rose] Sure.
– The way that you edit a fight scene together. How much time the human eye has to focus on a piece of action, that determines how people receive that action. – Yes, oh my gosh. I’m so, the level of depth you have and knowledge you have about fighting and people perceiving fighting
is just fascinating. (laughs) – It’s because I had to do it for 20 years to make these short
films, I had no choice. – You had to learn it, yeah. – I had to learn it. – If you’re gonna be the best,
be the best kind of thing. – Yeah, and just, you have to
learn all the different styles and it was, you know it was a requirement that I learn how to edit this. And what you’re hearing
is 20 years of failure. – (laughs) Okay.
– Right? And 20 years, and some
successes obviously, but you know, doing a lot of this stuff and directors saying, “Yeah,
but that editing’s too slow, “I’m trying to get people confused.” People would say that
kinda stuff, you know? – [Rose] Mm-hmm. – Or, “This choreography is
too, it’s like it’s too G-rated. “It really needs to kinda send a message “that this guy’s dangerous,” right? – Mm.
– Things like that. So, you know, as an amateur
choreographer back in the day I would go in with this
kind of vision of how I wanted things to work, but I wasn’t taking into consideration
what the audience was and what the product was. – Yes. – And that stuff is what
determines what kinda code you’re going to use– – So key.
– In the game, yeah. – Yeah, I think that that happens for a lot of beginning filmmakers. – [Eric] Sure.
– Right? It’s about the art. – Narrow vision, yeah,
– [Rose] (laughing) exactly. – But it has to serve the audience. – Totally. – I mean if it’s not, then you’re just, why make it? – Yeah.
– You know? – Yeah, and we had talked a little bit about the anthropology
of violence as well. – Yeah.
– So how have you used that study to inform your filmmaking? – Well if you think of
choreography as planned movement, then action choreography
would be planned violence. And my motto is that if we
wanna understand choreography we really need to understand violence. And that can be really
uncomfortable for people. But, you know, we’re adults here. And what we will do is, like my study goes into the neuroscience and the anthropology of
how humans not only do, but perceive violence, right? And what is it about
humans that is particular in that we have wars, civil
wars, wars blood feuds, we murder, we use tools, we use weapons and no animal does this. Right? And the short answer is that humans have a mirror neuron
system that is far more complex than other animals’ mirror neuron systems, and essentially the way the
mirror neuron system works is that when I grab a banana for example, a neuron fires in my brain. When I see somebody else grab a banana, that same neuron fires in my brain. So what my brain is doing
is creating a simulation of what I’m seeing other people doing. – Okay. – And I might act on that, I might not. If I see violence I might act
on that, I might not, right? What I’m doing a lot of the time, so if somebody’s gonna reach
their hand out and do this, then I can read their intention
and that’s what everybody– – About to grab something.
– Yeah, so what we’re actually doing is we’re
creating a simulation of the intentions of people around us. That’s what humans do from birth. That’s how we learn.
– All the time, yeah. That’s how we learn. That’s how we learn language, that’s how we learn social cues. And some people have deficiencies in this and they have a hard time
picking up social cues. Some people are inundated
by it and they get too much social cues and they
have to put headphones on. I think autism has this problem sometimes where they’re actually
over-inundated with social cues. So with that the thing about
human violence then is that if we’re in a fight, not
that we ever would be– – (laughs) For example.
– But if we were to get into a fight, right?
– Yes. – And you made me angry,
and I went like this, and I reached behind my
back, you immediately create a whole list of intentions potentially– – [Rose] Ooh.
– Of what I could be doing. – Whatever you could have back there. – Right. So if you’re expecting
a worst case scenario, like I have a gun, whatever
it is, you might then go for your gun first and kill me.
– [Rose] Right. – That’s how human violence escalates. – [Rose] Ah. – Animal violence
doesn’t have this problem because animals know exactly
what weapons are at steak. There are no tools.
– No weapons. No, yeah. – Yeah. You have their claws, teeth–
– No thumbs, yeah. (laughs) – Claws and teeth, right?
– Right. – So animals don’t fight
to the death very much. Not within their own clan that’s for sure. – Sure.
– They’ll kill others but they won’t rival to the death. And so that kind of
propensity of human violence to escalate to the extremes,
Rene Girard calls it escalation to the extremes.
– [Rose] Mm-hmm. And this is where I get
all of my information from about this is from him. And the ancient society
realized that this is such a contagious problem–
– Mm-hmm. – Where if you and I
are escalating violence and people are also reading that violence and I kill you and then
somebody avenges you– – Right, right. – This is particularly human.
– It’s ongoing, ongoing. – It’s particularly human. – Okay. – Other animals don’t
really do this stuff. They don’t have blood feuds. They don’t have civil wars, but we do. So human society created all
these interesting rituals in order to clamp down on
this stuff, ranging from, “Okay, stop, let’s set you up in a ring, “dual to the death, go.” – Okay. – And the winner–
– At least it’s in a ring. – Then it’s in a ring.
– Yeah. – And the winner is the winner and nobody can avenge the loser. Right?
– Got it. – That is what a dual to the death was. So that would be, you
know, samurai, gunslingers? They have that kind of
code with them, right? And then, but that still causes blood. Blood is a sign of violent contagion. If you wanna try to cut down on the blood you do sports, so maybe a fist fight. – [Rose] Sure.
– Put gloves on, now it’s a little bit safer. A little bit less blood, then
you can get into combat sports and then finally if you
really wanna make it totally uncontagious,
you do sort of ritualized choreography which would
be like a warrior dance or, you know, traditional Kung Fu. – Yeah. So you’ve done all this research. – [Eric] Yeah.
– And all of those are factors into how you decide to execute– – Yeah, and so all those different methods of controlling violence and
of perceiving violence too, because people are always
going to be party to this. – Mm-hmm. – Those all represent film genres. – [Rose] Ah, okay. – So the dual to the
death, well if you look at samurai films and western films, they’re shot the same in a lot of ways. – [Rose] Huh. – The pacing is very much the same. The fight scenes themselves
play out very similarly. Because there’s this extreme
deference to the weapon because they know that the
minute somebody is touched by that weapon it’s death. So a lot of the choreography is build up. Build up, build up, build up. Now it might be portrayed
differently in the western you go in here, with the eyes. And with the samurai film,
maybe they’ll just sit wide and like a kabuki theater. And then ba ba ba bam, the fight’s over. Both of ’em.
– Mm, quick, yeah. – Yeah, because there was
this respect for that weapon. – Yes, okay. And that might be something where we say, “Is it that kind of game?” – Right. Is your hero killed with one hit? Maybe we play it like that.
– Right, right. – Maybe we play the
cinematics like that too. – Mm.
– Wouldn’t make sense that the cinematics look
like a Kung Fu movie but you could be killed with one hit. Doesn’t really play together. Right?
– [Rose] Right, right. SO that’s the kind of stuff
that we consult people with. And try and lay out
all of that foundation. – I know you mentioned in one of the videos I saw
online, vertical integration. – [Eric] Yeah.
– Yeah, can you kinda break that down a little
bit for me as a process? – Yeah exactly and that
plays into this exactly. When we were doing our action films back in those indie days,
I guess we’re still indie in a lotta ways to this day. We still make stuff.
– [Rose] Good! – We’re always trying, you
know we always have to try to– – It’s always good to
keep making some things. – Yeah, you gotta keep making something even when you’re a pro–
– [Rose] Exactly. – You know, like if the only
thing that you’re able to do is with your latest contract it’s like, that might not come out for three years, how you gonna make a demo reel, so we’re always testing it out. – [Rose] Good, yes.
– The process that we use is something I call vertical integration. If you look at the best
action films, “John Wick”, “The Raid”, “Ong Bak”, “The Matrix”, what you’ll find is
they have a very similar process that Hong Kong films had, which is that every piece of the production pipeline
understand the action. – Mm. – Every piece of it. Old school Hong Kong
Chinese martial art movies, the camera guys probably had some kind of cultural understanding of martial arts. The editor understand it. And of course the performers and the choreographers understand it too. And we took that and did indie
films with that same process. So I liken it to a Jenga tower. Where if any one of those
pieces is outta whack, if the camera doesn’t see the
action then it falls over. – Right, right.
– The whole tower falls over. Now you’ll still have kind
of a rubble of a fight scene. You’ll still have a movie. – (laughing) You can have a scene I guess. – You have a scene, and there’s movement. Yeah, you know, they
moved around and fought. And unfortunately, if
that movie does well, despite that terrible action scene, the producer might look at that and say, “Well apparently that rubble works.” – Mm. And you’re like, “No,
but it could be better.” – It’s hard to get in there and do that but then when you have
movies like “The Matrix” that come in where you
have Hong Kong filmmakers leaving Hong Kong before
the handover in 1997 and they all came to the
US, you have “Rush Hour”, Sammo Hung did “Martial
Law”, Michelle Yeoh was in “Tomorrow Never Dies”, Jet
Li did “Lethal Weapon 4”, they all came over at that
time and introduced their way of filmmaking and it totally blew up. – Interesting.
– It was a huge success. And Yuen Woo Ping was
another guy and he worked with the Wachowskis and
the Wachowskis said, “You know what, just
make these action scenes “how you wanna make ’em.” And he integrated the filmmaking style with the rest of the film and that’s why those fights are seamless. – [Rose] Yeah.
– In that film. – Yeah, I wanna, so
just to kinda break down vertical integration a little bit more, the blocks on the Jenga tower
from what I have written there’s previs, choreo, performance, camera work, edit and sound. – Yes. – So if any of those
elements are out of place, that’s when the whole scene
gets a little outta whack. – Yup, that’s when it just doesn’t, it doesn’t feel like
it’s part of the movie. – [Rose] Sure.
– Somethin’s just off. – [Rose] Yeah.
– And again with action it might pass still, right? A better example would be comedy. – [Rose] Okay. – If you have a great
comedic scene it’s because you have all those elements in place. And if a joke or a comedy
scene fails, the movie fails. – [Rose] Mm, sure.
– So that’s like a, you know, even though we’re
in action, I actually use comedy as an example a lot of the time. – [Rose] Yeah.
– To try and push the message that if your action fails
the movie will fail. – [Rose] Mm-hmm.
– Even though it might not. But I do want them to believe that. – Depending on whether flashy
explosions or whatever. – [Eric] Yeah, you never know.
– Yeah, yeah, yeah. Yeah, well comedy is high risk. – [Eric] Absolutely.
– And action is high risk. – [Eric] Yeah, absolutely.
– So it makes sense that they have that kinda
handshake and parallel. – Yeah, and it’s really
hard, in the Hollywood kinda pipeline, it’s really
hard to make comedy work. – Mm.
– I think that’s why the sitcom is such a
success is because they made that work within that pipeline, but I mean making like another “Pink Panther” film, it’s almost impossible, you can’t do it. Because you have to have your
camera and editing department on board with the comedy
and you don’t even know who the editor is when you’re shooting it. – [Rose] Sure, yeah.
– So you just don’t really get those films unless
they’re these indie films. And plus, if you’re tryin’
to make a global film trying to sell comedy globally
now, it’s a huge risk. – Oh yeah.
– Even though I think it’s possible.
– Uh-huh. – But usually a distributor’s
not gonna want it. They’re just gonna– – It’s tough. Well, there’s so many
different types of comedy. – Yeah.
– I mean, just across the US. But around the world people have different feelings towards it. – But like the vertical integration is how we approach action to try
and really work the action into the process of the game in this case, and in movies too if we can. But with gaming it’s actually
easier, because in gaming all processes are happening
at the same time already. – [Rose] Okay, by how do
you mean all processes? – Because at the same
time that they’re creating characters and levels,
they’re doing the sound design at the same time.
– Ooh. – It doesn’t all come at the end. It’s all happening at the same time, so it looks like a spiral. All these processes going at the same time and there are these checkpoints where they make a vertical slice and
you know they’re doin’ alpha and beta and all that stuff. And so since they’re doing
everything at the same time they understand vertical integration. – [Rose] Mm, okay. – It’s like second nature to them. – Okay. – That’s how we can work
with them so easily. – So you’re kinda bringing
this game development technique into the film industry, almost. – [Eric] Well–
– Working on it. – We tried to use it in the film industry. And it’s hard because film is linear. You do the previs and that’s done. And you shoot it and that’s done. And then they edit it and that’s done. – [Rose] You have to plan ahead. – And then the sound and that’s done. – Yeah, yeah, yeah. – And if not everybody’s on
board, you make a mistake here, either you gotta reshoot
it or their just gonna try and fix it over here somehow. – Sure. – And it’s not integrated. It’s really hard to do that. – It’s tricky. Do you prefer working with video games? Or do you prefer cinema? – Oh that’s tough. That’s tough, I mean
there’s always a part of me that’s gonna always wanna do film. There’s just something, it’s so tactile. And you can never capture human movement exactly with motion capture. I look at motion capture as a new way of telling action stories. And video games in the same way, right? What’s nice about it
too is indie filmmakers can now get access to
software like Unreal, Unity and make action games,
action movies even, using 3-D and with a team of two people
you could actually make an incredible action movie in 3-D. So it’s just, it’s one
more tool to really kind of boost the quality–
– Yeah. – Of movies in general. – Very cool. Well, are there any other
takeaways that you wanna share either about the tech
behind motion capture or your experiences in stories? I know it’s a broad question. (laughs) – That is a broad question. – Tips or tricks for
someone getting started? – Okay. I would stick to my philosophy. When we started SuperAlloy
we had a motion capture suit and we were going almost door-to-door to indie game companies because we needed to make a demo reel. ‘Cause we, I’m not a game developer. I can make movies, I can do the action. But we needed a game to
put the motion capture in to demo what we could do. – [Rose] Sure.
– And so we would go to these indie developers and some of them we would almost work for free
just to get the footage right? At the beginning. And then we also went
and, I ended up learning how to use Unity in order
to make our demo real. – [Rose] That’s great. – And that was the only
way for us to prove that we were doing this. And I think that that
goes for almost anything in the industry and in media. If you’re not, you have
to be able to learn how to make something and promote whatever it is that you can do. Otherwise your resume
doesn’t really mean anything. Even your IMDB doesn’t
mean as much any more. People wanna see a reel.
– Mm. – People wanna see what
your doing and what makes, you know, what is it about
your process, your art that makes it unique and marketable. – [Rose] Very cool. Absolutely.
– And if you can actually get your hands around
the means of production, which it’s free to do almost anything now, then you can get that
information out there and– – Right. – And show how you can, and
that’s how you can innovate too. – Yeah, totally, totally. – If you’re innovating
you’re gonna do fine. – That’s a good way to
look at it for sure, yeah. That practical experience. The internet’s an amazing thing, and keep creating new things. – That’s right.
– I love it! Very cool, so where can our audience find out more about you. – You can find us at
superalloyinteractive.com, excuse me, and I’m on Twitter as Eric Jacobus. And YouTube is Eric Jacobus also. – Very cool. Well thank you so much
for joining us today. – Thank you. – It was awesome. My gosh, I can’t believe it’s over, there’s so many things to talk about. – Next time. – Next time.
– Part two. – Yeah, totally, totally. And thank you all for tuning into this episode of “Cinema Crunch”. Again, my name is Rose
Donahue and we’re filming at the Quantum Arc Media
studio in Las Vegas, Nevada. Catch you next time.

Leave a Reply

Your email address will not be published. Required fields are marked *