top of page
Game Programming and Design
Live Motion Capture Tech
For this project we used Unreal's Live Link app on Apple's iPhone's to link the camera to a skeleton rig in the engine. The result is live action motion capture that can be expressed and recorded onto any character model the skeleton is attached to. Allowing for simple, easy and cheap recreations of animation made quickly by simply booting up the engine.
Mocap Code: Bio
Hardware
Mocap Code: Welcome
Hardware for this project is actually fairly minimal. All that was needed was an iPhone 12 Pro model or later and a computer with a graphics card good enough to render a high poly model in Unreal. My PC did perfectly fine handling having the software open and recording it with a screen capture device and it is no where near top model. A lot of the operation cost is just in getting the iPhone as that recent of a model is usually around $1,000. However, it needs to be an iPhone with deep face tracking capabilities as this is how live link uses the AR kit plugin in Unreal to modify the model in the engine.
Mocap Code: Text
Software
Mocap Code: Welcome
The software we used for this project is actually all free and easy to use. Essentially the app uses the phones deep facial tracking features to send 51 different blend shape points to the engine so it can modify the curves of the model. since it is a feature unreal made for apple phones specifically, it makes it very easy to setup with no real problems on trying to send or receive the information the phone is transmitting.
Mocap Code: Text
How it Works
Mocap Code: Welcome
Setting up the model to receive inputs from the phone is a two step process. First, the phone bust be connected to the same WiFi as the computer running the project. Second, the computer needs to be setup to receive the phones inputs which usually involves disabling a firewall or two and setting the engines IPv4 address to the same as the network it's connected to.
​
Once the phone is connected there is a process with getting it to work with the model and I have done both. The model we made for the client was made for this tech in mind. So we named all the blend shapes on the models face the same as the names of the incoming data. This meant that we could hook up the model to an animation blueprint that was taking input from the phone and it would move the blend shapes directly since they were already named the same. I also set this up with the Weta Digital Meerkat model. This model was not made with this technology in mind for it to affect it so I had to make a renaming node that took incoming data and changed the names of it to the blend shapes that the meerkat model had. This process wasn't that difficult and it made me realize with enough work put in you could use this for any model with facial blend shapes already attached to it.
​
Below is the animation blueprint that I made for controlling the face of the model with the inputs from the phone.
Mocap Code: Text
Mocap Code: Image
As you can see, a majority of the animation detail is coming from the phone itself. However, some of the components like the eyes and added rotation to the head and body needed more to be interpreted. Since the body does not move on it's own at all, I added a clock to puff out his chest to give the appearance of breathing and I tied the body rotations to a fraction of the head movement to make his body less stiff and make it impossible to look like he broke his neck if the head turned too much.
​
This technology seems underutilized as there were only a handful of tutorials on how to setup and use it and even less in terms of documentation on how to fix bugs that occurred.
Mocap Code: Text
Hair
Mocap Code: Welcome
A smaller portion of my contributions to this project were working with the Groom plugin function in Unreal. I made some simple hair particles in blender on the model and then exported them into Unreal as an alembic file. It was a very new process for me as I am not much of a 3D modeler, but I could confidently say I can make some decent looking hair now.
Mocap Code: Text
bottom of page