Automatic 3D model rigging and real-time Linear Mesh Blending avatar animation in C++ and Kinect.

Key Features

• Kinect animation and caching
• 3D model animation based on LBS
• Automatic skeleton and bone weights generation
• Support for various 3D models: .obj .fbx .off, .ply and .stl file formats.
• Cross platform
  • Windows (only Kinect), macOS and Linux ready.

Dependencies

The project was built / run on Windows, Visual Studio 2019. The Desktop development with C++ bundle must include MSVC v142 - VS 2019 build tools, Windows 10 SDK, C++ MFC for MSVC v142.

• FLTK - library v1.3.5 is needed for rendering. Download and unzip it and follow the instructions, set the appropriate env FLTK_DIR.

• STB - is required for image texture loading. Clone the project, as it is a header library, set only the appropriate env STB_DIR.

• Eigen - is used for linear algebra related problems. Clone the project, as it is a header library, set only the appropriate env EIGEN_DIR.

• Kinect for Windows SDK v1.8 - allows skeleton tracking to cache animations and live animate a 3D model. Download and install the distribution and set KINECTSDK10_DIR if it's not defined to the installation path.

• FBX SDK - is useful to load complex 3D models that contain textures, skeleton... Download and install the distribution for Windows and set FBX_DIR to the installation path or follow the following instructions.

Older releases may work too...

How To Use

Loading a 3D model that is automatically rigged based on the joints.out and animate live based on Kinect:

data/human.obj data/joints.out data/skeleton.out non-rigged live

Loading a 3D model that is rigged and animated based on cached animation skeleton.out:

data/model.fbx data/joints.out data/skeleton.out rigged non-live

Note: Run in release mode to speed up things. To set up the command arguments in VS2019, select Project project > Properties > Configuration Properties > Debugging > Command Arguments

Interaction

• Press s to toggle skeleton, g for floor, f for flat shading rendering.
• Use mouse wheel or by mouse dragging to move the camera.

Miscellaneous

The file data/joints.out contain the definition of the desired base skeleton (human, monkey, koala...). The loaded 3D model is embedded / scaled into [-1, 1] world space, thus each line contains an unique joint, its desired / expected 3D location, and his parent joint. Specify -1 if the corresponding joint is the root.

The motion file data/skeleton.out is based on our custom motion format, where the skeleton is tracked each frame with first line representing 20 joints (Kinect v1) world positions and 20 new lines each representing the index of parent joint, index of child joint, absolute quaternion rotation w.t.r Y-axis for joints world orientation

Credits

The rigging is based on:

• Pinocchio - please refer also the paper for extra information about rigging process.