Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add XR_FB_face_tracking support #84

Merged
merged 1 commit into from
Feb 3, 2024

Conversation

Malcolmnixon
Copy link
Contributor

This pull request adds support for the XR_FB_face_tracking extension to the Meta vendor plugin. Users can:

  • Select face-tracking features of None, Optional, and Required
  • Access the tracking data through the OpenXRFbFaceTrackingExtensionWrapper singleton
  • Test if face-tracking is enabled by the is_enabled() method
  • Get all 63x expression weights using the get_weights() method (indexed via enums)
  • Get the 2x tracking confidences using the get_confidences() method (indexed via enums)

It includes a face model from https://booth.pm/en/items/5420804 which contains almost all the blend-shapes necessary to test the face-tracking feature; although the files are quite large for this.

OpenXR.Face.Tracking.mp4

@dsnopek
Copy link
Collaborator

dsnopek commented Feb 3, 2024

Wow, this looks so cool! I'll review in depth when I have a chance :-)

@m4gr3d m4gr3d self-requested a review February 3, 2024 16:10
@m4gr3d m4gr3d added this to the 2.1.0 milestone Feb 3, 2024
@m4gr3d m4gr3d added the enhancement New feature or request label Feb 3, 2024
Copy link
Collaborator

@m4gr3d m4gr3d left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The logic looks good!

@m4gr3d m4gr3d modified the milestones: 2.1.0, 3.0.0 Feb 3, 2024
Copy link
Collaborator

@dsnopek dsnopek left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just tested this with the Quest Pro and it worked great! (Except the face model in the demo is a little creepy :-)) Skimming the code, it looks good to me as well

@m4gr3d m4gr3d merged commit c382ba7 into GodotVR:master Feb 3, 2024
5 checks passed
@m4gr3d
Copy link
Collaborator

m4gr3d commented Feb 3, 2024

@Malcolmnixon Just to confirm, this only works on the Quest Pro, right?

@dsnopek
Copy link
Collaborator

dsnopek commented Feb 3, 2024

In my testing, it only works on the Quest Pro. I also tried on the Quest 3, but it does nothing there - it doesn't even show the permission overlay about face tracking

@Malcolmnixon
Copy link
Contributor Author

@Malcolmnixon Just to confirm, this only works on the Quest Pro, right?

I believe only the Quest Pro has the face-tracking cameras necessary to do this.

@m4gr3d
Copy link
Collaborator

m4gr3d commented Feb 23, 2024

@Malcolmnixon We should consider updating the implementation to use XR_FB_face_tracking2 instead.

@Malcolmnixon
Copy link
Contributor Author

@Malcolmnixon We should consider updating the implementation to use XR_FB_face_tracking2 instead.

That will need us to update openxr to 1.0.34 first. I'll get a separate PR ready for that.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants