---

NeuroFace is a Python framework containing tools for human face detection, recognition, analysis of human facial expressions and gestures on video.

Table of Contents

1. Environment
2. Installation
3. Supported Models
1. Face Detection
2. Face Embedding
3. Facial Landmark Detection
4. Pose Landmark Detection
5. Facial Expression Recognition
4. Usage
5. References

1. Environment

• torch ≥ 1.12.0: PyTorch models implementation and deployment.

• torchvision ≥ 0.13.0: Reading and preprocessing frames as PyTorch tensors.

• mediapipe ≥ 0.8.10.1: MediaPipe models implementation and deployment.

• opencv-python ≥ 4.6.0.66: Reading and preprocessing frames as NumPy arrays.

• Pillow ≥ 9.2.0: Reading and preprocessing frames as PIL Images.

• av ≥ 9.2.0: Binding Torchvision to FFmpeg to read streams.

• gdown ≥ 4.5.1: Downloading large files from Google Drive.

System requirements:

• CUDA ≥ 11.3: GPU compute access.

Quick dependency installation:

$ pip install -r requirements.txt

2. Installation

Cloning the repository into a project:

$ git clone https://github.com/mdvdv/neuroface.git

3. Supported Models

3.1 Face Detection

Model Name	Training Dataset	Backbone	AP Metric	Weights	Paper
MTCNN	WiderFace	P-Net	0.946	P-Net (28 KB)	PDF
		R-Net	0.954	R-Net (394 KB)
		O-Net	0.954	O-Net (1.5 MB)
RetinaFace		MobileNet V1	0.914	RetinaFace (1.7 MB)	PDF
MediaPipe Face	-	BlazeFace	0.9861	-	PDF

3.2 Face Embedding

Face recognition models use a multi-dimensional vector representation of a face. NeuroFace provides access to face vectorization directly using Inception-ResNet V1 model.

Model Name	Training Dataset	Backbone	AP Metric	Weights	Paper
Inception-ResNet V1	CASIA-WebFace	Inception	0.9905	CASIA-WebFace (110.5 MB)	PDF
	VGGFace2	Inception	0.9965	VGGFace2 (106.7 MB)

3.3 Facial Landmark Detection

NeuroFace provides MediaPipe Face Mesh that estimates 478 3D face landmarks in real-time. Face detector is the same BlazeFace model used in MediaPipe Face Detection.

Model Name	Backbone	MAD Metric	Paper
MediaPipe Face Mesh	BlazeFace	0.396	PDF

3.4 Pose Landmark Detection

Model Name	Backbone	MSE Metric		Paper
MediaPipe Hands	BlazePalm	Light	11.83	PDF
		Full	10.05

3.5 Facial Expression Recognition

Model Name	Training Dataset	Backbone	AP Metric	Weights	Paper
DAN	AffectNet-8	ResNet	0.6209	AffectNet-8 (226 MB)	PDF

4. Usage

Code execution examples are presented in the documentation section.

5. References

1. Face Recognition Using Pytorch: https://github.com/timesler/facenet-pytorch.
2. Cross-platform, Customizable ML Solutions for Live and Streaming Media: https://github.com/google/mediapipe.
3. Distract Your Attention: Multi-head Cross Attention Network for Facial Expression Recognition: https://github.com/yaoing/dan.