> ## Documentation Index
> Fetch the complete documentation index at: https://mintlify.com/sepinf-inc/IPED/llms.txt
> Use this file to discover all available pages before exploring further.

# Face Recognition

> Detect and search for similar faces in images and videos

IPED includes advanced face recognition capabilities powered by the Face Recognition library, enabling detection of faces in images and videos, and similarity-based face search across the entire case.

## Overview

The face recognition feature provides:

* **Face Detection** - Automatically locate faces in images and video frames
* **Face Encoding** - Generate 128-dimensional feature vectors for each face
* **Similar Face Search** - Find faces similar to a reference face across the case
* **Video Support** - Process video thumbnails for face detection
* **GPU Optimization** - Optimized to run efficiently without GPU requirements

## Technology

IPED uses the Face Recognition library built on dlib's state-of-the-art face recognition:

```python theme={null}
# Python face recognition based on Face Recognition Project
# https://pypi.org/project/face-recognition/
```

Key components:

* **dlib** - C++ machine learning library
* **face\_recognition** - Python wrapper with simple API
* **OpenCV** - Image processing operations
* **NumPy** - Numerical computing for feature vectors

## Configuration

Face recognition is configured in `FaceRecognitionConfig.txt`:

```properties theme={null}
# Enable face recognition processing
enableFaceRecognition = true

# Number of parallel face recognition processes
numFaceRecognitionProcesses = 2

# Maximum image dimension (larger images downscaled)
maxResolution = 1024

# Face detection model: 'hog' (faster, CPU) or 'cnn' (accurate, GPU)
faceDetectionModel = hog

# Upsampling factor (higher = detect smaller faces, slower)
upSampling = 1

# Minimum face size in pixels (smaller faces ignored)
minSize = 48
```

## Face Detection Models

### HOG (Histogram of Oriented Gradients)

**Recommended for CPU processing**

```python theme={null}
detection_model = 'hog'
```

Characteristics:

* Fast execution on CPU
* Good accuracy for frontal faces
* Lower memory usage
* Handles standard lighting conditions

Performance: \~200-300ms per image on modern CPU

### CNN (Convolutional Neural Network)

**Recommended for GPU processing**

```python theme={null}
detection_model = 'cnn'
```

Characteristics:

* Higher accuracy
* Better with various angles and lighting
* Detects partially occluded faces
* Requires GPU for acceptable speed

Performance: \~50-100ms per image on GPU, \~2-3s on CPU

## Implementation

The face recognition task is implemented as a Python script integrated into IPED's processing pipeline:

### Initialization

```python theme={null}
class FaceRecognitionTask:
    def init(self, configuration):
        # Check required modules
        import face_recognition
        import cv2
        import numpy as np
        
        # Verify numpy version (<2.x)
        if np.__version__ >= '2':
            logger.error('numpy version must be <2.x')
            return
        
        # Create external process pool
        createProcessQueue()
```

### Processing Flow

```python theme={null}
def process(self, item):
    # 1. Filter items
    if not item.getExtraAttribute('hasThumb'):
        return  # Skip non-image items
    
    # 2. Check minimum size
    width = item.getMetadata().get('Width')
    height = item.getMetadata().get('Height')
    if width < min_size or height < min_size:
        return  # Skip small images
    
    # 3. Get or create external process
    proc = processQueue.get(block=True)
    
    # 4. Send image path to process
    print(img_path, file=proc.stdin, flush=True)
    
    # 5. Read face detection results
    num_faces = int(proc.stdout.readline())
    
    # 6. Read face locations
    for i in range(num_faces):
        location = eval(proc.stdout.readline())
        face_locations.append(location)
    
    # 7. Read face encodings (128 values per face)
    for i in range(num_faces):
        encoding = []
        for j in range(128):
            encoding.append(float(proc.stdout.readline()))
        face_encodings.append(encoding)
    
    # 8. Store results
    item.setExtraAttribute('face_count', num_faces)
    item.setExtraAttribute('face_locations', face_locations)
    item.setExtraAttribute('face_encodings', face_encodings)
```

## Face Attributes

Detected faces stored as item extra attributes:

### face\_count

Number of faces detected in the image.

```java theme={null}
Integer count = item.getExtraAttribute(ExtraProperties.FACE_COUNT);
```

### face\_locations

Bounding box coordinates for each face (top, right, bottom, left).

```java theme={null}
List<List<Integer>> locations = 
    item.getExtraAttribute(ExtraProperties.FACE_LOCATIONS);
```

### face\_encodings

128-dimensional feature vectors for similarity comparison.

```java theme={null}
List<float[]> encodings = 
    item.getExtraAttribute(ExtraProperties.FACE_ENCODINGS);
```

## External Process Architecture

IPED uses external Python processes for face recognition:

### Benefits

1. **Stability** - Crashes don't affect main IPED process
2. **Parallelism** - Multiple processes for concurrent processing
3. **Resource isolation** - Memory leaks contained
4. **Process reuse** - Avoid model reload overhead

### Process Communication

Communication via stdin/stdout:

```python theme={null}
# Send image path
print(image_path, file=process.stdin)

# Receive results
num_faces = int(process.stdout.readline())
```

Health monitoring:

* Ping/pong mechanism verifies process alive
* Automatic restart if process crashes
* Error logging for debugging

## Similar Face Search

The analysis interface provides face search functionality:

### From Image Viewer

1. Right-click on detected face
2. Select "Search Similar Faces"
3. Adjust similarity threshold slider
4. View results ranked by similarity

### From External Image

1. Tools menu → Face Search
2. Select reference image containing face
3. Set similarity threshold
4. Search entire case for matches

### Similarity Calculation

Based on Euclidean distance between 128D encodings:

```python theme={null}
import numpy as np

def face_distance(face_encoding1, face_encoding2):
    return np.linalg.norm(face_encoding1 - face_encoding2)

# Typical threshold: 0.6
# Lower distance = more similar
```

Threshold guidelines:

* **0.3** - Very strict, same person in similar conditions
* **0.4** - Strict, likely same person
* **0.6** - Default, good balance (recommended)
* **0.8** - Relaxed, may include similar-looking people

## Performance Optimization

### Image Downscaling

Large images automatically downscaled:

```python theme={null}
max_size = 1024  # Configurable
if width > max_size or height > max_size:
    # Downscale maintaining aspect ratio
    scale = max_size / max(width, height)
    new_width = int(width * scale)
    new_height = int(height * scale)
```

Benefits:

* Faster processing
* Reduced memory usage
* Minimal accuracy impact

### Process Pooling

```python theme={null}
maxProcesses = int(max(1, numThreads / 2))
processQueue = queue.Queue(maxProcesses)
```

* Reuses Python processes
* Avoids model reload overhead
* Balances parallelism vs. memory

### Result Caching

```python theme={null}
cache[hash + '_locations'] = locations
cache[hash + '_encodings'] = encodings
cache[hash + '_count'] = count
```

* Duplicate files processed once
* In-memory cache for fast access
* Significant speedup for duplicates

## Video Support

Face recognition works on video files via thumbnails:

```python theme={null}
if mediaType.startswith('video') and videoSubitems:
    # Process video thumbnail
    img_path = previewManager.readPreview(item)
```

Configuration:

* Requires video thumbnail generation enabled
* Processes representative frame
* Detects faces in thumbnail image

## Installation Requirements

Required Python packages:

```bash theme={null}
pip install face-recognition
pip install opencv-python  
pip install numpy<2.0
```

System dependencies:

* **CMake** - For compiling dlib
* **C++ compiler** - GCC/Clang or Visual Studio
* **CUDA** (optional) - For GPU acceleration

Detailed setup: [IPED User Manual - FaceRecognition](https://github.com/sepinf-inc/IPED/wiki/User-Manual#facerecognition)

## Use Cases

### Person of Interest Search

Find all images containing a specific individual:

1. Identify reference image with target person
2. Run similarity search
3. Review ranked results
4. Bookmark matches for export

### Victim Identification

Locate victims in CSAM investigations:

* Compare against known victim database
* Identify co-occurrences
* Build victimology timeline

### Suspect Association

Map relationships:

* Find images showing multiple suspects together
* Identify unknown associates
* Build network diagrams

### Social Media Analysis

Analyze social media content:

* Identify persons in photos
* Track presence across platforms
* Timeline of appearances

## Limitations

### Detection Challenges

* **Extreme angles** - Profile views harder than frontal
* **Poor lighting** - Underexposed or overexposed faces
* **Occlusion** - Masks, sunglasses reduce accuracy
* **Low resolution** - Faces smaller than minSize ignored
* **Motion blur** - Blurry faces may not detect

### Search Limitations

* **Age progression** - Significant aging affects similarity
* **Facial hair** - Beards/mustaches impact matching
* **Cosmetics** - Heavy makeup changes features
* **Image quality** - Low quality reduces accuracy

## Troubleshooting

### No Faces Detected

* Check minimum image size (width/height > minSize)
* Verify faces are clearly visible
* Try increasing upSampling parameter
* Switch to 'cnn' model for better detection

### Slow Processing

* Reduce maxResolution for faster processing
* Use 'hog' instead of 'cnn' model
* Decrease upSampling value
* Increase numFaceRecognitionProcesses

### High Memory Usage

* Reduce numFaceRecognitionProcesses
* Lower maxResolution
* Process videos as subitems to reduce memory

### Process Crashes

* Check Python environment compatibility
* Verify numpy version less than 2.0
* Review error logs for stack traces
* Restart with fewer parallel processes


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.