face_detect — Face Detection Application¶

face_detect is a face detection application for the K230. It consists of two workflows: Python scripts for kmodel compilation and accuracy evaluation, and a C++ on-device application.

Prerequisites¶

K230 SDK must be built (toolchain extracted, MPP libraries compiled)
SDK placed at k230_sdk/ in the repository root
Python 3.8 or later (see requirements.txt)
Host OS: x86_64 Linux
CMake 3.16 or later

Building the SDK

For K230 SDK build instructions, see SDK Build.

Overview¶

face_detect is based on sample_face_ae, with the following additions:

Python scripts: Compile ONNX models to K230 kmodel format and evaluate accuracy
Capture feature: Press 'c' to save the current frame as PNG on the device
Input thread: 'c' to capture, 'q' to quit
OpenCV linking: Uses OpenCV for PNG encoding and saving

Differences from sample_face_ae¶

Feature	sample_face_ae	face_detect
Face detection + AE ROI	Yes	Yes
Capture feature	No	Yes (OpenCV)
Input thread	No	Yes ('c'/'q')
OpenCV linking	No	Yes
Python scripts	No	Yes

Overall Workflow¶

ONNX Model
  |
  +-- step1: Model analysis (inspect input/output info)
  +-- step2: Model simplification (onnxsim)
  +-- step3: kmodel compilation (PTQ quantization)
  +-- step4: Simulation execution
  +-- step5: Accuracy comparison with ONNX Runtime
                    |
                kmodel --> On-device app --> Captured images
                                              |
                                    Recompile with step3 --calib-dir
                                              |
                                    Verify with evaluate_kmodel.py

Part 1: kmodel Compilation (PC)¶

What is nncase?¶

nncase is a neural network compiler for the K230 KPU (Knowledge Process Unit). It converts ONNX models to kmodel format using PTQ (Post-Training Quantization).

References:

CanMV K230 Tutorial — Model Compilation
nncase examples — K230 Simulate (GitHub)
Official scripts bundled with the SDK: k230_sdk/src/big/nncase/examples/scripts/

Python Environment Setup¶

Use the .venv at the repository root (shared with MkDocs):

source .venv/bin/activate
pip install -r requirements.txt

Script List¶

Script	Description
`step1_parse_model.py`	Analyze ONNX model input/output
`step2_simplify_model.py`	Simplify ONNX with onnxsim
`step3_compile_kmodel.py`	Compile to kmodel (PTQ quantization)
`step4_simulate_kmodel.py`	Run kmodel simulation
`step5_compare_results.py`	Compare accuracy with ONNX Runtime
`evaluate_kmodel.py`	Batch accuracy evaluation

Step 1: Model Analysis¶

Inspect the ONNX model's input/output information.

python apps/face_detect/scripts/step1_parse_model.py

Key information:

Input: float32 [1, 3, 320, 320] (NCHW)
Output: 9 tensors — 3 scales x (classification, bbox, landmark)

Step 2: Model Simplification¶

Optimize the ONNX model using onnxsim. This removes redundant nodes and improves compilation stability.

python apps/face_detect/scripts/step2_simplify_model.py

Output: apps/face_detect/scripts/output/simplified.onnx

Step 3: kmodel Compilation¶

Compile the simplified ONNX to a K230-targeted kmodel.

Compilation Settings¶

Option	Value
preprocess	True
input_type	uint8
input_range	[0, 255]
mean	[123, 117, 104] (RGB order)
std	[1, 1, 1]
quant_type	uint8
calibrate_method	Kld

What preprocess=True means

The kmodel internally handles uint8-to-float32 conversion and mean/std normalization. On the device, raw camera data (uint8) can be fed directly as input.

Compile with Random Data (Initial)¶

python apps/face_detect/scripts/step3_compile_kmodel.py

Compile with Captured Images (Improved Accuracy)¶

python apps/face_detect/scripts/step3_compile_kmodel.py --calib-dir /path/to/captures/

Output: apps/face_detect/scripts/output/dump/mobile_retinaface.kmodel

Step 4: Simulation¶

Run the compiled kmodel on the PC simulator.

# Use the SDK-bundled test image
python apps/face_detect/scripts/step4_simulate_kmodel.py

# Use a custom image
python apps/face_detect/scripts/step4_simulate_kmodel.py --image photo.png

Output: .npy files for each output tensor are saved in apps/face_detect/scripts/output/dump/.

Step 5: Accuracy Comparison¶

Compare the kmodel simulation results with ONNX Runtime inference using cosine similarity.

python apps/face_detect/scripts/step5_compare_results.py

Accuracy guidelines:

Cosine Similarity	Rating
0.999 or above	excellent
0.99 or above	good
0.95 or above	acceptable
Below 0.95	poor — needs improvement

Batch Accuracy Evaluation¶

Compare kmodel and ONNX Runtime outputs in batch for all images in a directory.

python apps/face_detect/scripts/evaluate_kmodel.py /path/to/images/

This displays min/mean/max cosine similarity for each output.

Calibration Improvement Cycle¶

How to improve accuracy

Compile with random data (step3) and verify on the device
Use 'c' key on the device app to capture images from the real environment
Recompile with step3 --calib-dir using captured images
Verify accuracy with evaluate_kmodel.py
Repeat steps 2–4 as needed

Part 2: On-Device Application (C++)¶

Source Files¶

File	Description
`main.cc`	Main application — VICAP/VO initialization, inference loop, capture feature
`model.h` / `model.cc`	`Model` abstract base class — kmodel loading and inference pipeline
`mobile_retinaface.h` / `mobile_retinaface.cc`	`MobileRetinaface` class — face detection model (AI2D preprocessing, anchor decoding, NMS)
`face_ae_roi.h` / `face_ae_roi.cc`	`FaceAeRoi` class — maps face coordinates to ISP AE ROI
`util.h` / `util.cc`	Utility types (`box_t`, `face_coordinate`) and helpers
`anchors_320.cc`	Pre-computed anchor boxes for 320x320 input
`vo_test_case.h`	VO layer helper type (`layer_info`) declarations

Processing Flow¶

Sensor (OV5647)
  |
  +-- CHN0 (1920x1080 YUV420) --> VO Layer --> HDMI Display
  |                                    ^
  |                              Face box drawing (kd_mpi_vo_draw_frame)
  |
  +-- CHN1 (1280x720 RGB888P) --> AI Inference
                                    |
                              +-----+-----+
                              | AI2D      |
                              | Preprocess|
                              | (resize+pad) |
                              +-----+-----+
                                    |
                              +-----+-----+
                              | KPU       |
                              | Inference |
                              | (MobileRetinaface) |
                              +-----+-----+
                                    |
                              +-----+-----+
                              | Postprocess|
                              | (decode+NMS) |
                              +-----+-----+
                                    |
                          +---------+---------+
                          |         |         |
                    AE ROI      Face box   Capture
                    Update      Drawing    ('c' key)
                    (FaceAeRoi)             |
                          |                 v
                          v           PNG Save (OpenCV)
                    ISP AE Engine

Build Steps¶

1. Configure¶

cmake -B build/face_detect -S apps/face_detect \
  -DCMAKE_TOOLCHAIN_FILE="$(pwd)/cmake/toolchain-k230-rtsmart.cmake"

2. Build¶

cmake --build build/face_detect

3. Verify¶

file build/face_detect/face_detect

Expected output:

face_detect: ELF 64-bit LSB executable, UCB RISC-V, RVC, double-float ABI, version 1 (SYSV), statically linked, ...

CMakeLists.txt Details¶

The apps/face_detect/CMakeLists.txt handles:

MPP include paths: Headers from mpp/include/, mpp/include/comm/, mpp/include/ioctl/, and mpp/userapps/api/
NNCASE include paths: Headers from nncase/include/, nncase/include/nncase/runtime/, and rvvlib/include/
OpenCV include paths: Headers from opencv_thead/include/opencv4/
MPP static libraries: All MPP libraries linked with --start-group / --end-group to resolve circular dependencies
NNCASE libraries: Nncase.Runtime.Native, nncase.rt_modules.k230, functional_k230, rvv
OpenCV libraries: opencv_imgcodecs, opencv_imgproc, opencv_core and 3rdparty libraries
C++20: Requires C++20 via target_compile_features

Command-Line Arguments¶

./face_detect <kmodel> <ae_roi> [capture_dir]

Argument	Description
`<kmodel>`	Path to the face detection kmodel file (e.g., `/sharefs/mobile_retinaface.kmodel`)
`<ae_roi>`	Enable AE ROI: `1` = enabled, `0` = disabled
`[capture_dir]`	Directory to save captured images (optional)

Key Controls¶

Key	Action
c + Enter	Save current frame as PNG (only when `capture_dir` is specified)
q + Enter	Quit the application

Transferring and Running on K230¶

The CMake deploy / run targets handle transfer and execution in one command (see CMake Targets for details):

cmake --build build/face_detect --target deploy   # build + kmodel + SCP transfer
cmake --build build/face_detect --target run      # run via serial (Ctrl+C to disconnect)

Manual Transfer and Execution¶

Manual operation via SCP + minicom

Transfer via SCP¶

scp build/face_detect/face_detect root@<K230_IP_ADDRESS>:/sharefs/face_detect/
scp apps/face_detect/scripts/output/dump/mobile_retinaface.kmodel root@<K230_IP_ADDRESS>:/sharefs/face_detect/

Run on the K230 bigcore (msh)¶

On the K230 serial console (ACM1), run:

msh /> /sharefs/face_detect/face_detect /sharefs/face_detect/mobile_retinaface.kmodel 1

To run with AE ROI disabled:

msh /> /sharefs/face_detect/face_detect /sharefs/face_detect/mobile_retinaface.kmodel 0

Serial Connection¶

Bigcore (RT-Smart msh): /dev/ttyACM1 at 115200 bps

minicom -D /dev/ttyACM1 -b 115200

Run in Capture Mode¶

To capture images for calibration, specify capture_dir:

msh /> mkdir /sharefs/calib
msh /> /sharefs/face_detect/face_detect /sharefs/face_detect/mobile_retinaface.kmodel 1 /sharefs/calib

Calibration capture

Run with capture_dir specified and press 'c' + Enter several times to capture images from the real environment. Transfer the captured images to your PC and use them as calibration data for step3 --calib-dir.

# Transfer captured images from K230 to PC
scp root@<K230_IP_ADDRESS>:/sharefs/calib/*.png ./calib/

# Recompile with captured images for calibration
python apps/face_detect/scripts/step3_compile_kmodel.py --calib-dir ./calib/

CMake Targets¶

Configuration¶

cmake -B build/face_detect -S apps/face_detect \
  -DCMAKE_TOOLCHAIN_FILE="$(pwd)/cmake/toolchain-k230-rtsmart.cmake"

Target List¶

Target	Command	Description
(default)	`cmake --build build/face_detect`	Build C++ binary
`kmodel`	`cmake --build build/face_detect --target kmodel`	Compile kmodel (step2 + step3)
`deploy`	`cmake --build build/face_detect --target deploy`	Build + kmodel + SCP transfer to K230
`run`	`cmake --build build/face_detect --target run`	Run on K230 via serial (Ctrl+C to disconnect)

kmodel¶

Automates ONNX model simplification and kmodel compilation:

cmake --build build/face_detect --target kmodel

deploy¶

Builds the binary, compiles kmodel, and transfers everything to K230 via SCP:

cmake --build build/face_detect --target deploy

Files transferred:

Local	Path on K230
`build/face_detect/face_detect`	`/sharefs/face_detect/face_detect`
`apps/face_detect/scripts/output/dump/mobile_retinaface.kmodel`	`/sharefs/face_detect/mobile_retinaface.kmodel`

run¶

Sends a command to K230 bigcore (msh) via serial port and displays output in real time:

cmake --build build/face_detect --target run

Keyboard input is forwarded to K230 (q + Enter to quit the app)
Ctrl+C to disconnect from serial

K230 Connection Settings¶

Customize connection parameters via CMake cache variables:

Variable	Default	Description
`K230_IP`	(empty = auto-detect)	Littlecore IP address
`K230_USER`	`root`	SSH user
`K230_SERIAL`	`/dev/ttyACM1`	Bigcore serial port (for run)
`K230_SERIAL_LC`	`/dev/ttyACM0`	Littlecore serial (for IP auto-detect)
`K230_BAUD`	`115200`	Baud rate