Introduction to TensorFlow Lite
2025-07-25
TensorFlow Lite is a set of tools that enable on-device machine learning by helping developers run models on mobile, embedded, and edge devices. TensorFlow Lite use cases enable you to run models such as image classification, object detection, image segmentation, and pose estimation. Taking the Posenet machine learning model in pose estimation as an example, pose recognition can be performed on nodes in the Posenet_mobilenet dataset.
Download the Posenet model
Deploy Model
- Deploy the model and label files to QuecPi Alpha, and use adb or scp to deploy them to the /opt/ directory.
- Model features
-Model checkpoint: mobilenet-v1_101
-Input resolution: 513x257
-Number of parameters: 3.31M
-Model size: 12.7 MB
Connect the camera and monitor
- Prepare the MIPI camera and connect it to the first CSI slot of QuecPi Alpha.
- Prepare an HDMI monitor and connect it to QuecPi Alpha.
Using TFLite for pose estimation and display
Preconditions
export XDG_RUNTIME_DIR=/dev/socket/weston && export WAYLAND_DISPLAY=wayland-1
Camera stream real-time pose estimation display
setprop persist.overlay.use_c2d_blit 2
gst-launch-1.0 -e \
qtiqmmfsrc name=camsrc camera=0 ! video/x-raw\(memory:GBM\),format=NV12,width=1280,height=720,framerate=30/1,compression=ubwc ! queue ! tee name=split \
split. ! queue ! qtimetamux name=metamux ! queue ! qtioverlay ! queue ! waylandsink fullscreen=true sync=false \
split. ! queue ! qtimlvconverter ! queue ! qtimltflite delegate=external external-delegate-path=libQnnTFLiteDelegate.so external-delegate-options="QNNExternalDelegate,backend_type=htp;" model=/opt/posenet_mobilenet_v1.tflite ! queue ! qtimlvpose threshold=51.0 results=2 module=posenet labels=/opt/posenet_mobilenet_v1.labels constants="Posenet,q-offsets=<128.0,128.0,117.0>,q-scales=<0.0784313753247261,0.0784313753247261,1.3875764608383179>;" ! text/x-raw ! queue ! metamux.
Command Description:
- gst-launch-1.0:
This is a command-line tool for GStreamer, used to start a GStreamer pipeline. The e parameter indicates exiting at the end of the pipeline, rather than continuing to run. - qtiqmmfsrc name=camsrc camera=0:
Based on QMMF (Qualcomm Multimedia Framework), open the camera numbered 0 (usually the rear main camera). - !:
is a connection symbol used to connect the output of the previous element to the input of the next element. - video/x-raw(memory:GBM),format=NV12,width=1280,height=720,framerate=30/1,compression=ubwc：
The stream type is raw video frames, and the memory type is GBM (Generic Buffer Management), which means GPU/DRM shared memory to avoid copying. Color format NV12, resolution 720p, frame rate 30 fps, using Qualcomm UBWC (Universal Bandwidth Compression) compression to reduce bus bandwidth. - queue:
Thread boundary buffer to prevent upstream and downstream threads from blocking each other. - tee name=split:
queue Thread boundary buffer to prevent upstream and downstream threads from blocking each other. tee copies a stream into N identical streams; Here it is named split, and later uses split. to reference its two src pad. - split. ! queue ! qtimetamux name=metamux:
split. is the first output of tee, and queue creates another thread buffer. qtimetamux is a metadata multiplexer provided by Qualcomm, which is used to attach the subsequently detected keypoint results (as GstMeta) to video frames, with the name metamux, so that they can be referenced later. - qtioverlay ! queue ! waylandink fullscreen=true sync=false:
The overlay component provided by Qualcomm is responsible for drawing key points from GstMeta on the screen. Directly send frames to Wayland Composer (such as Weston) for full screen display. - split. ! queue ! Qtimlvconverter:
The second output of tee. Qualcomm MediaLib Video to ML Tensor Format Converter, which converts NV12 to RGB floating-point tensors required by PoseNet and performs scaling/normalization. - qtimltflite delegate=external external-delegate-path=libQnnTFLiteDelegate.so external-delegate-options="QNNExternalDelegate,backend_type=htp;" model=/opt/posenet_mobilenet_v1.tflite:
qtimltflite is Qualcomm TFLite inference plugin, using QNN (Qualcomm Neural Network) backend to run TensorFlow Lite models on Hexagon/HTP. Use an external delegate instead of the default CPU. Use the shared library of QNN TFLite delegate. Which will tell delegate to use HTP (Hexagon Tensor Processor) backend and PoseNet model file path. - qtimlvpose threshold=51.0 results=2 module=posenet labels=/opt/posenet_mobilenet_v1.labels constants="Posenet,q-offsets=<128.0,128.0,117.0>,q-scales=<0.0784313753247261,0.0784313753247261,1.3875764608383179>;"：
The Qualcomm ML post-processing plugin parses the PoseNet raw tensor output to obtain the coordinates of human keypoints. Threshold is confidence threshold, above which it is considered a valid keypoint. Keep a maximum of 2 results. The algorithm module is named posenet. The label file is posenet_mobililenet_v1.labels. q-offsets, q-scales are quantization parameters (mean/scaling factor) used to restore the quantized value of int8 output by the network to floating-point coordinates. - text/x-raw ! queue ! metamux.:
Encapsulate keypoint results into a simple text meta (actually GstMeta internally). Return meta to the previously named qtimetamux component (matched by pad name), and have it paste meta onto the corresponding video frame.