Qualcomm Model Deployment

Qualcomm provides the qai_hub_models Python library. It enables developers to easily perform model conversion and quantization, and export the optimized models as BIN files that can be directly loaded by Qualcomm NPU. The library also supports model inference and validation using Qualcomm's online virtual devices.

Preparations

1. Install qai_hub_models.

pip3 install qai_hub_models

2. Configure API Token.

export PATH=~/.local/bin/:$PATH
qai-hub configure --api_token <API_TOKEN>

Model Core Information

The models supported by qai_hub_madels include Computer Vision, Multinodal, Audio, and Generation. Please refer to the table below for details.

Computer Vision

Multimodal

Audio

Generation

Model Compilation

The details of compilation take real_esrgan-x4plus as an example:

export PRODUCT_CHIP=qualcomm-qcs6490-proxy
python3 -m qai_hub_models.models.real_esrgan_x4plus.export --chipset ${PRODUCT_CHIP} \
        --target-runtime qnn_context_binary  --height 128 --width 128 --quantize w8a8 \
        --num-calibration-samples 10

• --chipset: Specifies the target chip to run on
• --target: Specifies the target runtime
• --Height: Target model input height
• --width: Target model input width
• --quantize: Specifies the quantization method
• --num-calibration-samples: Specifies the number of quantization calibration set images

The above command generates a mode-id and a file named real_ esrgan-x4plus. qnd_text-binary, which is the runtime model file for the target chipset.

Run Demo

python3 -m qai_hub_models.models.real_esrgan_x4plus.demo --eval-mode on-device --hub-model-id <mode-id> --chipset ${PRODUCT_CHIP}

• Please modify the placeholder mode-id following the hub-model-id parameter with the mode-id generated by the command above.

Use NPU of Local Device for Inference Validation

Please refer NPU Development Guide.

Q&A

• How to use models to develop app ？

  Qualcomm provides the ai-engine-direct-helper SDK, which includes Python and C++ interfaces for developing apps that can load models and perform inference. For details, please refer to ai-engine-direct-helper.