If you read my previous blog about Physical AI, you’ll know that QNX is enabling people to make their robots smart. I didn’t provide a lot of details about how though. If you’re easily bored by technical minutia, the TL;DR is that both QNX and the QNX community support a ton of packages that are used by engineers to add AI capability to their products and projects.
You do want the gory details? Great! This is a sampling of some of the packages available on https://oss.qnx.com that are AI-related. (Check back there regularly for updates…)
Self-hosted Packages
Available through Alpine-based APK packages that are buildable natively on QNX with the QNX Developer Desktop (QDD). QDD is our self-hosted environment, meaning that the developer tools actually run on the development target itself, instead of requiring a separate host computer for building the software that’s typical for embedded software. (For more information, check out QDD in our docs, which explains how to use it and where to get it.)
Core infrastructure (aports/core)
These form the base toolchain. While these aren’t AI-specific, lots of other pieces depend on them.
bash, binutils, bison, brotli, bzip2, cmake, curl, flex, gawk, gmp, gzip, icu, jansson, libarchive, libffi, libiconv, libuv, libxml2, llvm, lz4, make, meson, mpfr, ninja, openssl, perl, python3, readline, sqlite, xz, zlib, zlib-ng, zstd.
AI/ML/Vision relevant (aports/extra)
1
1
Package
--clr-nb-sand
center
middle
1
1
Category
--clr-nb-sand
center
middle
1
1
Notes
--clr-nb-sand
center
middle
1
1
abseil-cpp
--clr-b-light
center
middle
1
1
Foundational C++
--clr-b-light
center
middle
1
1
Google's base library — required by protobuf, grpc, RE2, ONNX Runtime, MediaPipe, TensorFlow.
--clr-b-light
center
middle
1
1
assimp
--clr-b-light
center
middle
1
1
3D / Vision
--clr-b-light
center
middle
1
1
3D asset importer; useful for scene understanding and 3D model loading in perception pipelines.
--clr-b-light
center
middle
1
1
boost
--clr-b-light
center
middle
1
1
Foundational C++
--clr-b-light
center
middle
1
1
Required by PCL, ROS 2 components, and many ML-adjacent libraries.
--clr-b-light
center
middle
1
1
ceres-solver
--clr-b-light
center
middle
1
1
Math / SLAM
--clr-b-light
center
middle
1
1
Non-linear least squares optimizer from Google; widely used in SLAM, camera calibration, bundle adjustment.
--clr-b-light
center
middle
1
1
cpuinfo
--clr-b-light
center
middle
1
1
Runtime dispatch
--clr-b-light
center
middle
1
1
Google's CPU feature detection; used by XNNPACK and TFLite to dispatch SIMD code paths at runtime.
--clr-b-light
center
middle
1
1
cython
--clr-b-light
center
middle
1
1
Runtime dispatch
--clr-b-light
center
middle
1
1
Google's CPU feature detection; used by XNNPACK and TFLite to dispatch SIMD code paths at runtime.
--clr-b-light
center
middle
1
1
double-conversion
--clr-b-light
center
middle
1
1
Foundational C++
--clr-b-light
center
middle
1
1
Fast float/string conversion; TensorFlow and abseil dependency.
--clr-b-light
center
middle
1
1
eigen
--clr-b-light
center
middle
1
1
Math
--clr-b-light
center
middle
1
1
Header-only linear algebra, the single most common dependency in ML C++ codebases. (Used by TFLite, ONNX Runtime, ceres, and PCL).
--clr-b-light
center
middle
1
1
fftw
--clr-b-light
center
middle
1
1
Signal processing
--clr-b-light
center
middle
1
1
Fast Fourier Transform, used in signal/sensor processing pipelines, audio ML, radar.
--clr-b-light
center
middle
1
1
flatbuffers
--clr-b-light
center
middle
1
1
Serialization
--clr-b-light
center
middle
1
1
FlatBuffers — the model format for TFLite files and used by MediaPipe.
--clr-b-light
center
middle
1
1
fmt
--clr-b-light
center
middle
1
1
Foundational C++
--clr-b-light
center
middle
1
1
Formatting library used by spdlog and many modern C++ ML codebases.
--clr-b-light
center
middle
1
1
geographiclib
--clr-b-light
center
middle
1
1
Localization
--clr-b-light
center
middle
1
1
Geographic coordinate conversions, used in GPS-fused localization and mapping pipelines.
--clr-b-light
center
middle
1
1
gflags
--clr-b-light
center
middle
1
1
Foundational
--clr-b-light
center
middle
1
1
Google command-line flags (TensorFlow and gRPC dependency).
--clr-b-light
center
middle
1
1
glib
--clr-b-light
center
middle
1
1
Foundational
--clr-b-light
center
middle
1
1
Base library for many Linux ports and indirect dependency for some CV tools.
--clr-b-light
center
middle
1
1
glfw
--clr-b-light
center
middle
1
1
Graphics
--clr-b-light
center
middle
1
1
Window/input for OpenGL; useful for vision debug visualization.
--clr-b-light
center
middle
1
1
glog
--clr-b-light
center
middle
1
1
Foundational
--clr-b-light
center
middle
1
1
Google logging library (TensorFlow, ONNX Runtime, ceres dependency).
--clr-b-light
center
middle
1
1
graphene
--clr-b-light
center
middle
1
1
Math / Graphics
--clr-b-light
center
middle
1
1
SIMD-optimized 2D/3D math types used by GStreamer and some CV pipelines.
--clr-b-light
center
middle
1
1
harfbuzz
--clr-b-light
center
middle
1
1
Text rendering
--clr-b-light
center
middle
1
1
Text shaping — useful for HMI/OCR pipelines that process rendered text.
--clr-b-light
center
middle
1
1
highway
--clr-b-light
center
middle
1
1
SIMD
--clr-b-light
center
middle
1
1
Google Highway — portable SIMD abstraction layer; used by libjpeg-xl, libavif, libwebp, and increasingly by ML libs as an alternative to raw intrinsics.
--clr-b-light
center
middle
1
1
jax
--clr-b-light
center
middle
1
1
ML Framework
--clr-b-light
center
middle
1
1
Google JAX — NumPy-compatible ML framework with XLA JIT.
--clr-b-light
center
middle
1
1
jsoncpp
--clr-b-light
center
middle
1
1
Serialization
--clr-b-light
center
middle
1
1
JSON parsing; used by OpenCV, TensorFlow, and many inference pipeline configs.
--clr-b-light
center
middle
1
1
lcms2
--clr-b-light
center
middle
1
1
Color / Vision
--clr-b-light
center
middle
1
1
Little CMS color management; used in image processing pipelines and camera calibration.
--clr-b-light
center
middle
1
1
libavif
--clr-b-light
center
middle
1
1
Codec / Vision
--clr-b-light
center
middle
1
1
AVIF image format; modern high-efficiency image codec for vision data pipelines.
--clr-b-light
center
middle
Cross-compile only
The port to QNX is validated (the cross-compile build works), although these modules haven’t been APK-packaged yet. (Again, as of this blog’s date – this work is constant and very active, so check on https://oss.qnx.com for the latest.)
Inference Runtimes
1
1
Package
--clr-nb-sand
center
middle
1
1
Source
--clr-nb-sand
center
middle
1
1
Notes
--clr-nb-sand
center
middle
1
1
onnxruntime
--clr-b-light
center
middle
1
1
Individual repo (MS fork)
--clr-b-light
center
middle
1
1
THE universal inference engine — runs models from PyTorch, TF, sklearn, and any ONNX-compatible framework.
--clr-b-light
center
middle
1
1
pytorch
--clr-b-light
center
middle
1
1
build-files/ports/pytorch
--clr-b-light
center
middle
1
1
Full PyTorch.
--clr-b-light
center
middle
1
1
openvino
--clr-b-light
center
middle
1
1
Individual repo (v2023.0.0)
--clr-b-light
center
middle
1
1
Intel's inference optimization toolkit — critical for x86 QNX platforms and Intel automotive SoCs (Tiger Lake, Alder Lake).
--clr-b-light
center
middle
1
1
mediapipe
--clr-b-light
center
middle
1
1
Individual repo (in-flight)
--clr-b-light
center
middle
1
1
Google's cross-platform ML pipeline framework; includes pre-built graphs for hand tracking, pose estimation, face mesh, object detection.
--clr-b-light
center
middle
1
1
xla
--clr-b-light
center
middle
1
1
Individual repo (in-flight)
--clr-b-light
center
middle
1
1
OpenXLA compiler — JIT compiles ML computations for CPUs, GPUs, and accelerators. Powers JAX (already self-hosted).
--clr-b-light
center
middle
ML Support Libraries
1
1
Package
--clr-nb-sand
center
middle
1
1
Source
--clr-nb-sand
center
middle
1
1
Notes
--clr-nb-sand
center
middle
1
1
numpy
--clr-b-light
center
middle
1
1
build-files/ports/numpy + numpy2
--clr-b-light
center
middle
1
1
Foundation for all Python ML.
--clr-b-light
center
middle
1
1
pandas
--clr-b-light
center
middle
1
1
build-files/ports/pandas
--clr-b-light
center
middle
1
1
DataFrame library; used in ML data preprocessing, feature engineering, training data pipelines.
--clr-b-light
center
middle
1
1
ruy
--clr-b-light
center
middle
1
1
Individual repo + build-files
--clr-b-light
center
middle
1
1
Google's high-performance matrix multiplication library; TFLite's primary CPU matrix backend.
--clr-b-light
center
middle
1
1
kineto
--clr-b-light
center
middle
1
1
Individual repo + build-files
--clr-b-light
center
middle
1
1
PyTorch CPU+GPU profiler. Useful for profiling ML workload performance on QNX targets.
--clr-b-light
center
middle
1
1
gloo
--clr-b-light
center
middle
1
1
Individual repo
--clr-b-light
center
middle
1
1
PyTorch's collective communications library for distributed inference.
--clr-b-light
center
middle
1
1
farmhash
--clr-b-light
center
middle
1
1
Individual repo + build-files
--clr-b-light
center
middle
1
1
Google's hash functions (and TensorFlow dependency).
--clr-b-light
center
middle
1
1
sleef
--clr-b-light
center
middle
1
1
build-files/ports/sleef
--clr-b-light
center
middle
1
1
Vectorized libm (SIMD math functions) used by XNNPACK and PyTorch for fast transcendentals.
--clr-b-light
center
middle
Math / Linear Algebra
1
1
Package
--clr-nb-sand
center
middle
1
1
Source
--clr-nb-sand
center
middle
1
1
Notes
--clr-nb-sand
center
middle
1
1
OpenBLAS
--clr-b-light
center
middle
1
1
build-files/ports/OpenBLAS
--clr-b-light
center
middle
1
1
OpenBLAS provides optimized BLAS/LAPACK for ARM NEON/SVE.
--clr-b-light
center
middle
1
1
clapack
--clr-b-light
center
middle
1
1
build-files/ports/clapack
--clr-b-light
center
middle
1
1
C-translated LAPACK; lighter alternative to full LAPACK.
--clr-b-light
center
middle
1
1
SuiteSparse
--clr-b-light
center
middle
1
1
build-files/ports/SuiteSparse
--clr-b-light
center
middle
1
1
Sparse matrix algorithms (CHOLMOD, UMFPACK, etc.); required by PCL, ceres-solver's sparse backend, and many SLAM solvers.
--clr-b-light
center
middle
1
1
METIS
--clr-b-light
center
middle
1
1
build-files/ports/METIS
--clr-b-light
center
middle
1
1
Graph partitioning library; SuiteSparse dependency.
--clr-b-light
center
middle
1
1
gtsam
--clr-b-light
center
middle
1
1
build-files/ports/gtsam
--clr-b-light
center
middle
1
1
Georgia Tech Smoothing and Mapping — factor graph optimizer used in SLAM, state estimation, and sensor fusion.
--clr-b-light
center
middle
1
1
oneTBB
--clr-b-light
center
middle
1
1
build-files/ports/oneTBB
--clr-b-light
center
middle
1
1
Intel Threading Building Blocks — parallel execution framework used by OpenVINO, ONNX Runtime, and Intel-optimized ML code.
--clr-b-light
center
middle
Computer Vision
1
1
Package
--clr-nb-sand
center
middle
1
1
Source
--clr-nb-sand
center
middle
1
1
Notes
--clr-nb-sand
center
middle
1
1
opencv
--clr-b-light
center
middle
1
1
Individual repo + build-files
--clr-b-light
center
middle
1
1
Full OpenCV 4.9.
--clr-b-light
center
middle
1
1
libyuv
--clr-b-light
center
middle
1
1
Individual repo + build-files
--clr-b-light
center
middle
1
1
YUV color format conversion; used in camera/video preprocessing pipelines feeding into ML inference.
--clr-b-light
center
middle
1
1
libcamera
--clr-b-light
center
middle
1
1
Individual repo + build-files
--clr-b-light
center
middle
1
1
Camera pipeline library (RPi variant); abstracts camera hardware and ISP for ML vision pipelines.
--clr-b-light
center
middle
1
1
FreeImage
--clr-b-light
center
middle
1
1
build-files/ports/FreeImage
--clr-b-light
center
middle
1
1
Multi-format image loading library; useful for loading training/test images in diverse formats.
--clr-b-light
center
middle
Serialization / Comms
1
1
Package
--clr-nb-sand
center
middle
1
1
Source
--clr-nb-sand
center
middle
1
1
Notes
--clr-nb-sand
center
middle
1
1
protobuf
--clr-b-light
center
middle
1
1
build-files/ports/protobuf
--clr-b-light
center
middle
1
1
Required by: ONNX Runtime, gRPC, TFLite .pb format, MediaPipe, Apache Arrow, OpenVINO.
--clr-b-light
center
middle
1
1
grpc
--clr-b-light
center
middle
1
1
build-files/ports/grpc
--clr-b-light
center
middle
1
1
High-performance RPC; used for ML model serving, TensorFlow Serving, ROS 2 service bridges, and inter-process inference pipelines.
--clr-b-light
center
middle
1
1
capnproto
--clr-b-light
center
middle
1
1
build-files/ports/capnproto
--clr-b-light
center
middle
1
1
Cap'n Proto — zero-copy serialization; faster than protobuf for some use cases; used in some ROS 2 and automotive middleware stacks.
--clr-b-light
center
middle
1
1
libzmq
--clr-b-light
center
middle
1
1
build-files/ports/libzmq
--clr-b-light
center
middle
1
1
ZeroMQ — high-performance async messaging; used in distributed sensor pipelines and some ML serving architectures.
--clr-b-light
center
middle
Sensor / Device
1
1
Package
--clr-nb-sand
center
middle
1
1
Source
--clr-nb-sand
center
middle
1
1
Notes
--clr-nb-sand
center
middle
1
1
libusb
--clr-b-light
center
middle
1
1
build-files/ports/libusb
--clr-b-light
center
middle
1
1
USB device access library. Needed by librealsense (Intel RealSense), OpenNI2, and any USB-attached sensor.
--clr-b-light
center
middle
1
1
libev
--clr-b-light
center
middle
1
1
build-files/ports/libev
--clr-b-light
center
middle
1
1
Lightweight event loop, used by libcamera and some sensor middleware.
--clr-b-light
center
middle
1
1
libevent
--clr-b-light
center
middle
1
1
build-files/ports/libevent
--clr-b-light
center
middle
1
1
Event notification library, used by some DDS and networking middleware.
--clr-b-light
center
middle
Other notable cross-compile ports
100px
1
1
Package
--clr-nb-sand
center
middle
1
1
Notes
--clr-nb-sand
center
middle
1
1
webrtc
--clr-b-light
center
middle
1
1
Full WebRTC stack — relevant for real-time video streaming from cameras to ML backends or remote monitoring.
--clr-b-light
center
middle
1
1
ComputeLibrary
--clr-b-light
center
middle
1
1
ARM Compute Library (individual repo) — optimized NEON/SVE2 kernels for CNN inference on Cortex-A processors.
--clr-b-light
center
middle
1
1
XNNPACK
--clr-b-light
center
middle
1
1
Google's highly optimized NN operator library; TFLite's primary inference backend.
--clr-b-light
center
middle
1
1
boringssl
--clr-b-light
center
middle
1
1
Google's TLS library fork; required by gRPC and some ML serving stacks.
--clr-b-light
center
middle
1
1
s2n-tls
--clr-b-light
center
middle
1
1
AWS TLS library; used by AWS ML SDKs.
--clr-b-light
center
middle
1
1
xtl
--clr-b-light
center
middle
1
1
xtensor type-list library; natural dependency for xtensor (N-dimensional arrays for C++ ML).
--clr-b-light
center
middle
1
1
zenoh-cpp / zenoh-pico
--clr-b-light
center
middle
1
1
Next-gen pub/sub protocol for robotics/edge; increasingly used as a DDS alternative in modern ROS 2 and AV stacks.
--clr-b-light
center
middle
1
1
perfetto
--clr-b-light
center
middle
1
1
Google's system tracing library; useful for profiling ML inference pipelines on QNX targets.
--clr-b-light
center
middle
1
1
pdfium
--clr-b-light
center
middle
1
1
PDF rendering — lower ML relevance but occasionally useful.
--clr-b-light
center
middle
Summary
Do you have something you need that isn’t in this list? Do you have your own QNX port of an important building block that we aren’t yet tracking? Have other feedback we should hear? Then reach out to us at everywhere@qnx.com or our Discord channel.