Use Case

On-Device Machine Learning and Autodiff

Automatic differentiation was supposed to require a garbage-collected runtime. numx proves otherwise. Forward mode runs with dual numbers that fit in two floats. Reverse mode uses a static tape you size at compile time. Exact gradients. On a chip with 512 KB of RAM.

Forward mode: dual numbers

A dual number is two floats: a value and a derivative. All arithmetic automatically propagates both. No tape, no allocation. The entire derivative computation lives on the stack.

Best when: few inputs, computing the derivative of a scalar function. Sensor calibration, PID tuning, single-parameter sensitivity.

#include "numx/numx.h"

/* Forward mode: exact derivative of a scalar function */
static numx_dual_t activation(numx_dual_t x) {
    /* ReLU-like: f(x) = x^2 when x > 0, 0 otherwise */
    if (x.re > 0.0f) {
        return numx_dual_mul(x, x);
    }
    return numx_dual_const(0.0f);
}

int main(void) {
    numx_real_t val, grad;
    numx_ad_dual_eval(activation, 2.5f, &val, &grad);
    /* val = 6.25, grad = 5.0 (2*x at x=2.5) */
    return 0;
}

Reverse mode: static tape

The tape records operations during the forward pass. One backward pass computes gradients with respect to all inputs simultaneously. This is backpropagation without a heap.

Best when: many inputs, one scalar output (loss function). On-device model fine-tuning, gradient-based parameter adaptation.

#include "numx/numx.h"

#define TAPE_CAP 128
static numx_tape_node_t tape_buf[TAPE_CAP];
static numx_tape_t tape;

/* Compute gradient of a 3-parameter model */
int main(void) {
    numx_ad_tape_init(&tape, tape_buf, TAPE_CAP);

    /* Model parameters */
    numx_ad_var_t w0 = numx_ad_var(&tape, 0.5f);
    numx_ad_var_t w1 = numx_ad_var(&tape, -1.2f);
    numx_ad_var_t w2 = numx_ad_var(&tape,  0.8f);

    /* Input features (constants, not differentiated) */
    numx_ad_var_t x0 = numx_ad_const(&tape, 1.0f);
    numx_ad_var_t x1 = numx_ad_const(&tape, 0.5f);
    numx_ad_var_t x2 = numx_ad_const(&tape, 2.0f);

    /* Forward: loss = (w0*x0 + w1*x1 + w2*x2 - target)^2 */
    numx_ad_var_t target = numx_ad_const(&tape, 1.0f);
    numx_ad_var_t pred   = numx_ad_add(&tape,
        numx_ad_add(&tape,
            numx_ad_mul(&tape, w0, x0),
            numx_ad_mul(&tape, w1, x1)
        ),
        numx_ad_mul(&tape, w2, x2)
    );
    numx_ad_var_t err  = numx_ad_sub(&tape, pred, target);
    numx_ad_var_t loss = numx_ad_mul(&tape, err, err);

    /* Backward: exact gradients */
    numx_ad_backward(&tape, loss);

    numx_real_t dw0 = numx_ad_grad(&tape, w0);
    numx_real_t dw1 = numx_ad_grad(&tape, w1);
    numx_real_t dw2 = numx_ad_grad(&tape, w2);

    /* One gradient descent step */
    float lr = 0.01f;
    /* w0 -= lr * dw0; ... */
    return 0;
}

autodiff module docs →

Sparse signal recovery: compressed_sensing

On bandwidth-constrained sensors that can only transmit a fraction of their measurements, compressed sensing recovers the full signal from the compressed observations. The compressed_sensing module runs OMP and ISTA directly on the device, no cloud round-trip needed.

compressed_sensing docs →