Skill

Pass

Audit score 90

domain-ml

actionbook/rust-skills

Build ML/AI applications in Rust with efficient tensor operations, GPU acceleration, and model inference.

What is domain-ml?

This skill provides domain constraints and design patterns for machine learning and AI applications in Rust. Use it when building inference servers, training pipelines, or data processing systems that require efficient memory usage, GPU utilization, and model portability.

Design memory-efficient tensor operations using zero-copy references and views
Implement GPU-accelerated inference with batching for optimal throughput
Load and cache ML models once using singleton patterns (OnceLock)
Handle standard model formats (ONNX) for portability between training and deployment
Build async data pipelines to parallelize loading and computation
Apply streaming and batching patterns for large-scale data processing

How to install domain-ml

npx skills add https://github.com/actionbook/rust-skills --skill domain-ml

Claude Code

Cursor

Windsurf

Cline

How to use domain-ml

1.Identify your use case: inference-only (use tract), training+inference (use candle/burn), or PyTorch models (use tch-rs)
2.Load your model once at startup using OnceLock to avoid repeated initialization overhead
3.Structure input data into batches to maximize GPU utilization and throughput
4.Use ndarray views and references instead of cloning tensors to minimize memory copies
5.Implement async data loading pipelines to keep GPU busy while preparing next batch
6.Apply streaming iterators for datasets larger than available memory

Use cases

Good for

Building inference servers that load ONNX models once and serve predictions with batched requests
Processing large datasets with streaming iterators to avoid memory copies
Deploying PyTorch models trained in Python as efficient Rust services using tch-rs or tract
Implementing GPU-accelerated ML pipelines with async data loading parallel to compute
Creating reproducible ML systems with seeded randomness and deterministic operations

Who it's for

Rust backend engineers building ML inference services
Data engineers optimizing ML data pipelines for performance
ML practitioners deploying Python-trained models to production Rust systems
Systems programmers requiring memory-efficient tensor operations

domain-ml FAQ

Should I use tract, candle, burn, or tch-rs?

Use tract for lightweight ONNX inference, candle/burn for pure-Rust training+inference, and tch-rs for direct PyTorch model bindings.

How do I avoid memory overhead with large tensors?

Use ndarray views and references instead of cloning, apply in-place operations, and stream data through iterators rather than loading everything into memory.

Why is batching important for GPU inference?

GPU kernels have launch overhead; batching amortizes this cost across multiple samples, significantly improving throughput compared to single-sample inference.

How should I load ML models in a server?

Use OnceLock or similar lazy-initialization patterns to load the model once at startup and reuse it across all requests, avoiding expensive reloads.

Can I use Python-trained models in Rust?

Yes—export models to ONNX format and load with tract or candle, or use tch-rs for direct PyTorch model bindings.

Full instructions (SKILL.md)

Source of truth, from actionbook/rust-skills.

name: domain-ml description: "Use when building ML/AI apps in Rust. Keywords: machine learning, ML, AI, tensor, model, inference, neural network, deep learning, training, prediction, ndarray, tch-rs, burn, candle, 机器学习, 人工智能, 模型推理" user-invocable: false

Machine Learning Domain

Layer 3: Domain Constraints

Domain Constraints → Design Implications

Domain Rule	Design Constraint	Rust Implication
Large data	Efficient memory	Zero-copy, streaming
GPU acceleration	CUDA/Metal support	candle, tch-rs
Model portability	Standard formats	ONNX
Batch processing	Throughput over latency	Batched inference
Numerical precision	Float handling	ndarray, careful f32/f64
Reproducibility	Deterministic	Seeded random, versioning

Critical Constraints

Memory Efficiency

RULE: Avoid copying large tensors
WHY: Memory bandwidth is bottleneck
RUST: References, views, in-place ops

GPU Utilization

RULE: Batch operations for GPU efficiency
WHY: GPU overhead per kernel launch
RUST: Batch sizes, async data loading

Model Portability

RULE: Use standard model formats
WHY: Train in Python, deploy in Rust
RUST: ONNX via tract or candle

Trace Down ↓

From constraints to design (Layer 2):

"Need efficient data pipelines"
    ↓ m10-performance: Streaming, batching
    ↓ polars: Lazy evaluation

"Need GPU inference"
    ↓ m07-concurrency: Async data loading
    ↓ candle/tch-rs: CUDA backend

"Need model loading"
    ↓ m12-lifecycle: Lazy init, caching
    ↓ tract: ONNX runtime

Use Case → Framework

Use Case	Recommended	Why
Inference only	tract (ONNX)	Lightweight, portable
Training + inference	candle, burn	Pure Rust, GPU
PyTorch models	tch-rs	Direct bindings
Data pipelines	polars	Fast, lazy eval

Key Crates

Purpose	Crate
Tensors	ndarray
ONNX inference	tract
ML framework	candle, burn
PyTorch bindings	tch-rs
Data processing	polars
Embeddings	fastembed

Design Patterns

Pattern	Purpose	Implementation
Model loading	Once, reuse	`OnceLock<Model>`
Batching	Throughput	Collect then process
Streaming	Large data	Iterator-based
GPU async	Parallelism	Data loading parallel to compute

Code Pattern: Inference Server

use std::sync::OnceLock;
use tract_onnx::prelude::*;

static MODEL: OnceLock<SimplePlan<TypedFact, Box<dyn TypedOp>, Graph<TypedFact, Box<dyn TypedOp>>>> = OnceLock::new();

fn get_model() -> &'static SimplePlan<...> {
    MODEL.get_or_init(|| {
        tract_onnx::onnx()
            .model_for_path("model.onnx")
            .unwrap()
            .into_optimized()
            .unwrap()
            .into_runnable()
            .unwrap()
    })
}

async fn predict(input: Vec<f32>) -> anyhow::Result<Vec<f32>> {
    let model = get_model();
    let input = tract_ndarray::arr1(&input).into_shape((1, input.len()))?;
    let result = model.run(tvec!(input.into()))?;
    Ok(result[0].to_array_view::<f32>()?.iter().copied().collect())
}

Code Pattern: Batched Inference

async fn batch_predict(inputs: Vec<Vec<f32>>, batch_size: usize) -> Vec<Vec<f32>> {
    let mut results = Vec::with_capacity(inputs.len());

    for batch in inputs.chunks(batch_size) {
        // Stack inputs into batch tensor
        let batch_tensor = stack_inputs(batch);

        // Run inference on batch
        let batch_output = model.run(batch_tensor).await;

        // Unstack results
        results.extend(unstack_outputs(batch_output));
    }

    results
}

Common Mistakes

Mistake	Domain Violation	Fix
Clone tensors	Memory waste	Use views
Single inference	GPU underutilized	Batch processing
Load model per request	Slow	Singleton pattern
Sync data loading	GPU idle	Async pipeline

Trace to Layer 1

Constraint	Layer 2 Pattern	Layer 1 Implementation
Memory efficiency	Zero-copy	ndarray views
Model singleton	Lazy init	OnceLock<Model>
Batch processing	Chunked iteration	chunks() + parallel
GPU async	Concurrent loading	tokio::spawn + GPU

Related Skills

When	See
Performance	m10-performance
Lazy initialization	m12-lifecycle
Async patterns	m07-concurrency
Memory efficiency	m01-ownership

Related skills

More from actionbook/rust-skills and the wider catalog.

domain-web

actionbook/rust-skills

Build concurrent, type-safe web services with async handlers, middleware, and shared state patterns.

2.8k installsAudited

m01-ownership

actionbook/rust-skills

Diagnose and fix Rust ownership, borrowing, and lifetime errors by understanding data roles and scope boundaries.

3.0k installsAudited

m02-resource

actionbook/rust-skills

Master Rust smart pointers and resource management for correct ownership patterns.

3.0k installsAudited

m03-mutability

actionbook/rust-skills

Resolve Rust mutability conflicts by understanding ownership, borrowing rules, and interior mutability patterns.

3.0k installsAudited

m04-zero-cost

actionbook/rust-skills

Master generics, traits, and zero-cost abstraction—choose static vs. dynamic dispatch correctly.

3.0k installsAudited

m05-type-driven

actionbook/rust-skills

Use Rust's type system to make invalid states unrepresentable at compile time.

3.0k installsAudited