robertcprice/nCPU: nCPU: model-native and tensor-optimized CPU research runtimes with organized workloads, tools, and docs · GitHub

A CPU that runs entirely on GPU — registers, memory, flags, and program counter are all tensors.
Every ALU operation is a trained neural network.
Addition uses Kogge-Stone carry-lookahead. Multiplication uses a learned byte-pair lookup table.
Bitwise ops use neural truth tables. Shifts use attention-based bit routing. No hardcoded arithmetic.

pip install -e ".[dev]"

# Run a program — all arithmetic through trained neural networks
python main.py --program programs/sum_1_to_10.asm

# Run with execution trace
python main.py --program programs/fibonacci.asm --trace

# Inline assembly
python main.py --inline "MOV R0, 42; HALT"

# GPU tensor mode (maximum speed, native tensor ops)
python main.py --binary firmware.bin --fast

The entire CPU lives on GPU. Registers, memory, flags, and the program counter are
PyTorch tensors. Instruction decode, ALU dispatch, and state updates all happen on-device
— nothing round-trips to the host CPU. Every ALU operation routes through a trained .pt model:

Math functions (sin, cos, sqrt, exp, log, atan2) also wired through trained models.

Result: 100% accuracy on integer arithmetic, verified by 347 automated tests.

23 models (~135 MB total), 13 actively wired:

See models/MODEL_INDEX.md for full details.

Benchmarked on Apple Silicon (MPS backend, PyTorch 2.10.0), 1,000 iterations per operation:

All models load in 60ms. Programs execute at 136–262 us/cycle depending on instruction mix (~4,975 IPS).

# Run benchmarks
python benchmarks/benchmark_neural.py

Multiplication is 12x faster than addition, even with carry-lookahead. In conventional CPUs,
MUL is slower than ADD. In the neural CPU, it’s inverted: the byte-pair LUT (21 us) has zero
sequential dependency, while the CLA adder (248 us) requires O(log n) carry-combine stages.
Before CLA, the gap was 38x (826 us with 32 ripple-carry passes).

Carry-lookahead works in neural networks. The Kogge-Stone parallel-prefix algorithm,
using a trained carry-combine network (100% accuracy on all 16 inputs), reduced ADD/SUB/CMP
from ~826 us to ~248 us — a 3.3x speedup. Classical hardware design principles transfer
directly to neural architectures.

Vectorization recovers most of the attention cost. Shift operations went from ~2,833 us
(64 sequential passes) to ~434 us (3 batched passes) — a 6.5x speedup.

O(1) / O(log n) / O(n) hierarchy. Operations fall into three tiers: O(1) single-pass
lookups (~21 us), O(log n) parallel-prefix carry (~248 us), and O(n) sequential passes
(sqrt ~522 us, atan2 ~935 us).

See the research paper for detailed analysis.

The NeuralCPU is not a simulator that happens to use a GPU — it is a GPU program.
All CPU state lives permanently on-device as PyTorch tensors:

• Registers: 31 x 64-bit (torch.int64 on GPU)
• Memory: Flat byte-addressable (torch.uint8 on GPU)
• Flags: N, Z, C, V as GPU-resident tensors
• Program Counter: GPU tensor, incremented on-device

The CPU is 64-bit ARM64. Registers, addresses, and data paths are all 64-bit.
Instruction fetch, decode, execute, and writeback all happen on GPU. The ALU is a bank
of trained neural networks running as GPU inference. No host CPU arithmetic in the
execution loop.

Neural Mode (default) — Every ALU operation is a forward pass through a trained .pt model:

from ncpu.model import CPU
cpu = CPU(neural_execution=True)
cpu.load_program("MOV R0, 7\nMOV R1, 6\nMUL R2, R0, R1\nHALT")
cpu.run()
print(cpu.get_register("R2"))  # 42 (computed by neural byte-pair LUT)

Fast Mode (--fast) — Same GPU-resident architecture, but ALU uses torch.add/torch.mul
instead of model inference. Targets 1.35M IPS at batch_size=32768 on Apple Silicon MPS:

from ncpu.neural import NeuralCPU
cpu = NeuralCPU(fast_mode=True)  # Native GPU tensor ops
cpu.load_binary(arm64_binary)

For maximum throughput, the kernels/ directory contains native Metal GPU implementations
that run the entire ARM64 fetch-decode-execute loop on GPU with zero CPU-GPU synchronization:

• MLX Metal (kernels/mlx/): Python interface to custom Metal compute kernels via Apple MLX.
  Full ARM64 decode and execute in Metal Shading Language.
• Rust Metal (kernels/rust_metal/): Direct Metal API via objc2-metal with PyO3 Python
  bindings. Includes GPU-side syscall handling, basic block caching, neural dispatch, and
  out-of-order execution. Ships with a native DOOM benchmark.

Full ARM64 instruction set — real binary encoding. The NeuralCPU decodes and executes
real ARM64 instructions with GPU-resident state.

nCPU/
├── ncpu/
│   ├── neural/           # Full GPU neural CPU (ARM64, 12K lines)
│   │   ├── cpu.py        # NeuralCPU — all state on GPU as tensors
│   │   └── neural_alu_bridge.py  # Routes ops through trained models
│   ├── model/            # Model-based CPU (text assembly)
│   │   ├── cpu.py        # CPU orchestrator
│   │   ├── neural_ops.py # Loads and runs .pt models
│   │   └── architectures.py  # All model class definitions
│   └── tensor/           # Tensor-based ARM64 kernel
├── kernels/
│   ├── mlx/              # MLX Metal compute kernels (Python + Metal Shading Language)
│   └── rust_metal/       # Rust Metal GPU kernel (objc2-metal + PyO3 bindings)
│       └── src/          # Full ARM64 execute loop in Metal, zero GPU-CPU sync
├── models/               # 23 trained .pt models
│   ├── alu/              # arithmetic, carry_combine, multiply, divide, logical, compare
│   ├── shifts/           # lsl, lsr, asr, rol
│   ├── math/             # sincos, sqrt, exp, log, atan2, doom_trig
│   ├── decode_llm/       # Qwen2.5 LoRA adapter
│   └── MODEL_INDEX.md    # Complete model status
├── demos/                # DOOM raycaster and other demos
├── programs/             # Assembly programs (.asm)
├── tests/                # 347 tests
├── docs/                 # Architecture docs (neural mode + tensor mode)
├── benchmarks/           # Performance benchmarks
├── paper/                # Research paper
└── main.py               # CLI entry point

A DDA raycaster that runs all arithmetic through trained neural networks. Every ADD, SUB, MUL,
and CMP is a forward pass through a real .pt model.

# Neural mode — every op through trained models (~2.5 FPS)
python demos/doom_raycaster.py

# Fast mode — native Python arithmetic (~5,000 FPS)
python demos/doom_raycaster.py --fast

# Side-by-side comparison (verifies identical output)
python demos/doom_raycaster.py --both

Fixed-point arithmetic (scale 1024) keeps everything in 32-bit integers, matching the
nCPU’s integer-only ISA. Both modes execute identical algorithms and produce identical
frame output — the only difference is whether arithmetic routes through neural networks.

pytest tests/ -v
# 347 tests: decode, programs, registry, state, neural ops (incl. CLA + batch),
#            neural bridge, math ops, architecture forward-pass, division

MIT