Classical Implementations

Long-Context KV-Cache Compression — NVIDIA Nemotron-H Family

Phoenix Attention is a drop-in long-context serving layer for the NVIDIA Nemotron-H hybrid family (8B–120B). It compresses each attention layer's KV cache while preserving retrieval, so a single node serves far more concurrent long-context requests — no retraining, no model changes. Verified on 8× H100. Support for Llama and other transformer families is in active development.

Nemotron-3-Super-120B — KV Compression & Concurrency, 8× H100

Full Nemotron-H Family — Retrieval Preserved

Quality Preservation

Measured on 8× H100 80 GB (tensor-parallel, bf16). KV compression is the attention-layer cache reduced to a fixed budget; concurrency is additional simultaneous long-context requests per node at preserved single-fact recall. Support for Llama and other transformer families is in active development.

CUDA SVD Engine — Exact, GPU-Accelerated

A fp64 CUDA singular-value decomposition engine for weight decomposition, model compression, and LoRA / PiSSA initialization. Drop-in torch.linalg.svd-compatible API — exact, not approximate.

Real Production Model Weights — GPU-Fair, NVIDIA H100 (fp64)

Measured on NVIDIA H100 80 GB, fp64, on real production weight matrices, against torch.linalg.svdvals — the like-for-like singular-value baseline. Dominant singular values match torch to 10⁻⁹–10⁻¹⁴. 9–19× on real attention/FFN weights (up to ~25× against a full SVD).

Scale — Large fp64 Matrices, NVIDIA H100

Measured on NVIDIA H100 80 GB, fp64. At 1M×4,096 (34.4 GB) the torch GPU SVD runs out of memory; Phoenix completes in 1.4 s. Dominant singular values match torch to ~10⁻¹².

Cross-Domain — Single-Cell Genomics PCA, 1M-Cell Atlas

1,000,000-cell single-cell atlas, top-50 PCA, fp64. 180× vs CPU scikit-learn, machine-exact — matching the GPU randomized method's speed while staying exact, where exact CPU SVD is intractable and the GPU randomized method carries ~10⁻² error. The SVD primitive carries outside ML weights: single-cell PCA, signal processing, recommendation systems — anywhere SVD is on the critical path.

Nemotron-H Serving Economics — More Requests per GPU

Phoenix Attention's KV-cache compression on NVIDIA's Nemotron-3-Super-120B flagship, measured against the unmodified stock configuration on 8× H100 80 GB. The compressed cache is what lets a node hold more long-context requests at once — the lever that lowers cost per request. Drop-in vLLM deployment, no retraining; every number reproducible from a single script.

Concurrent Long-Context Requests per Node (recall preserved)

The advantage grows with context: at 256K, 2.27× more concurrent requests per node is roughly 56% fewer GPUs for the same long-context serving load. Under sustained long-decode saturation the realized throughput gain reaches 3.32×.

Measured on 8× H100 80 GB, tensor-parallel, vLLM. Retrieval preserved (RULER niah 100, multikey 100, multivalue 97.5); quality matches the unmodified model (MMLU, GSM8K); prefill bit-exact.

Standard Transformers — In Active Development

We are extending the long-context KV technology from the hybrid Nemotron-H family to standard transformer models such as Meta Llama. Standard transformers place different demands on the cache, and our in-development approach preserves full long-context retrieval under aggressive KV compression.

Early Validation — Llama-3.1-8B, Long-Context Retrieval

Early validation on Llama-3.1-8B at 60K context, 12-prompt needle-retrieval benchmark (8× H100). At the same memory footprint where commodity 3-bit KV quantization drops roughly 1 in 6 long-context requests, our in-development method preserves full retrieval. In active development — we are seeking compute and funding to harden and scale transformer support.

Constant-Memory Long Context — NVIDIA Nemotron-H-4B

Aleph gives an LLM a fixed-size internal memory, so serving cost stops scaling with context: per-request memory stays flat no matter how long the input grows, instead of a KV cache that grows with every token. Trained into NVIDIA Nemotron-H-4B — a natively 8K-context model — Aleph extends its long-context retrieval out to 128K tokens at flat memory cost, with the base model frozen so its reasoning and generation are unchanged. Support for the transformer-hybrid conversions is in active development.

Official RULER — Long-Context Retrieval, Nemotron-H-4B

Per-Request Memory — Flat vs Growing KV Cache

Base Model Preserved

Measured on NVIDIA Nemotron-H-4B, 8× H100 80 GB (bf16). RULER is the published long-context retrieval benchmark (official data and scoring). Nemotron-H-4B is a natively-8K model; Aleph extends its retrieval to 128K at constant per-request memory, with the base model frozen and its behavior unchanged. Validated at research scale (4B); support for transformer-hybrid models is in active development.

GPU-Native Vector Search — Faster Than FAISS

VOIS is our patented GPU-native similarity search engine. Benchmarked head-to-head against Meta's FAISS on the same machine, same dataset, same ground truth.

Head-to-Head at Matched Recall — SIFT-1M (1 Million Vectors)

Scaling — VOIS QPS at ~97% Recall

VOIS — Recall vs Throughput Sweep (1M Vectors)

FAISS Comparison Data — 1M Vectors

SIFT-1M dataset: 128 dimensions, 1,000 queries, K=10 nearest neighbors. All methods on the same RTX 4060 (8GB), same data, same ground truth. VOIS uses a single GPU. FAISS HNSW tested at 1 and 16 CPU threads. Best of 5 runs.

PX Compute — Systems-Level Memory Pool for GPU and CPU

A slab-based memory pool allocator for NVIDIA GPUs and Linux CPUs. Per-device size-class free lists, golden-ratio block growth for the long tail, NVML-aware adaptive thresholds, and a hard pool-size cap to prevent runaway VRAM retention. Engineered for workloads that re-cycle buffers in tight loops — the alloc/free pattern LLM KV caches exhibit is exactly what this allocator targets.

PX Compute vs cudaMalloc — NVIDIA H100 80GB (single GPU)

Cold (non-reuse) workloads: 1.7–3.2× consistent. The huge speedups on reuse patterns are the relevant case: LLM inference re-cycles KV cache buffers, attention scratch buffers, and gradient buffers in tight loops — that's where PX Compute lands.

8× H100 in Parallel — Driver-Lock Contention Amplifies the Win

Under multi-worker inference, 8 processes contending on the NVIDIA driver lock serialize through cudaMalloc. PX Compute serves from in-process free lists with zero driver calls — the advantage grows under parallel load. This is the production-realistic regime for cloud LLM serving.

CPU Memory Pool — Large-Block Reuse

Linux CPU pool. Wins concentrate on large-block reuse where libc has to mmap/munmap. Real use cases: CPU-offloaded LLM weight buffers, large embedding lookup tables, distributed-training intermediate buffers.

Every number above is reproducible from a single bench script on stock NVIDIA Brev or AWS hardware. JSON manifests of per-workload timings available under NDA.

Aegis-EC — Training-Integrity Codec

Catches silent data corruption (SDC) in long training runs the moment it happens — across weights, gradients, and checkpoints — so a corrupted run restarts from the last clean checkpoint instead of being found broken at the end, after the compute is spent. Runs inline every N steps / per checkpoint; a full-model scan costs a small fraction of one training step.

Detection & Correction — vs SECDED

GPU Throughput — NVIDIA H100 80 GB

Measured on NVIDIA H100 80 GB. Detect: 27.8 GB/s on CPU, ~2.25 TB/s GPU (67% of HBM peak). Correct: word-level correct + double-word detect, 0.0000% miscorrect over 200K trials per lane. The underlying coding is standard; the asset is the measured, GPU-accelerated engine packaged for training integrity.

LeanStack — Efficiency-First LLM Architecture

LeanStack is an efficiency-first LLM architecture built to pair with our inference and reliability stack. At matched parameter count it holds or beats a standard-attention baseline on quality while using a smaller long-context KV-cache footprint. Validated from 38M to 3B parameters. In active development — we are seeking compute and funding to train at higher token budgets and larger model sizes, where the efficiency gains compound.

60+ Identified Application Domains

Because the framework operates at a fundamental mathematical level, it applies across computational domains. We have identified over 60 specific application areas spanning search, security, optimization, signal processing, defense, energy, medical imaging, telecommunications, and autonomous systems. These are covered by our patent claims.

Detailed application analysis and domain-specific benchmarks available under NDA.