Transformer moderation systems processing billions of events per day.

• Typical models: BERT, RoBERTa, multimodal transformers
• Typical latency: 5–20ms
• Typical scale: 100k–750k+ requests/sec
• Typical annual inference cost: Millions/year
• 20% lower average FLOPs/inference can save hundreds of thousands to millions per year in GPU infrastructure and compute-time costs.
• Lower inference load reduces peak GPU utilization, enabling smaller serving fleets and higher throughput on existing hardware.

Large-scale indexing and retrieval pipelines operating on billions of assets.

• Typical models: BERT, CLIP, embedding models
• Typical latency: 5–50ms
• Typical scale: Billions of classifications/day
• Typical annual inference cost: Millions/year
• 20% lower average FLOPs/inference can save hundreds of thousands annually in indexing and embedding compute costs.
• Reduced compute per classification enables faster index refreshes, higher indexing throughput, and more models within the same infrastructure budget.

Real-Time and Batched Fraud Detection is latency constrained and compute-constrained respectively.

• Real-Time Fraud Detection Systems: moco unlocks compute budget to apply more analytics to the hard cases, increasing the accuracy of the system, saving false positives and false negatives.
• Batched Fraud Detection Systems: moco unlocks reduces runs the same inference with less physical hardware, saving money in compute-time and infra.

moco supports embedded inference pipelines where compute and power are constrained.

• Existing PyTorch models can be transformed into routed hierarchical classifiers.
• Expensive model execution is conditionally avoided for easy predictions.
• Reduced average compute-time enables lower-power inference deployments.
• Compatible with ONNX export and edge deployment workflows.

Edge inference systems benefit from lower power consumption and reduced average computation.

• Lower average FLOPs/inference extends battery life for always-on systems.
• Reduced compute demand lowers thermal load on constrained hardware.
• Smaller inference workloads enable more models to run on-device.

Latency-sensitive systems benefit from avoiding unnecessary expensive computation.

• Many production systems are bottlenecked by feature loading, graph queries, and deep model execution.
• moco helps determine when expensive computation can be skipped safely.
• Reduced infrastructure load enables smaller serving fleets and higher throughput on existing hardware.

Large offline inference jobs directly benefit from lower compute-time per prediction.

• Reduced average computation increases total system throughput.
• Lower GPU-hours reduce infrastructure and cloud compute costs.
• Existing hardware can process larger datasets without scaling infrastructure.