⚡ moco

Efficient ML

Problem: Machine learning models can be too slow, require a lot of energy, and are thus expensive in inference.

Root Problem: Machine Learning Models are far from computationally efficient on their expected data distribution.

Solution: I am building algorithms to take trained machine learning models and make them more computationally efficient when running in inference. This results in more energy-efficient models, faster models, and models that require less infra, resulting in cheaper systems.

Results: I’ve achieved 20% reductions in inference speed on image, text and tabular classification models.

Use Cases

Natural Language Processing (NLP)

Accelerating TinyBERT Classification on IMDB dataset Achieve 21.5% reduction in latency & throughput on the IMDB movie review dataset with the transformer architecture, with no accuracy loss.

Tabular

Credit Card Fraud Detection Latency improvement: Achieving a 1.27x speed-up on wide and deep MLPs, sklearn, batched.
Credit Card Fraud Detection XGBoost Achieve ~-20% latency on XGBoost models.
Rapid Breast Cancer Diagnosis with sensor data: Achieving a 2.4x speed-up on a wide MLP, h = 2048 in-parallel, as well as >1.5 cost-savings across model sizes in the real-time setting.

Computer Vision (CV)

Rapid Handwritten Digit Classification Achieves a 20% latency improvement on the canonical MNIST dataset with a CNN model architecture.

Achieving Faster Models

Quantization, pruning, and knowledge distillation all involve achieving acceleration at the cost of accuracy.

Hardware acceleration with GPUs, faster CPUs or parallelizing ML models on many machines involve more money, infrastructure cost and energy cost.

Use moco to find subsets of data that are easy to classify early in the model, so the model does not need to execute fully for every single data point.

📢 Stay Tuned: I am building a software package. Indicate interest here.

🚀 Try it on your model — Contact me

View the project on GitHub at moco-client. PyPi package coming soon for you to try out.

About Me

I use my background in graph theory and topological data analysis to develop algorithms built into software that data scientists and ML performance engineers can use to optimize latency-critical models.