Low-Latency, Low-Energy Models

moco: makes rate-limited and energy-limited classification ML models 15-30% more efficient.

Use Cases

How it works:

As input, moco takes data and the model’s classifications.
In turn, it clusters the data and identifies groups of data points that have the same prediction.
Then, classifiers are fit on these clusters.
At runtime, the new system first checks if a data point is within any of the clusters. If it is, the model outputs the relevant prediction. If not, the model runs as usual.

The basic Python usage (with an explanation) is the following:


from sklearn.neural_network import MLPClassifier
from moco import LoggedFunction

# Train your model.
m = MLPClassifier()
m.fit(X, y)

# Wrap your predictor
lf = LoggedFunction(m.predict)

# Run your data through the wrapped model prediction function.
# Internally, we collect the input data as well as the output.
out = lf(X)

# We identify natural clusters in X that are entirely one class or another
# and construct rules based on these clusters.
rules = lf.optimize()

# Now when we run,
lf(X)

# the original model is only run when none of the rules indicate certainty in prediction.

Notebooks

Natural Language Processing (NLP)

Accelerating TinyBERT Classification on IMDB dataset Achieve 21.5% reduction in latency & throughput on the IMDB movie review dataset with the transformer architecture, with no accuracy loss.

Tabular

Credit Card Fraud Detection Latency improvement: Achieving a 1.27x speed-up on wide and deep MLPs, sklearn, batched.
Credit Card Fraud Detection XGBoost Achieve ~-20% latency on XGBoost models.
Rapid Breast Cancer Diagnosis with sensor data: Achieving a 2.4x speed-up on a wide MLP, h = 2048 in-parallel, as well as >1.5 cost-savings across model sizes in the real-time setting.

Computer Vision (CV)

Rapid Handwritten Digit Classification Achieves a 20% latency improvement on the canonical MNIST dataset with a CNN model architecture.

Achieving Faster Models

📢 Stay Tuned: I am building a software package. Indicate interest here.

🚀 Try it on your model — Contact me

View the project on GitHub at moco-client. PyPi package coming soon for you to try out.

About Me

My name is Sam Randall. I attended Stanford University for my MS in Computational and Mathematical Engineering and my Bachelors in Applied Mathematics and Public Health from Johns Hopkins University. My research interests are in computational geometry, graph theory and their applications to machine learning.

I seek to make sustainability profitable and our world healthier so I am especially interested in partnering with organizations that are working to deploy models in very resource-constrained settings on edge devices to promote health and sustainability.