API Documentation

Purpose of moco

moco takes machine learning models and optimizes them for computational efficiency with no accuracy loss.

moco analyzes data to find early-exit strategies to exploit:

• moco analyzes the input data or the latent representations of the data according to the model. moco cross-correlates the correlations found in the input data/latent representations with the eventual model predictions to find rules; these rules are then used to early-exit the model.

Form Factor

moco is a pip-installable Python package that offers a helper library that wraps models in early-exit gates. It is supported by a set of API endpoints, available remotely that offer proprietary algorithms for finding rules.

Early-Exiting

The early-exit algorithm involves an initial analysis step to analyze an embedding and a set of corresponding predictions. The analysis algorithm outputs a hyperplane that on one side is homogenous with respect to a certain class. At runtime, an input comes in and if the input is on the + side of the hyperplane, the corresponding class is outputted otherwise the original model is executed. Savings come from the fact that the full model is not always executed.