On Anytime Learning At Macroscale

Learning from sequential data dumps

(key) Requirements

Python 3.7
Pytorch 1.9.0
Hydra 1.1.0 (pip install hydra-core & pip install hydra-submitit-launcher)

Structure

├── crlapi           
  ├── benchmark.py    # Creates the data stream, feeds it to the model and evaluates it
  ├── core.py         # Abstract classes for 
  ├── logger.py   
  ├── sl
    ├── architectures
      ├── ...         # NN architectures used in this project
    ├── clmodels
      ├── ...         # Models (e.g. Single, gEns, ..., )
    ├── streams
      ├── ...         # CIFAR and MNIST stream implementatins

Running Experiments

To run experiments, you need to call the dataset specific run file, and you need to pass the configuration of the run. We have place the configurations in the previous directory (../configs). The config structure is as follows

    ├── configs
        ├── mnist
           ├── run.py                 # run file
           ├── test_usage_gmoe.yaml   # This is the "gMoE" model
           ├── test_finetune_mlp.yaml # This is the "Single Model"
           ... 
        ├── cifar
           ├── run.py                 # run file
           ├── test_finetune_vgg.yaml # This is the "Single Model"
           ├── test_usage_gmoe.yaml   # This is the "gMoE" model
           ...

To run an e.g. mnist gMoE run, the command is (launched from the directory just above (so cd ..)

PYTHONPATH=./ python configs/mnist/run.py -cn test_usage_gmoe n_megabatches=2 replay=1 clmodel.max_epochs=200

Important arguments

n_megabatches : controls the number of megabatches. So n_megabatches=1 is your regular full dataset training
replay : whether to use replay or not
clmodel.init_from_scratch : whether to reinitialize the model at every MB. Should only be used when replay=1
device : use cuda or cpu depending on your hardware

License

alma is released under the MIT license. See LICENSE for additional details about it. See also our Terms of Use and Privacy Policy.

Anytime Learning At Macroscale

Related tags

Overview

On Anytime Learning At Macroscale

(key) Requirements

Structure

Running Experiments

Important arguments

License

Owner

Meta Research

Predict the demand for electricity (R) - FRENCH

Implementation of different ML Algorithms from scratch, written in Python 3.x

Project to deploy a machine learning model based on Titanic dataset from Kaggle

Deploy AutoML as a service using Flask

inding a method to objectively quantify skill versus chance in games, using reinforcement learning

Book Recommender System Using Sci-kit learn N-neighbours

Python implementation of Weng-Lin Bayesian ranking, a better, license-free alternative to TrueSkill

PyHarmonize: Adding harmony lines to recorded melodies in Python

Python package for causal inference using Bayesian structural time-series models.

Time-series momentum for momentum investing strategy

icepickle is to allow a safe way to serialize and deserialize linear scikit-learn models

Decision Tree Regression algorithm implemented on Python from scratch.

Machine Learning from Scratch

Bayesian Modeling and Computation in Python

Data Version Control or DVC is an open-source tool for data science and machine learning projects

Apache (Py)Spark type annotations (stub files).

Skforecast is a python library that eases using scikit-learn regressors as multi-step forecasters

This repository demonstrates the usage of hover to understand and supervise a machine learning task.

NumPy-based implementation of a multilayer perceptron (MLP)

Fundamentals of Machine Learning