Hypernets: A General Automated Machine Learning framework to simplify the development of End-to-end AutoML toolkits in specific domains.

Related tags

Machine Learningreinforcement-learningkerasmctshyperparameter-optimizationevolutionary-algorithmsnasmonte-carlo-tree-searchhyperparameter-tuningautomlneural-architecture-searchnasnetenasautodl
Overview

Hypernets

Python Versions Downloads PyPI Version

We Are Hiring！

Dear folks, we are opening several precious positions based in Beijing both for professionals and interns avid in AutoML/NAS, please send your resume/cv to [email protected]. (Application deadline: TBD.)

Hypernets: A General Automated Machine Learning Framework

Hypernets is a general AutoML framework, based on which it can implement automatic optimization tools for various machine learning frameworks and libraries, including deep learning frameworks such as tensorflow, keras, pytorch, and machine learning libraries like sklearn, lightgbm, xgboost, etc. We introduced an abstract search space representation, taking into account the requirements of hyperparameter optimization and neural architecture search(NAS), making Hypernets a general framework that can adapt to various automated machine learning needs.

Overview

Conceptual Model

Illustration of the Search Space

Installation

Install Hypernets with pip command:

pip install hypernets

Optional, to run Hypernets in JupyterLab notebooks, install Hypernets and JupyterLab with command:

pip install hypernets[notebook]

Optional, to run Hypernets in distributed Dask cluster, install Hypernets with command:

pip install hypernets[dask]

Optional, to support dataset with simplified Chinese in feature generation, install jieba package before run Hypernets, or install Hypernets with command:

pip install hypernets[zhcn]

Optional, install all Hypernets components and dependencies with one command:

pip install hypernets[all]

Verify installation:

python -m hypernets.examples.smoke_testing

Related Links

Hypernets related projects

  • HyperGBM: A full pipeline AutoML tool integrated various GBM models.
  • HyperDT/DeepTables: An AutoDL tool for tabular data.
  • HyperKeras: An AutoDL tool for Neural Architecture Search and Hyperparameter Optimization on Tensorflow and Keras.
  • Cooka: Lightweight interactive AutoML system.
  • Hypernets: A general automated machine learning framework.

DataCanvas AutoML Toolkit

Documents

Neural Architecture Search

DataCanvas

Hypernets is an open source project created by DataCanvas.

Comments
  • window pip fail.

    window pip fail.

    ERROR: Complete output from command 'D:\Anaconda\Anaconda3\python.exe' -u -c 'import setuptools, tokenize;file='"'"'C:\Users\John\AppData\Local\Temp\pip-install-4hgmcjg1\python-geohash\setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(file);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' bdist_wheel -d 'C:\Users\John\AppData\Local\Temp\pip-wheel-gijlszf9' --python-tag cp37: ERROR: running bdist_wheel running build running build_py creating build creating build\lib.win-amd64-3.7 copying geohash.py -> build\lib.win-amd64-3.7 copying quadtree.py -> build\lib.win-amd64-3.7 copying jpgrid.py -> build\lib.win-amd64-3.7 copying jpiarea.py -> build\lib.win-amd64-3.7 running build_ext building '_geohash' extension error: Microsoft Visual C++ 14.0 is required. Get it with "Microsoft Visual C++ Build Tools": https://visualstudio.microsoft.com/downloads/

    ERROR: Failed building wheel for python-geohash Running setup.py clean for python-geohash Failed to build python-geohash ERROR: spyder 3.3.6 requires pyqt5<5.13; python_version >= "3", which is not installed. ERROR: spyder 3.3.6 requires pyqtwebengine<5.13; python_version >= "3", which is not installed. ERROR: distributed 2021.11.1 has requirement dask==2021.11.1, but you'll have dask 2.1.0 which is incompatible. ERROR: woodwork 0.9.0 has requirement pandas>=1.3.0, but you'll have pandas 1.2.3 which is incompatible. ERROR: featuretools 1.2.0 has requirement dask[dataframe]>=2021.10.0, but you'll have dask 2.1.0 which is incompatible. ERROR: featuretools 1.2.0 has requirement pandas<2.0.0,>=1.3.0, but you'll have pandas 1.2.3 which is incompatible. Installing collected packages: psutil, pyyaml, tblib, distributed, woodwork, featuretools, python-geohash, hypernets Found existing installation: psutil 5.6.3 Uninstalling psutil-5.6.3: Successfully uninstalled psutil-5.6.3 Found existing installation: PyYAML 5.1.1 ERROR: Cannot uninstall 'PyYAML'. It is a distutils installed project and thus we cannot accurately determine which files belong to it which would lead to only a partial uninstall.

    Installation 
    opened by zhangxjohn 2
  • dataset about dsutils

    dataset about dsutils

    an error about "from hypernets.frameworks.ml.datasets import dsutils", I think it should be "from deeptables.datasets import dsutils".

    bug 
    opened by ninanxiaoguai 2
  • A suggestion for GreedyEnsemble

    A suggestion for GreedyEnsemble

    On line 107 of voting's code, i.e. if self.ensemble_size <= 0: size = predictions.shape[1]

    Is it possible to replace it with self.ensemble_size <= 0 or predictions.shape[1] < self.ensemble_size: size = predictions.shape[1]?

    opened by zhangxjohn 1
  • scorer & calc_score

    scorer & calc_score

    Please make sure that this is a feature request.

    System information

    • Hypernets version (you are using):
    • Are you willing to contribute it (Yes/No):

    Describe the feature and the current behavior/state. I found that the user-defined evaluation metric function could not be set in the hypernets experiment, and calc_score() contained few metrics, such as no mape.

    Will this change the current api? How?

    Any Other info.

    opened by zhangxjohn 1
  • Bump url-parse from 1.5.1 to 1.5.3 in /hypernets/hn_widget/js

    Bump url-parse from 1.5.1 to 1.5.3 in /hypernets/hn_widget/js

    Bumps url-parse from 1.5.1 to 1.5.3.

    Commits
    • ad44493 [dist] 1.5.3
    • c798461 [fix] Fix host parsing for file URLs (#210)
    • 201034b [dist] 1.5.2
    • 2d9ac2c [fix] Sanitize only special URLs (#209)
    • fb128af [fix] Use 'null' as origin for non special URLs
    • fed6d9e [fix] Add a leading slash only if the URL is special
    • 94872e7 [fix] Do not incorrectly set the slashes property to true
    • 81ab967 [fix] Ignore slashes after the protocol for special URLs
    • ee22050 [ci] Use GitHub Actions
    • d2979b5 [fix] Special case the file: protocol (#204)
    • Additional commits viewable in compare view

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
    • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
    • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
    • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

    You can disable automated security fix PRs for this repo from the Security Alerts page.

    dependencies 
    opened by dependabot[bot] 1
  • Bump tar from 6.1.0 to 6.1.10 in /hypernets/hn_widget/js

    Bump tar from 6.1.0 to 6.1.10 in /hypernets/hn_widget/js

    Bumps tar from 6.1.0 to 6.1.10.

    Commits
    • 188badd 6.1.10
    • 23312ce drop dirCache for symlink on all platforms
    • 4f1f4a2 6.1.9
    • 875a37e fix: prevent path escape using drive-relative paths
    • b6162c7 fix: reserve paths properly for unicode, windows
    • 3aaf19b fix: prune dirCache properly for unicode, windows
    • 6a9c51d 6.1.8
    • dfc5923 fix: skip extract if linkpath is stripped entirely
    • 575a511 fix: reserve paths case-insensitively
    • d61628c 6.1.7
    • Additional commits viewable in compare view

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
    • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
    • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
    • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

    You can disable automated security fix PRs for this repo from the Security Alerts page.

    dependencies 
    opened by dependabot[bot] 1
  • Bump tar from 6.1.0 to 6.1.5 in /hypernets/hn_widget/js

    Bump tar from 6.1.0 to 6.1.5 in /hypernets/hn_widget/js

    Bumps tar from 6.1.0 to 6.1.5.

    Commits
    • bd4691c 6.1.5
    • d694c4f ci: test on node 16
    • 84acbd3 fix(unpack): fix hang on large file on open() fail
    • 97c46fc fix(unpack): always resume parsing after an entry error
    • 488ab8c chore: WriteEntry cleaner write() handling
    • be89aaf WriteEntry backpressure
    • ba73f5e chore: track fs state on WriteEntry class, not in arguments
    • bf69383 6.1.4
    • 06cbde5 Avoid an unlikely but theoretically possible redos
    • 0b78386 6.1.3
    • Additional commits viewable in compare view

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
    • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
    • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
    • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

    You can disable automated security fix PRs for this repo from the Security Alerts page.

    dependencies 
    opened by dependabot[bot] 1
  • Bump dns-packet from 1.3.1 to 1.3.4 in /js

    Bump dns-packet from 1.3.1 to 1.3.4 in /js

    Bumps dns-packet from 1.3.1 to 1.3.4.

    Commits

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
    • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
    • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
    • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

    You can disable automated security fix PRs for this repo from the Security Alerts page.

    dependencies 
    opened by dependabot[bot] 0
  • Bump path-parse from 1.0.6 to 1.0.7 in /js

    Bump path-parse from 1.0.6 to 1.0.7 in /js

    Bumps path-parse from 1.0.6 to 1.0.7.

    Commits

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
    • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
    • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
    • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

    You can disable automated security fix PRs for this repo from the Security Alerts page.

    dependencies 
    opened by dependabot[bot] 0
  • Bump url-parse from 1.5.1 to 1.5.3 in /js

    Bump url-parse from 1.5.1 to 1.5.3 in /js

    Bumps url-parse from 1.5.1 to 1.5.3.

    Commits
    • ad44493 [dist] 1.5.3
    • c798461 [fix] Fix host parsing for file URLs (#210)
    • 201034b [dist] 1.5.2
    • 2d9ac2c [fix] Sanitize only special URLs (#209)
    • fb128af [fix] Use 'null' as origin for non special URLs
    • fed6d9e [fix] Add a leading slash only if the URL is special
    • 94872e7 [fix] Do not incorrectly set the slashes property to true
    • 81ab967 [fix] Ignore slashes after the protocol for special URLs
    • ee22050 [ci] Use GitHub Actions
    • d2979b5 [fix] Special case the file: protocol (#204)
    • Additional commits viewable in compare view

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
    • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
    • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
    • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

    You can disable automated security fix PRs for this repo from the Security Alerts page.

    dependencies 
    opened by dependabot[bot] 0
Releases(0.2.5.7)
Owner
DataCanvas
We love data science!
DataCanvas
GitHub Repository https://hypernets.readthedocs.io/
This is the code repository for LRM Stochastic watershed model.

LRM-Squannacook Input data for generating stochastic streamflows are observed and simulated timeseries of streamflow. their format needs to be CSV wit

1 Feb 14, 2022
DaCeML - Machine learning powered by data-centric parallel programming.

Data-centric machine learning powered by DaCe

SPCL 48 Dec 12, 2022
Implementation of K-Nearest Neighbors Algorithm Using PySpark

KNN With Spark Implementation of KNN using PySpark. The KNN was used on two separate datasets (https://archive.ics.uci.edu/ml/datasets/iris and https:

Zachary Petroff 4 Dec 30, 2022
moDel Agnostic Language for Exploration and eXplanation

moDel Agnostic Language for Exploration and eXplanation Overview Unverified black box model is the path to the failure. Opaqueness leads to distrust.

Model Oriented 1.2k Jan 04, 2023
2D fluid simulation implementation of Jos Stam paper on real-time fuild dynamics, including some suggested extensions.

Fluid Simulation Usage Download this repo and store it in your computer. Open a terminal and go to the root directory of this folder. Make sure you ha

Mariana Ávalos Arce 5 Dec 02, 2022
Kaggle Competition using 15 numerical predictors to predict a continuous outcome.

Kaggle-Comp.-Data-Mining Kaggle Competition using 15 numerical predictors to predict a continuous outcome as part of a final project for a stats data

moisey alaev 1 Dec 28, 2021
This is an implementation of the proximal policy optimization algorithm for the C++ API of Pytorch

This is an implementation of the proximal policy optimization algorithm for the C++ API of Pytorch. It uses a simple TestEnvironment to test the algorithm

Martin Huber 59 Dec 09, 2022
The MLOps is the process of continuous integration and continuous delivery of Machine Learning artifacts as a software product, keeping it inside a loop of Design, Model Development and Operations.

MLOps The MLOps is the process of continuous integration and continuous delivery of Machine Learning artifacts as a software product, keeping it insid

Maykon Schots 25 Nov 27, 2022
Can a machine learning project be implemented to estimate the salaries of baseball players whose salary information and career statistics for 1986 are shared?

END TO END MACHINE LEARNING PROJECT ON HITTERS DATASET Can a machine learning project be implemented to estimate the salaries of baseball players whos

Pinar Oner 7 Dec 18, 2021
scikit-learn is a python module for machine learning built on top of numpy / scipy

About scikit-learn is a python module for machine learning built on top of numpy / scipy. The purpose of the scikit-learn-tutorial subproject is to le

Gael Varoquaux 122 Dec 12, 2022
Simple Machine Learning Tool Kit

Getting started smltk (Simple Machine Learning Tool Kit) package is implemented for helping your work during data preparation testing your model The g

Alessandra Bilardi 1 Dec 30, 2021
Lseng-iseng eksplor Machine Learning dengan menggunakan library Scikit-Learn

Kalo dengar istilah ML, biasanya rada ambigu. Soalnya punya beberapa kepanjangan, seperti Mobile Legend, Makan Lontong, Ma**ng L*v* dan lain-lain. Tapi pada repo ini membahas Machine Learning :)

Alfiyanto Kondolele 1 Apr 06, 2022
CVXPY is a Python-embedded modeling language for convex optimization problems.

CVXPY The CVXPY documentation is at cvxpy.org. We are building a CVXPY community on Discord. Join the conversation! For issues and long-form discussio

4.3k Jan 08, 2023
使用数学和计算机知识投机倒把

偷鸡不成项目集锦 坦率地讲，涉及金融市场的好策略如果公开，必然导致使用的人多，最后策略变差。所以这个仓库只收集我目前失败了的案例。 加密货币组合套利 中国体育彩票预测 我赚不上钱的项目，也许可以帮助更有能力的人去赚钱。

Roy 28 Dec 29, 2022
BentoML is a flexible, high-performance framework for serving, managing, and deploying machine learning models.

Model Serving Made Easy BentoML is a flexible, high-performance framework for serving, managing, and deploying machine learning models. Supports multi

BentoML 4.4k Jan 04, 2023
Lightweight Machine Learning Experiment Logging 📖

Simple logging of statistics, model checkpoints, plots and other objects for your Machine Learning Experiments (MLE). Furthermore, the MLELogger comes with smooth multi-seed result aggregation and co

Robert Lange 65 Dec 08, 2022
This machine-learning algorithm takes in data from the last 60 days and tries to predict tomorrow's price of any crypto you ask it.

Crypto-Currency-Predictor This machine-learning algorithm takes in data from the last 60 days and tries to predict tomorrow's price of any crypto you

Hazim Arafa 6 Dec 04, 2022
Free MLOps course from DataTalks.Club

MLOps Zoomcamp Our MLOps Zoomcamp course Sign up here: https://airtable.com/shrCb8y6eTbPKwSTL (it's not automated, you will not receive an email immed

DataTalksClub 4.6k Dec 31, 2022
Machine Learning for Time-Series with Python.Published by Packt

Machine-Learning-for-Time-Series-with-Python Become proficient in deriving insights from time-series data and analyzing a model’s performance Links Am

Packt 124 Dec 28, 2022
MCML is a toolkit for semi-supervised dimensionality reduction and quantitative analysis of Multi-Class, Multi-Label data

MCML is a toolkit for semi-supervised dimensionality reduction and quantitative analysis of Multi-Class, Multi-Label data. We demonstrate its use

Pachter Lab 26 Nov 29, 2022