The Multi-Mission Maximum Likelihood framework (3ML)

Overview

CI Conda Build and Publish Test Against XSPEC codecov Documentation Status License

GitHub pull requests GitHub issues

PyPi

PyPI version fury.io PyPI - Downloads PyPI - Python Version

Conda

Conda Conda

drawing

The Multi-Mission Maximum Likelihood framework (3ML)

A framework for multi-wavelength/multi-messenger analysis for astronomy/astrophysics.


Astrophysical sources are observed by different instruments at different wavelengths with an unprecedented quality. Putting all these data together to form a coherent view, however, is a very difficult task. Indeed, each instrument and data type has its own ad-hoc software and handling procedure, which present steep learning curves and do not talk to each other.

The Multi-Mission Maximum Likelihood framework (3ML) provides a common high-level interface and model definition, which allows for an easy, coherent and intuitive modeling of sources using all the available data, no matter their origin. At the same time, thanks to its architecture based on plug-ins, 3ML uses under the hood the official software of each instrument, the only one certified and maintained by the collaboration which built the instrument itself. This guarantees that 3ML is always using the best possible methodology to deal with the data of each instrument.

drawing

Though Maximum Likelihood is in the name for historical reasons, 3ML is an interface to several Bayesian inference algorithms such as MCMC and nested sampling as well as likelihood optimization algorithms. Each approach to analysis can be seamlessly switched between allowing users to try different approaches quickly and without having to rewrite their model or data interfaces.

Like your XPSEC models? You can use them in 3ML as well as our growing selection of 1-,2- and 3-D models from our fast and customizable modeling language astromodels.

Installation

Installing with pip or conda is easy

pip install astromodels threeml
conda  install astromodels threeml -c threeml conda-forge 

Please refer to the Installation instructions for more details and trouble-shooting.

Press

Who is using 3ML?

Here is a highlight list of teams and their publications using 3ML.

A full list of publications using 3ML is here.

Citing

If you find this package useful in you analysis, or the code in your own custom data tools, please cite:

Vianello et al. (2015)

Acknowledgements

3ML makes use of the Spanish Virtual Observatory's Filter Profile service (http://svo2.cab.inta-csic.es/svo/theory/fps3/index.php?mode=browse&gname=NIRT).

If you use these profiles in your research, please consider citing them by using the following suggested sentence in your paper:

"This research has made use of the SVO Filter Profile Service (http://svo2.cab.inta-csic.es/theory/fps/) supported from the Spanish MINECO through grant AyA2014-55216"

and citing the following publications:

The SVO Filter Profile Service. Rodrigo, C., Solano, E., Bayo, A. http://ivoa.net/documents/Notes/SVOFPS/index.html The Filter Profile Service Access Protocol. Rodrigo, C., Solano, E. http://ivoa.net/documents/Notes/SVOFPSDAL/index.html

ThreeML is supported by National Science Foundation (NSF)

Comments
  • Use from astromodels Xspec import * error

    Use from astromodels Xspec import * error

    Hello! When I want to use xspec library to combine GBM and XRT to analyze energy spectrum, I try to use from astromodels Xspec import *has the following error messages. How did this happen?


    ImportError Traceback (most recent call last) /tmp/ipykernel_12482/3995023305.py in 5 6 # we will need XPSEC models for extinction ----> 7 from astromodels.xspec import * 8 from astromodels.xspec.xspec_settings import *

    ~/miniconda3/envs/threeML/lib/python3.7/site-packages/astromodels/xspec/init.py in ----> 1 from .factory import * 2 #from .xspec_settings import x

    ~/miniconda3/envs/threeML/lib/python3.7/site-packages/astromodels/xspec/factory.py in 16 pass 17 ---> 18 from astromodels.xspec import _xspec 19 20 # When running in a Anaconda environment, the package xspec-modelsonly

    ImportError: cannot import name '_xspec' from 'astromodels.xspec' (/home/hank/miniconda3/envs/threeML/lib/python3.7/site-packages/astromodels/xspec/init.py)

    opened by ghost 64
  • How do I install a stable threeml module?

    How do I install a stable threeml module?

    Hello,

    The threeml module is currently latest, causing many modules to fail to install, including the Xspexcmodule. So I want to install a more stable threeml module, so how do I do that?

    opened by ghost 18
  • Background Modeling

    Background Modeling

    I've been looking at the background modeling and it may be better to approach it by adding a node onto PointSource:

    ps = PointSource(...,background_shape=Powerlaw() )
    

    Otherwise, we are just adding a function inside of the plugin and have to dig into private members to access the free parameters of that function.

    Is this too involved of an approach @giacomov ?

    opened by grburgess 18
  • properties of the spectra and choosing sources for SpectrumLike

    properties of the spectra and choosing sources for SpectrumLike

    would it make sense to assign some properties in the SpectrumLike plugin. it is quite useful when one wants to associate only some sources to a given spectrum, instead of summing all, as it is done now.

    opened by volodymyrss 17
  • XS_gabs model is not correctly imported

    XS_gabs model is not correctly imported

    I am not sure yet what is the cause, but the model takes only 0 or 1 values, no intermediate, independently of the strength. Position and width still work, but fit breaks, naturally, due to lack of sensitivity to strength. cyclabs e.g. works well (but I don't need it now).

    opened by volodymyrss 17
  • Installation with Fermitools

    Installation with Fermitools

    Discussed in https://github.com/threeML/threeML/discussions/576

    Originally posted by PepeJoseHU September 11, 2022 Hello, I plan to install 3ML for Fermi data with CONDA, but looks 3ML have dependence with fermitools. But when I install 'fermitools', it also make an environment with CONDA, so how to fix this problem or if there have way to make connection between different CONDA environments?

    Thanks!

    opened by grburgess 16
  • Response and GBM files

    Response and GBM files

    There seems to be an issue with the reading of GBM DRMs (similar to what I have seen with gammapy)

    
    /usr/local/Cellar/python/HEAD/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/threeML-0.3.2-py2.7.egg/threeML/plugins/ogip.pyc in _read_matrix(self, data, n_channels)
        750             for j in range(n_grp[i]):
        751 
    --> 752                 rsp[i, f_chan[i][j]: f_chan[i][j] + n_chan[i][j]] = matrix[i][m_start:m_start + n_chan[i][j]]
        753 
        754                 m_start += n_chan[i][j]
    
    ValueError: could not broadcast input array from shape (128) into shape (127)
    
    
    
    bug 
    opened by grburgess 16
  • Plotting

    Plotting

    I propose some updates to plotting routines...

    1. Added a function to joint_likelihood.py that will return 1D/2D likelihood profiles/contours for all parameters/combinations of parameters for convenience.

    2. Added an option to plot_point_source_spectra() to also plot extended source spectra (this will just plot whatever was set as the spectral shape, i.e. the 'spatially integrated' spectrum.

    3a. Moved convergence_plots() and plot_chains() from bayesian_analysis.py to analysis_results.py, so that the plots can be produced from analysis results loaded from disk. 3b. Changed convergence_plots() to use the same number of samples for the bootstrapping average as for the sliding window, I hope that was the original intent.

    opened by henrikef 15
  • added upper/lower limits

    added upper/lower limits

    Added three functions based on the equal_tail_intervals to get the X percent upper/lower limits.

    This makes things rather convenient for finding lower limits on cutoffs (or flux normalizations) after doing Bayesian analysis, although I suppose it can also be used in a frequentist analysis once the variates are computed.

    opened by cbrisboi 13
  • "Current minimum stored after fit and current do not correspond!" messages with all PyGMO algorithms

    I am getting "Current minimum stored after fit and current do not correspond!" messages with all PyGMO algorithms for a very simple model and data, which works with MINUIT.

    opened by volodymyrss 12
  • Alternative to root_numpy

    Alternative to root_numpy

    I was looking at updating hawc_hal to use uproot rather than the old ROOT interface for its .root file io and saw that a big part of the root_numpy dependency is carried over from 3ML via the dependency on threeML/io/cern_root_utils. I would generally prefer to stay consistent with file handling between 3ML and hawc_hal. Is there a preference for uproot vs the new ROOT interface? This is already an issue in hawc_hal, see https://github.com/threeML/hawc_hal/issues/42

    The code looks straightforward to update, but before I run off and implement something that is not preferred, I thought asking would be wise first.

    no-issue-activity 
    opened by cbrisboi 11
  • "FitFailed: MIGRAD call failed. This is usually due to unconstrained parameters."

    Hi,

    I'm running into the following fitting error while trying to fit just Fermi-LAT data: "FitFailed: MIGRAD call failed. This is usually due to unconstrained parameters."

    I'm primarily interested in using 3ML to create joint fits of Fermi-LAT data with X-ray (NuSTAR/NICER/XMM/etc) data. I have already downloaded and created intermediate (ltcube, expo maps, etc). To recreate my error: I create a data from the DataList like so:

    eventFile = os.path.join( 'photon_file.fits' ) ft2File = os.path.join('spacecraft_file.fits' ) expomap = os.path.join( 'bexpmap_00.fits' ) ltcube = os.path.join( 'ltcube_00.fits' )

    if is_plugin_available("FermiLATLike"):

    LAT = FermiLATLike( "LAT", eventFile, ft2File, ltcube, 'binned', binned_expo_map=expomap, source_maps=srcmap )
    #I'm performing binned analysis on my source and have created the binned exposure and src maps.
    

    data = DataList(LAT)

    #%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%

    Now I define my spectral model:

    spectrum=Log_parabola() source1=PointSource(test_src,ra=ra,dec=dec,spectral_shape=spectrum)

    model=Model(source1)

    fluxUnit=1/(u.MeVu.cm**2u.s)

    model.test_src.spectrum.main.Log_parabola.alpha.value=-2.7 model.test_src.spectrum.main.Log_parabola.alpha.bounds=(-5.0,-0.5) model.test_src.spectrum.main.Log_parabola.alpha.fix= False

    model.test_src.spectrum.main.Log_parabola.piv.value= 2134*u.MeV model.test_src.spectrum.main.Log_parabola.piv.fix= True

    model.test_src.spectrum.main.Log_parabola.K.value=2.2e-6fluxUnit model.test_src.spectrum.main.Log_parabola.K.bounds=(1e-12fluxUnit,1e1*fluxUnit) model.test_src.spectrum.main.Log_parabola.K.fix= False

    model.test_src.spectrum.main.Log_parabola.beta.value=0.3 model.test_src.spectrum.main.Log_parabola.beta.bounds=(0.0,1.0) model.test_src.spectrum.main.Log_parabola.beta.fix= False

    #%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%

    And start the likelihood analysis: jl = JointLikelihood(model, data) res = jl.fit()

    #%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%%#%

    At this point, I run into the aforementioned error: "FitFailed: MIGRAD call failed. This is usually due to unconstrained parameters."

    When investigating the model, there don't seem to be any constraints on the normalization (but this may be a product of assigning it a unit?):

                                           value min_value max_value  \
    

    test_src.spectrum.main.Log_parabola.K 0.0 0.0 0.0

    Any help would be appreciated.

    opened by aplangrangian 0
  • Graphics bug - Kills kernel

    Graphics bug - Kills kernel

    Describe the bug Importing TriggerSelector from GtBurst AFTER importing threeML causes crashes when viewing lightcurves or other plots. Specifically, after importing threeML then TriggerSelector, if one downloads GBM data, creates a TimeSeriesBuilder, and then goes to view the lightcurve, a Python3.9 window pops up but is unresponsive, and after force quitting it, the kernel also dies.

    NOTE: This is for BUG reports and not questions about generic data analysis.

    To Reproduce

    Run these commands in jupyter notebook in sequential order:

    from threeML import *

    from GtBurst.TriggerSelector import TriggerSelector

    gbm_data = download_GBM_trigger_data("bn090926181",detectors=['n3'])

    ts1 = TimeSeriesBuilder.from_gbm_cspec_or_ctime('n3', cspec_or_ctime_file=gbm_data['n3']["cspec"], rsp_file=gbm_data['n3']["rsp"] )

    fig = ts1.view_lightcurve(-50,100);

    Expected behavior I expect to view a light curve in the notebook, but a python3.9 unresponsive window pops up, and the kernel dies. This is fixed when importing TriggerSelector first, and then importing threeML.

    Screenshots Screen Shot 2022-12-16 at 11 11 28 AM

    Desktop (please complete the following information):

    • OS: iOS
    • Montery version 12.6
    • Apple M1 Pro chip
    opened by nkirschner 1
  • Finishing CastorLike and Ability to Build flux points.

    Finishing CastorLike and Ability to Build flux points.

    Feature Request

    I have been working on cleaning up and speeding up the castorLike plugin which is designed to fit flux points that have been generated by likelihood profiles (ew). However, I never use this feature, so it would be good to have some help.

    Describe the solution you'd like

    1. Clean up plotting
    2. Documentation and examples of usage
    3. We should build a tool that can actually generate these profiles from any spectral fit. Is it possible from just the FitResults alone?

    @henrikef @giacomov @omodei

    You peeps are probably the experts here. @giacomov started it off, so he might have some directions. Perhaps Eric Charles could be helpful as well?

    help wanted plugin issue 
    opened by grburgess 0
  • reuse extended event class files too

    reuse extended event class files too

    I added a line to include the extended LAT files with a different name also to the list of files to be checked if they were previously downloaded already.

    opened by maklinger 1
  • asymmetric error in XY like

    asymmetric error in XY like

    I have spectral points which have asymmetric error and I am hoping to use XY Like to fit those spectral points. However, it seems like XY Like only supports symmetric errors. Is it possible to make XY Like support asymmetric error?

    opened by jasonfan1997 15
  • Value of BIC

    Value of BIC

    Discussed in https://github.com/threeML/threeML/discussions/585

    Originally posted by changxuezhao September 28, 2022 Dear Dr. Burgess, I have two questions for you about 3ML. First of all, the statistical (whether chi2 or pgstat or BIC) obtained using XSPEC will be smaller than 3ML, I think this is the reason for the likelihood, my question is why the likelihood of 3ML is larger than other tools(xspec,rmfit). Second, why can only the line graph of vFv be drawn without data points, and how to add data points to the vFv graph. Thanks and looking forward to your reply。

    no-issue-activity statistics 
    opened by grburgess 1
Releases(v0.1.0)
  • v0.1.0(Apr 19, 2016)

  • v0.0.5(Feb 6, 2015)

    All dependencies are now explicitly stated in the setup.py file, with the exception of ROOT (we might get rid of this dependency and get MINUIT from somewhere else)

    Source code(tar.gz)
    Source code(zip)
  • v0.0.4(Feb 6, 2015)

    The user can install this version by using pip ("pip install threeML").

    The dependencies (numpy, scipy, numexpr, numdifftools...) must be installed separately.

    Source code(tar.gz)
    Source code(zip)
Owner
The Multi-Mission Maximum Likelihood (3ML)
The Multi-Mission Maximum Likelihood (3ML)
Official code of Team Yao at Multi-Modal-Fact-Verification-2022

Official code of Team Yao at Multi-Modal-Fact-Verification-2022 A Multi-Modal Fact Verification dataset released as part of the De-Factify workshop in

Wei-Yao Wang 11 Nov 15, 2022
Programming with Neural Surrogates of Programs

Programming with Neural Surrogates of Programs

0 Dec 12, 2021
Redash reset for python

redash-reset This will use a default REDASH_SECRET_KEY key of c292a0a3aa32397cdb050e233733900f this allows you to reset the password of the user ID bu

Robert Wiggins 5 Nov 14, 2022
Offical implementation of Shunted Self-Attention via Multi-Scale Token Aggregation

Shunted Transformer This is the offical implementation of Shunted Self-Attention via Multi-Scale Token Aggregation by Sucheng Ren, Daquan Zhou, Shengf

156 Dec 27, 2022
Auto HMM: Automatic Discrete and Continous HMM including Model selection

Auto HMM: Automatic Discrete and Continous HMM including Model selection

Chess_champion 29 Dec 07, 2022
An Approach to Explore Logistic Regression Models

User-centered Regression An Approach to Explore Logistic Regression Models This tool applies the potential of Attribute-RadViz in identifying correlat

0 Nov 12, 2021
Python code to generate art with Generative Adversarial Network

GAN_Canvas_Maker Generating Art using Generative Adversarial Network (GAN) Python code to generate art with Generative Adversarial Network: https://to

Jonny Banana 10 Aug 22, 2022
FOSS Digital Asset Distribution Platform built on Frappe.

Digistore FOSS Digital Assets Marketplace. Distribute digital assets, like a pro. Video Demo Here Features Create, attach and list digital assets (PDF

Mohammad Hussain Nagaria 30 Dec 08, 2022
An official implementation of the Anchor DETR.

Anchor DETR: Query Design for Transformer-Based Detector Introduction This repository is an official implementation of the Anchor DETR. We encode the

MEGVII Research 276 Dec 28, 2022
Code for "Causal autoregressive flows" - AISTATS, 2021

Code for "Causal Autoregressive Flow" This repository contains code to run and reproduce experiments presented in Causal Autoregressive Flows, present

Ricardo Pio Monti 35 Dec 16, 2022
[CVPR 2022] Official PyTorch Implementation for "Reference-based Video Super-Resolution Using Multi-Camera Video Triplets"

Reference-based Video Super-Resolution (RefVSR) Official PyTorch Implementation of the CVPR 2022 Paper Project | arXiv | RealMCVSR Dataset This repo c

Junyong Lee 151 Dec 30, 2022
Learning from Synthetic Data with Fine-grained Attributes for Person Re-Identification

Less is More: Learning from Synthetic Data with Fine-grained Attributes for Person Re-Identification Suncheng Xiang Shanghai Jiao Tong University Over

SunchengXiang 68 Dec 13, 2022
Skyformer: Remodel Self-Attention with Gaussian Kernel and Nystr\"om Method (NeurIPS 2021)

Skyformer This repository is the official implementation of Skyformer: Remodel Self-Attention with Gaussian Kernel and Nystr"om Method (NeurIPS 2021).

Qi Zeng 46 Sep 20, 2022
[CVPR 2016] Unsupervised Feature Learning by Image Inpainting using GANs

Context Encoders: Feature Learning by Inpainting CVPR 2016 [Project Website] [Imagenet Results] Sample results on held-out images: This is the trainin

Deepak Pathak 829 Dec 31, 2022
CV backbones including GhostNet, TinyNet and TNT, developed by Huawei Noah's Ark Lab.

CV Backbones including GhostNet, TinyNet, TNT (Transformer in Transformer) developed by Huawei Noah's Ark Lab. GhostNet Code TinyNet Code TNT Code Pyr

HUAWEI Noah's Ark Lab 3k Jan 08, 2023
Photo2cartoon - 人像卡通化探索项目 (photo-to-cartoon translation project)

人像卡通化 (Photo to Cartoon) 中文版 | English Version 该项目为小视科技卡通肖像探索项目。您可使用微信扫描下方二维码或搜索“AI卡通秀”小程序体验卡通化效果。

Minivision_AI 3.5k Dec 30, 2022
All supplementary material used by me while TA-ing CS3244: Machine Learning

CS3244-Tutorial-Material All supplementary material used by me while TA-ing CS3244: Machine Learning at NUS School of Computing. What is this? I teach

Rishabh Anand 18 Sep 23, 2022
Code for the paper "Balancing Training for Multilingual Neural Machine Translation, ACL 2020"

Balancing Training for Multilingual Neural Machine Translation Implementation of the paper Balancing Training for Multilingual Neural Machine Translat

Xinyi Wang 21 May 18, 2022
A flexible and extensible framework for gait recognition.

A flexible and extensible framework for gait recognition. You can focus on designing your own models and comparing with state-of-the-arts easily with the help of OpenGait.

Shiqi Yu 335 Dec 22, 2022
Code for SIMMC 2.0: A Task-oriented Dialog Dataset for Immersive Multimodal Conversations

The Second Situated Interactive MultiModal Conversations (SIMMC 2.0) Challenge 2021 Welcome to the Second Situated Interactive Multimodal Conversation

Facebook Research 81 Nov 22, 2022