bayesianInference

3.4. Bayesian Inference

StatisticalRethinkingJulia

StanJulia

Julia:

The Turing Language

cscherrer/Soss.jl: Probabilistic programming via source rewriting

probcomp/Gen.jl: A general-purpose probabilistic programming system with programmable inference

Laboratory of Applied Mathematical Programming and Statistics

BIASlab

FRBNY-DSGE/DSGE.jl: Solve and estimate Dynamic Stochastic General Equilibrium models (including the New York Fed DSGE)

StatisticalRethinkingJulia/StatisticalRethinking.jl: Julia package with selected functions in the R package rethinking. Used in the SR2... projects.

Python:

pymc-devs/pymc: Probabilistic Programming in Python: Bayesian Modeling and Probabilistic Machine Learning with Aesara

pints-team/pints: Probabilistic Inference on Noisy Time Series

pyro-ppl/pyro: Deep universal probabilistic programming with Python and PyTorch

pyro-ppl/numpyro: Probabilistic programming with NumPy powered by JAX for autograd and JIT compilation to GPU/TPU/CPU.

blackjax-devs/blackjax: BlackJAX is a sampling library designed for ease of use, speed and modularity.

tensorflow/probability: Probabilistic reasoning and statistical analysis in TensorFlow

google/edward2: A simple probabilistic programming language.

thu-ml/zhusuan: A probabilistic programming library for Bayesian deep learning, generative models, based on Tensorflow

jmschrei/pomegranate: Fast, flexible and easy to use probabilistic modelling in Python.

csynbiosysIBioEUoE/BOMBs.jl: Repository for the Julia BOMBS package

3.4.1. MCMC

Methods like HMC, SGLD are Covered by above-mentioned packages.

Julia:

mauro3/KissMCMC.jl: Keep it simple, stupid, MCMC

Nicescheidan/BarkerMCMC.jl: gradient based MCMC sampler

BigBayes/SGMCMC.jl: Stochastic Gradient Markov Chain Monte Carlo and Optimisation

tpapp/DynamicHMC.jl: Implementation of robust dynamic Hamiltonian Monte Carlo methods (NUTS) in Julia.

madsjulia/AffineInvariantMCMC.jl: Affine Invariant Markov Chain Monte Carlo (MCMC) Ensemble sampler

TuringLang/EllipticalSliceSampling.jl: Julia implementation of elliptical slice sampling.

Nested SamplingTuringLang/NestedSamplers.jl: Implementations of single and multi-ellipsoid nested sampling

bat/UltraNest.jl: Julia wrapper for UltraNest: advanced nested sampling for model comparison and parameter estimation

itsdfish/DifferentialEvolutionMCMC.jl: A Julia package for Differential Evolution MCMC

Python:

AdamCobb/hamiltorch: PyTorch-based library for Riemannian Manifold Hamiltonian Monte Carlo (RMHMC) and inference in Bayesian neural networks

Reviewjeremiecoullon/SGMCMCJax: Lightweight library of stochastic gradient MCMC algorithms written in JAX.

Nested Samplingjoshspeagle/dynesty: Dynamic Nested Sampling package for computing Bayesian posteriors and evidences

JohannesBuchner/UltraNest: Fit and compare complex models reliably and rapidly. Advanced nested sampling.

ruqizhang/csgmcmc: Cyclical Stochastic Gradient MCMC for Bayesian Deep Learning

Custom Tensorflow optimizer cSGLD (Stochastic Langevin Dynamics) in TF2: correct update ops? · Issue #2469 · tensorflow/addons

3.4.2. Approximate Bayesian Computation (ABC)

Also called likelihood free or simulation based methods

Reviewsbi-benchmark/sbibm: Simulation-based inference benchmark

Julia: (few)

JuliaApproxInference

tanhevg/GpABC.jl

marcjwilliams1/ApproxBayes.jl: Approximate Bayesian Computation (ABC) algorithms for likelihood free inference in julia

francescoalemanno/KissABC.jl: Pure julia implementation of Multiple Affine Invariant Sampling for efficient Approximate Bayesian Computation

Python:

elfi-dev/elfi: ELFI - Engine for Likelihood-Free Inference

eth-cscs/abcpy: ABCpy package

pints-team/pints: Probabilistic Inference on Noisy Time Series

mackelab/sbi: Simulation-based inference in PyTorch

ICB-DCM/pyABC: distributed, likelihood-free inference

diyabc/abcranger: ABC random forests for model choice and parameter estimation, pure C++ implementation

3.4.3. Data Assimilation (SMC, particles filter)

Julia:

Alexander-Barth/DataAssim.jl: Implementation of various ensemble Kalman Filter data assimilation methods in Julia

baggepinnen/LowLevelParticleFilters.jl: Simple particle/kalman filtering, smoothing and parameter estimation

JuliaGNSS/KalmanFilters.jl: Various Kalman Filters: KF, UKF, AUKF and their Square root variant

CliMA/EnsembleKalmanProcesses.jl: Implements Optimization and approximate uncertainty quantification algorithms, Ensemble Kalman Inversion, and Ensemble Kalman Processes.

FRBNY-DSGE/StateSpaceRoutines.jl: Package implementing common state-space routines.

simsurace/FeedbackParticleFilters.jl: A Julia package that provides (feedback) particle filters for nonlinear stochastic filtering and data assimilation problems

mjb3/DiscretePOMP.jl: Bayesian inference for Discrete state-space Partially Observed Markov Processes in Julia. See the docs:

Python:

nchopin/particles: Sequential Monte Carlo in python

rlabbe/filterpy: Python Kalman filtering and optimal estimation library. Implements Kalman filter, particle filter, Extended Kalman filter, Unscented Kalman filter, g-h (alpha-beta), least squares, H Infinity, smoothers, and more. Has companion book 'Kalman and Bayesian Filters in Python'.

tingiskhan/pyfilter: Particle filtering and sequential parameter inference in Python

3.4.4. Variational Inference

SVGDSearch · Stein Variational Gradient DescentAlso see pyro, Stein method part

Red-Portal/KLpqVI.jl

Julia:

bat/MGVI.jl: Metric Gaussian Variational Inference

TuringLang/AdvancedVI.jl: A library for variational Bayesian methods in Julia

ngiann/ApproximateVI.jl: Approximate variational inference in Julia

Python:

3.4.5. Gaussion, non-Gaussion and Kernel

Julia:

Gaussian Processes for Machine Learning in Julia

Laboratory of Applied Mathematical Programming and Statistics

JuliaRobotics

JuliaStats/KernelDensity.jl: Kernel density estimators for Julia

JuliaRobotics/KernelDensityEstimate.jl: Kernel Density Estimate with product approximation using multiscale Gibbs sampling

theogf/AugmentedGaussianProcesses.jl: Gaussian Process package based on data augmentation, sparsity and natural gradients

JuliaGaussianProcesses/TemporalGPs.jl: Fast inference for Gaussian processes in problems involving time

aterenin/SparseGaussianProcesses.jl: A Julia implementation of sparse Gaussian processes via path-wise doubly stochastic variational inference.

PieterjanRobbe/GaussianRandomFields.jl: A package for Gaussian random field generation in Julia

JuliaGaussianProcesses/Stheno.jl: Probabilistic Programming with Gaussian processes in Julia

STOR-i/GaussianProcesses.jl: A Julia package for Gaussian Processes

Python:

cornellius-gp/gpytorch: A highly efficient and modular implementation of Gaussian Processes in PyTorch

GPflow/GPflow: Gaussian processes in TensorFlow

SheffieldML/GPy: Gaussian processes framework in python

3.4.6. Bayesian Optimization

Julia:

SciML/Surrogates.jl: Surrogate modeling and optimization for scientific machine learning (SciML)

jbrea/BayesianOptimization.jl: Bayesian optimization for Julia

baggepinnen/Hyperopt.jl: Hyperparameter optimization in Julia.

Python:

fmfn/BayesianOptimization: A Python implementation of global optimization with gaussian processes.

pytorch/botorch: Bayesian optimization in PyTorch

optuna/optuna: A hyperparameter optimization framework

huawei-noah/HEBO: Bayesian optimisation library developped by Huawei Noah's Ark Library

3.4.7. Information theory

Julia: entropy and kldivengence for distributions or vectors can be seen in Distributions.jl

KL divergence for functionsRafaelArutjunjan/InformationGeometry.jl: Methods for computational information geometry

not maintainedkzahedi/Shannon.jl: Entropy, Mutual Information, KL-Divergence related to Shannon's information theory and functions to binarize data

gragusa/Divergences.jl: A Julia package for evaluation of divergences between distributions

Tchanders/InformationMeasures.jl: Entropy, mutual information and higher order measures from information theory, with various estimators and discretisation methods.

JuliaDynamics/TransferEntropy.jl: Transfer entropy (conditional mutual information) estimators for the Julia language

cynddl/Discreet.jl: A Julia package to estimate discrete entropy and mutual information

3.4.8. Uncertainty

Uncertainty propogation

Julia:

JuliaPhysics/Measurements.jl: Error propagation calculator and library for physical measurements. It supports real and complex numbers with uncertainty, arbitrary precision calculations, operations with arrays, and numerical integration.

baggepinnen/MonteCarloMeasurements.jl: Propagation of distributions by Monte-Carlo sampling: Real number types with uncertainty represented by samples.

ReviewUncertainty Programming, Generalized Uncertainty Quantification

AnderGray/MomentArithmetic.jl: Rigorous moment propagation with partial information about moments and dependencies in Julia

mschauer/Mitosis.jl: Automatic probabilistic programming for scientific machine learning and dynamical models

GoodJuliaReach/RangeEnclosures.jl: A Julia package to compute range enclosures of real-valued functions.

Python

uncertainty-toolbox/uncertainty-toolbox: A python toolbox for predictive uncertainty quantification, calibration, metrics, and visualization

pyro-ppl/funsor: Functional tensors for probabilistic programming

lebigot/uncertainties: Transparent calculations with uncertainties on the quantities involved (aka "error propagation"); calculation of derivatives.

3.4.9. Casual

zenna/Omega.jl: Causal, Higher-Order, Probabilistic Programming

mschauer/CausalInference.jl: Causal inference, graphical models and structure learning with the PC algorithm.

JuliaDynamics/CausalityTools.jl: Algorithms for causal inference and the detection of dynamical coupling from time series, and for approximation of the transfer operator and invariant measures.

python

Review: rguo12/awesome-causality-algorithms: An index of algorithms for learning causality with data

3.4.10. Sampling

MrUrq/LatinHypercubeSampling.jl: Julia package for the creation of optimised Latin Hypercube Sampling Plans

SciML/QuasiMonteCarlo.jl: Lightweight and easy generation of quasi-Monte Carlo sequences with a ton of different methods on one API for easy parameter exploration in scientific machine learning (SciML)

3.4.11 Message Passing

Julia:

biaslab/ReactiveMP.jl: Julia package for automatic Bayesian inference on a factor graph with reactive message passing

biaslab/ForneyLab.jl: Julia package for automatically generating Bayesian inference algorithms through message passing on Forney-style factor graphs.