Gammatone-based spectrograms, using gammatone filterbanks or Fourier transform weightings.

Related tags

Audiogammatone
Overview

Gammatone Filterbank Toolkit

Utilities for analysing sound using perceptual models of human hearing.

Jason Heeris, 2013

Summary

This is a port of Malcolm Slaney's and Dan Ellis' gammatone filterbank MATLAB code, detailed below, to Python 2 and 3 using Numpy and Scipy. It analyses signals by running them through banks of gammatone filters, similar to Fourier-based spectrogram analysis.

Gammatone-based spectrogram of Für Elise

Installation

You can install directly from this git repository using:

pip install git+https://github.com/detly/gammatone.git

...or you can clone the git repository however you prefer, and do:

pip install .

...or:

python setup.py install

...from the cloned tree.

Dependencies

  • numpy
  • scipy
  • nose
  • mock
  • matplotlib

Using the Code

See the API documentation. For a demonstration, find a .wav file (for example, Für Elise) and run:

python -m gammatone FurElise.wav -d 10

...to see a gammatone-gram of the first ten seconds of the track. If you've installed via pip or setup.py install, you should also be able to just run:

gammatone FurElise.wav -d 10

Basis

This project is based on research into how humans perceive audio, originally published by Malcolm Slaney:

Malcolm Slaney (1998) "Auditory Toolbox Version 2", Technical Report #1998-010, Interval Research Corporation, 1998.

Slaney's report describes a way of modelling how the human ear perceives, emphasises and separates different frequencies of sound. A series of gammatone filters are constructed whose width increases with increasing centre frequency, and this bank of filters is applied to a time-domain signal. The result of this is a spectrum that should represent the human experience of sound better than, say, a Fourier-domain spectrum would.

A gammatone filter has an impulse response that is a sine wave multiplied by a gamma distribution function. It is a common approach to modelling the auditory system.

The gammatone filterbank approach can be considered analogous (but not equivalent) to a discrete Fourier transform where the frequency axis is logarithmic. For example, a series of notes spaced an octave apart would appear to be roughly linearly spaced; or a sound that was distributed across the same linear frequency range would appear to have more spread at lower frequencies.

The real goal of this toolkit is to allow easy computation of the gammatone equivalent of a spectrogram — a time-varying spectrum of energy over audible frequencies based on a gammatone filterbank.

Slaney demonstrated his research with an initial implementation in MATLAB. This implementation was later extended by Dan Ellis, who found a way to approximate a "gammatone-gram" by using the fast Fourier transform. Ellis' code calculates a matrix of weights that can be applied to the output of a FFT so that a Fourier-based spectrogram can easily be transformed into such an approximation.

Ellis' code and documentation is here: Gammatone-like spectrograms

Interest

I became interested in this because of my background in science communication and my general interest in the teaching of signal processing. I find that the spectrogram approach to visualising signals is adequate for illustrating abstract systems or the mathematical properties of transforms, but bears little correspondence to a person's own experience of sound. If someone wants to see what their favourite piece of music "looks like," a normal Fourier transform based spectrogram is actually quite a poor way to visualise it. Features of the audio seem to be oddly spaced or unnaturally emphasised or de-emphasised depending on where they are in the frequency domain.

The gammatone filterbank approach seems to be closer to what someone might intuitively expect a visualisation of sound to look like, and can help develop an intuition about alternative representations of signals.

Verifying the port

Since this is a port of existing MATLAB code, I've written tests to verify the Python implementation against the original code. These tests aren't unit tests, but they do generally test single functions. Running the tests has the same workflow:

  1. Run the scripts in the test_generation directory. This will create a .mat file containing test data in tests/data.

  2. Run nosetest3 in the top level directory. This will find and run all the tests in the tests directory.

Although I'm usually loathe to check in generated files to version control, I'm willing to make an exception for the .mat files containing the test data. My reasoning is that they represent the decoupling of my code from the MATLAB code, and if the two projects were separated, they would be considered a part of the Python code, not the original MATLAB code.

Owner
Jason Heeris
Jason Heeris
Scalable audio processing framework written in Python with a RESTful API

TimeSide : scalable audio processing framework and server written in Python TimeSide is a python framework enabling low and high level audio analysis,

Parisson 340 Jan 04, 2023
Expressive Digital Signal Processing (DSP) package for Python

AudioLazy Development Last release PyPI status Real-Time Expressive Digital Signal Processing (DSP) Package for Python! Laziness and object representa

Danilo de Jesus da Silva Bellini 642 Dec 26, 2022
Deep learning transformer model that generates unique music sequences.

music-ai Deep learning transformer model that generates unique music sequences. Abstract In 2017, a new state-of-the-art was published for natural lan

xacer 6 Nov 19, 2022
Python tools for the corpus analysis of popular music.

CATCHY Corpus Analysis Tools for Computational Hook discovery Python tools for the corpus analysis of popular music recordings. The tools can be used

Jan VB 20 Aug 20, 2022
Extract the songs from your osu! libary into proper mp3 form, complete with metadata and album art!

osu-Extract Extract the songs from your osu! libary into proper mp3 form, complete with metadata and album art! Requirements python3 mutagen pillow Us

William Carter 2 Mar 09, 2022
Just-Music - Spotify API Driven Music Web app, that allows to listen and control and share songs

Just Music... Just Music Is A Web APP That Allows Users To Play Song Using Spoti

Ayush Mishra 3 May 01, 2022
cross-library (GStreamer + Core Audio + MAD + FFmpeg) audio decoding for Python

audioread Decode audio files using whichever backend is available. The library currently supports: Gstreamer via PyGObject. Core Audio on Mac OS X via

beetbox 419 Dec 26, 2022
Python I/O for STEM audio files

stempeg = stems + ffmpeg Python package to read and write STEM audio files. Technically, stems are audio containers that combine multiple audio stream

Fabian-Robert Stöter 72 Dec 23, 2022
Speech Algorithms Collections

Speech Algorithms Collections

Ryuk 498 Jan 06, 2023
In this project we can see how we can generate automatic music using character RNN.

Automatic Music Genaration Table of Contents Project Description Approach towards the problem Limitations Libraries Used Summary Applications Referenc

Pronay Ghosh 2 May 27, 2022
Pianote - An application that helps musicians practice piano ear training

Pianote Pianote is an application that helps musicians practice piano ear traini

3 Aug 17, 2022
Musillow is a music recommender app that finds songs similar to your favourites.

MUSILLOW The music recommender app Check it out now!!! View Demo · Report Bug · Request Feature About The App Musillow is a music recommender app that

3 Feb 03, 2022
DeepSpeech is an open source embedded (offline, on-device) speech-to-text engine which can run in real time on devices ranging from a Raspberry Pi 4 to high power GPU servers.

Project DeepSpeech DeepSpeech is an open-source Speech-To-Text engine, using a model trained by machine learning techniques based on Baidu's Deep Spee

Mozilla 20.8k Jan 03, 2023
Convert complex chord names to midi notes

ezchord Simple python script that can convert complex chord names to midi notes Prerequisites pip install midiutil Usage ./ezchord.py Dmin7 G7 C timi

Alex Zhang 2 Dec 20, 2022
Accompanying code for our paper "Point Cloud Audio Processing"

Point Cloud Audio Processing Krishna Subramani1, Paris Smaragdis1 1UIUC Paper For the necessary libraries/prerequisites, please use conda/anaconda to

Krishna Subramani 17 Nov 17, 2022
MIDI-DDSP: Detailed Control of Musical Performance via Hierarchical Modeling

MIDI-DDSP: Detailed Control of Musical Performance via Hierarchical Modeling Demos | Blog Post | Colab Notebook | Paper | MIDI-DDSP is a hierarchical

Magenta 239 Jan 03, 2023
Datamoshing with FFmpeg

ffmosher Datamoshing with FFmpeg Drag and drop video onto mosh.bat to create a datamoshed video. To datamosh an image, please ensure the file is in a

18 Sep 11, 2022
Open-Source bot to play songs in your Telegram's Group Voice Chat. Powered by @Akki_ThePro

VcPlayer Telegram Voice-Chat Bot [PyTGCalls] ⇝ Requirements ⇜ Account requirements A Telegram account to use as the music bot, You cannot use regular

Akki ThePro 2 Dec 25, 2021
A Python wrapper for the high-quality vocoder "World"

PyWORLD - A Python wrapper of WORLD Vocoder Linux Windows WORLD Vocoder is a fast and high-quality vocoder which parameterizes speech into three compo

Jeremy Hsu 583 Dec 15, 2022
Code for paper 'Audio-Driven Emotional Video Portraits'.

Audio-Driven Emotional Video Portraits [CVPR2021] Xinya Ji, Zhou Hang, Kaisiyuan Wang, Wayne Wu, Chen Change Loy, Xun Cao, Feng Xu [Project] [Paper] G

197 Dec 31, 2022