NeuralMidiFx

NeuralMidiFx

Loading...
Behzad Haki
Access

## Source Code

The wrapper is open-source and freely available on GitHub.

License

## License

This project is released under the MIT License.

Publication
  1. NeuralMidiFx: A Wrapper Template for Deploying Neural Networks as VST3 Plugins

    NeuralMidiFx: A Wrapper Template for Deploying Neural Networks as VST3 Plugins

    Behzad Haki, Julian Lenz, Sergi Jorda
    Proceedings of the 4th International Conference on on AI and Musical Creativity · 2023
    Abstract

    Proper research, development and evaluation of AI-based generative systems of music that focus on performance or composition require active user-system interactions. To include a diverse group of users that can properly engage with a given system, researchers should provide easy access to their developed systems. Given that many users (i.e. musicians) are non-technical to the field of AI and the development frameworks involved, the researchers should aim to make their systems accessible within the environments commonly used in production/composition workflows (e.g. in the form of plugins hosted in digital audio workstations). Unfortunately, deploying generative systems in this manner is highly expensive. As such, researchers with limited resources are often unable to provide easy access to their works, and subsequently, are not able to properly evaluate and encourage active engagement with their systems. Facing these limitations, we have been working on a solution that allows for easy, effective and accessible deployment of generative systems. To this end, we propose a wrapper/template called NeuralMidiFx, which streamlines the deployment of neural network based symbolic music generation systems as VST3 plugins. The proposed wrapper is intended to allow researchers to develop plugins with ease while requiring minimal familiarity with plugin development.

    BibTeX
    @inproceedings{Haki2023NeuralMidiFx,
      author = {Haki, Behzad and Lenz, Julian and Jorda, Sergi},
      booktitle = {{Proceedings of the 4th International Conference on on AI and Musical Creativity}},
      publisher = {},
      title = {{NeuralMidiFx: A Wrapper Template for Deploying Neural Networks as VST3 Plugins}},
      year = {2023},
      month = sep,
    }
Contact

## Contact

Behzad Haki — behzad.haki@upf.edu
Music Technology Group, Universitat Pompeu Fabra, Barcelona, Spain

Overview

## Overview

NeuralMidiFx is a wrapper template that simplifies the deployment of neural networks as VST3 plugins. The wrapper is built on the JUCE framework and uses the PyTorch C++ API.

This project was presented at AIMC 2023.

Note: The following content corresponds to the first release. The wrapper has been updated since the conference. For the latest release, visit the project page.

AIMC 2023 Content

Tutorials

A number of tutorials are available here.

Documentation

## Documentation

Full documentation is available at neuralmidifx.github.io.

The documentation covers:

  • Getting started / setup
  • Creating your first plugin
  • Deploying a neural network model
  • Advanced configuration