MusicLM

What is MusicLM?

Google has released a machine learning tool that can create musical pieces based on a specified text, called MusicLM. The tool itself is not yet available to the public, but Google has posted a research paper and examples online. According to Google, MusicLM is a hierarchical sequence-to-sequence machine learning model.

What can MusicLM do?

The tool can create musical pieces based on text, whistling, humming, a picture or a painting. It outperforms previous systems and is capable of producing complex songs. MusicLM has been trained on a dataset of 280,000 hours of music and is able to generate coherent songs for descriptions with "considerable complexity." The samples sound remarkably realistic, though not necessarily as inventive or musically coherent as a human artist. Google gives Salvador Dali's painting as an example, with MusicLM composing its own song.

Google's plans with MusicLM?

Google trained the tool on a dataset of 280,000 hours of music and has also made the MusicCaps dataset publicly available to researchers. This dataset consists of 5,500 music descriptions with accompanying original music. In the paper, Google writes that they have no immediate plans to release MusicLM because of the risks of the tool producing protected material (an approximate 1% chance).