Meta has introduced a brand new device that may generate high-quality, lifelike audio from textual content prompts referred to as AudioCraft.
The AudioCraft program options three AI instruments referred to as MusicGen, AudioGen, and EnCodec to construct its prompts from scratch. The MusicGen mannequin was skilled completely on Meta-owned and particularly licensed music, whereas AudioGen was skilled on public sound results. The third element, the EnCodec decoder permits high-quality music technology with fewer artifacts. Meta says it’s releasing its pre-trained AudioGen fashions, whereas sharing all the AudioCraft mannequin weights and code.
The AudioCraft household of AI fashions can produce high-quality audio with long-term consistency—one thing present music AI technology fashions lack. Meta says it has simplified the general design of generative fashions for audio in comparison with prior work within the subject, whereas giving folks the total recipe for folks to play with its current fashions.
Meta says producing high-fidelity audio of any type requires modeling complicated alerts and patterns at various scales. “Music is arguably essentially the most difficult sort of audio to generate because it’s composed of native and long-range patterns, from a set of notes to a world musical construction with a number of devices,” the weblog notes.
AudioCraft works for music, sound, compression, and technology—all in the identical place. The purpose was to construct a device that’s straightforward to reuse, so individuals who wish to construct higher sound mills or algorithms can accomplish that.
“We see the AudioCraft household of fashions as instruments for musicians and sound designers to supply inspiration, assist folks rapidly brainstorm and iterate on their compositions in new methods,” the Meta weblog shares. “We are able to’t wait to see what folks create with AudioCraft.”
“Having a strong open supply basis will foster innovation and complement the way in which we produce and take heed to audio and music sooner or later. With much more controls, we expect MusicGen can flip into a brand new sort of instrument—similar to synthesizers after they first appeared.”