Artificial intelligence can help you understand music better

Chords, beat, timbre, rhythm and harmony. All these elements of music contribute to make it sound the way it does. But have you thought about why you like particular kinds of music?

"Music is a magical thing, when you think about it. When you listen, you feel many emotions. You understand that it is a kind of language, but you cannot see what is happening. For most people, it is a mystery."

So says Olivier Lartillot, researcher at RITMO Center for Interdisciplinary Studies in Rhythm, Time and Motion. He is developing new digital tools that he hopes will make the magic of music available to everyone.

First on his list is Norwegian folk music.

"Folk music is so rich, and a treasure for Norwegian culture. Still, not many people listen to it. If we create a tool that can help people understand music, folk music may have a renaissance in Norway," says Lartillot.

Better understanding will open a world of music

Olivier Lartillot believes technology can expand people's taste in music through a better understanding of musical elements.

"Often, people are interested in the music that they listen to all the time, and that they understand easily. Some types of music seem more complicated, and are therefore less accessible," he says.

"If we can give people tools to understand music better, we will also provide access to a lot of new music. It will benefit not only the individual, but the music itself - and the diversity of the entire ecosystem of music."

Musicologists also lack complete knowledge about the music they study. Lartillot, who has combined advanced computational music analysis with insights from musicology for many years now, can bring them closer to the answers they are looking for.

"We are now approaching a stage in the research where we can make tools that can understand the logic of the music," says Lartillot.

Training artificial intelligence with a fiddle

Currently, the best tool for music analysis with computers is artificial intelligence and so-called machine learning.

"You train the machine by teaching it that this is a certain type of music and these are the notes it should recognize. By 'hearing' examples, the machine tries to understand what is happening. After working through enough examples, it can detect notes automatically."

The music the machine is intended to transcribe is the National Library of Norway's catalog of folk music. According to Lartillot, Norwegian folk music and especially the Hardanger fiddle is difficult material for the machine.

"The large amount of examples needed was initially not available. Therefore, we asked musicians, the professional fiddler Olav Luksengård Mjelva, and students from the Norwegian Academy of Music, to play for us, and designed a software where the sounds were visualized and they could place the notes for us."

Lartillot's colleague, postdoctoral fellow Anders Elowsson, is currently using the manual annotations to teach the machine how to automatically detect the notes played by the Hardanger fiddle.

"This is work that can take hundreds of hours, on large powerful computers," says Lartillot.

The next step, which is where they are now, is detecting the beats. This is complex in fiddle music. But when the machine masters that too, an interactive tool in the form of, for example, an app is within reach.