The University of Birmingham unveils the JAMS platform, enabling musicians to collaborate in real-time using responsive avatars. This innovation promises to bring the magic of in-person musical experiences to the virtual world, enhancing practice, teaching and performance.
The University of Birmingham has unveiled a groundbreaking innovation that promises to revolutionize the world of virtual music. The Joint Active Music Sessions (JAMS) platform, developed through interdisciplinary research, allows musicians to engage in lifelike practice, performance and teaching sessions using avatar technology.
“A musician records themselves and sends the video to another musician. The software creates a responsive avatar that plays in perfect synchrony with the music partner. All you need is an iPhone and a VR headset to bring musicians together for performance, practice or teaching,” Massimiliano (Max) Di Luca, an associate professor at the University of Birmingham who led the development team, said in a news release.
The JAMS platform transcends traditional virtual music environments by using avatars that capture the subtleties and nuances of musical performance. These avatars can realistically replicate the bow movements of a violinist or allow for meaningful eye contact between virtual band members, thereby enhancing the overall musical interaction.
Unlike other virtual music collaborations, JAMS eliminates latency issues.
“Latency is the delay between a sound production and when it reaches the listener, and performers can start to feel the effects of latency as low as 10 milliseconds, throwing them ‘off-beat’, breaking their concentration or distracting them from the technical aspects of playing,” Di Luca added.
Developed during the Augmented Reality Music Ensemble (ARME) project, JAMS is empowered by a dynamic algorithm that captures timely adjustments between performers. This innovative project pulled together experts from psychology, computer science, engineering, music, sports science and mathematics.
“We’re aiming to bring the magic of playing music in person to the virtual world. You can adapt the avatar that other people play with or learn to play better through practice with a maestro,” added Dr. Di Luca.
By offering a realistic, immersive backdrop and keeping avatars’ faces at eye level, JAMS enhances the sense of connectedness among musicians. The platform opens up new opportunities for both seasoned and novice musicians to perform in interactive virtual groups, leading to possibilities for large-scale virtual concerts and innovative music teaching methods.
Moreover, JAMS has commercial potential, including features for lipsyncing, dubbing in media and creating digital twins of musicians. These avatars could be licensed for various applications, promoting new ways to exploit catalogues and publishing rights.