Mackenzie Mathis’ lab at EPFL introduces SuperAnimal, an AI model that autonomously tracks animal movements, revolutionizing behavioral analysis across a diverse range of species.
A groundbreaking advancement in animal behavior analysis has been achieved by Mackenzie Mathis’ laboratory at EPFL, a public research university in Lausanne, Switzerland, with the development of “SuperAnimal,” a cutting-edge AI tool. This new open-source technology can automatically detect the positions of keypoints in animals without any human input, dramatically reducing the time and labor required for such tasks.
The innovative tool, detailed in a recent Nature Communications article, can analyze movements in over 45 different animal species, including even some mythical creatures. Traditionally, behavioral phenotyping involved significant human effort to label keypoints, leading to inconsistencies and duplication of work. SuperAnimal changes the game by providing a harmonized, pre-trained model that streamlines the process.
“The current pipeline allows users to tailor deep learning models, but this then relies on human effort to identify keypoints on each animal to create a training set,” Mackenzie Mathis, said in a news release. “This leads to duplicated labeling efforts across researchers and can lead to different semantic labels for the same keypoints, making merging data to train large foundation models very challenging. Our new method provides a new approach to standardize this process and train large-scale datasets. It also makes labeling 10 to 100 times more effective than current tools.”
This technology builds upon Mathis’ previous work with the pose estimation tool “DeepLabCut™️.” SuperAnimal simplifies the laborious task of compiling and labeling training data by employing a foundation model approach.
“Here, we have developed an algorithm capable of compiling a large set of annotations across databases and train the model to learn a harmonized language – we call this pre-training the foundation model,” Shaokai Ye, the study’s first author and a doctoral student researcher at EPFL. “Then users can simply deploy our base model or fine-tune it on their own data, allowing for further customization if needed.”
With potential applications ranging from veterinary diagnostics to neuroscientific research, SuperAnimal is poised to become an indispensable tool.
“Veterinarians could be particularly interested, as well as those in biomedical research – especially when it comes to observing the behavior of laboratory mice. But it can go further,” added Mathis, hinting at uses in sports science and other fields involving precise motion analysis.
Additionally, Mathis and Ye are looking to expand their model’s capabilities.
“We also will leverage these models in natural language interfaces to build even more accessible and next-generation tools,” Mathis said.
One such project, AmadeusGPT, enables querying video data through spoken or written text and aims to introduce innovative methods in complex behavioral analysis.
SuperAnimal’s model is now freely accessible to researchers worldwide through its open-source distribution, allowing for broad adoption and continued advancements in the field.