branches

By Jessica Weiss ’05

Students learning classical violin usually have to wait until a session with a music teacher to get personalized feedback on their playing. Soon they may have a new tool to use between lessons: an app that can observe them play and guide them toward better posture and form—key elements both for sounding their best and avoiding overuse injuries.

Two University of Maryland researchers are drawing on very different academic backgrounds—one in classical violin and music education, the other in robotics and computer science—to develop this virtual “teacher’s aide” system powered by artificial intelligence (AI) technology. In addition to expanding the market for violin instruction, it will allow students who may not have access to private lessons to receive feedback on their playing.

Associate Professor of Violin in the School of Music Irina Muresanu, who is collaborating with Cornelia Fermüller, associate research scientist in UMD’s Institute for Advanced Computer Studies, said the technology will be revolutionary for a field rooted in tradition.

“While I believe that traditional methods are still the best way to pass on to our students the legacy and heritage of the classical music world, I am excited to explore ways in which artificial intelligence can be integrated as a feedback mechanism into daily practice—the central experience of any musician’s life,” she said.

The project is part of Arts for All, a new initiative to expand arts programming across campus and bolster interdisciplinary offerings through a fusion of the arts, technology and social justice.

Muresanu and Fermüller were recently awarded a $115,000 Phase I Maryland Innovation Initiative award by the Maryland Technology Development Corporation to support the project. The award, a partnership between the state of Maryland and five of its public universities, is designed to help propel research ideas from the lab to the commercial market.

An internationally renowned Romanian violinist, Muresanu has spent the last decade working at the intersection of music and technology. She previously collaborated with Amitabh Varshney, dean of the College of Computer, Mathematical, and Natural Sciences, on “Four Strings Around the Virtual World,” which embedded Muresanu’s solo violin project in famous global locales including concert halls, cathedrals and outdoor spaces.

When the COVID-19 pandemic made in-person teaching impossible, Muresanu began seeking new ways to allow violin students to continue learning remotely. Last year, she partnered with UM Ventures, a joint technology commercialization initiative of the University of Maryland, Baltimore and University of Maryland, College Park, to explore high-tech approaches for enhancing remote lessons.

Fermüller was a natural fit for the project. A researcher of computer vision and robotics, she works to enable computers to understand and enhance what people are doing in their daily activities.

In the Autonomy Robotics Cognition Lab, Muresanu and Fermüller, along with computer science Ph.D. student Snehesh Shrestha, are studying human-robot interaction in the context of playing the violin and how to integrate AI into the learning process. The technology they are producing—which will enable computers and phones to derive information from digital video—will let music teachers customize the type and amount of feedback students receive and survey the results.

Fermüller said the technology will be a major step forward in using AI for music education, and could potentially be applied to other instruments.

“The platform we are currently working on provides feedback to students based on their specific needs, and this is very novel,” she said. “I believe this is the future of AI-supported education.”

By Jessica Weiss ’05

Assistant Professor of Sculpture and Emerging Technology Cy Keener has received a grant from the National Science Foundation to continue his work blending science, technology and art to convey the thinning of Arctic Sea ice.

The five-year grant, totaling nearly $207,000, will allow Keener to develop and test a low-cost, open-source buoy to provide meteorological and oceanographic data, a project he has been working on since 2018. In collaboration with research scientist Ignatius Rigor, a senior principal research scientist at the University of Washington, Keener will also travel to the Arctic and make visual art with data collected through the instruments deployed.

Given that such buoys normally cost thousands—or even tens of thousands—of dollars, the $300-$500 device Keener is seeking to develop could potentially “double the number of sensors” currently being used in the Arctic Ocean, he said. That could enhance a critical dataset used in weather forecasting and studies of climate and climate change.

“It’s an honor to be involved in this work,” said Keener, who teaches art and electronics at UMD. “And to use my art to get people to understand what’s happening up there.”

Keener first traveled to the Arctic in Spring 2019 with Rigor, who is the coordinator of the International Arctic Buoy Program (IABP), whose members maintain a network of buoys across the expanse of the Arctic Ocean. On that trip, Keener installed measuring instruments that he then used to create a series of art pieces. 

At VisArts Gallery in Rockville, Maryland, he displayed “Sea Ice 71.348778º N, 156.690918º W,” an installation that used hanging strips of 6-foot-long, blue-green polyester film to reflect the thickness and color of the Arctic ice as collected daily via satellite from the buoys.  

He also created various versions of “Digital Ice Core,” a sculpture piece that used electronics, data and satellite communication to link a remote field site with a digital light sculpture, made up of one thousand LED lights. Viewers were then able to see a recreated version of the ambient light in the air, ice and ocean in close to real time. 

Currently, he is seeking to improve a custom circuit and code that he has been working on since 2016, that will go in the sensors. He is hoping to travel to the Arctic in Spring 2021.

Photos courtesy of Cy Keener.