To help combat the danger of people near traffic wearing headphones or earbuds, oblivious to their surroundings, researchers at the Data Science Institute at Columbia University and University of North Carolina (UNC) at Chapel Hill are designing an intelligent headphone system that warns pedestrians of imminent hazards.
Known as ‘twalking’, the behaviour is prevalent on most city streets and can be dangerous for all road users, as headphone-wearing pedestrians often cannot hear the auditory cues that signal imminent harm, such as horns, shouts, or the sound of approaching cars. As a result, the number of injuries and deaths caused by twalking in the USA has tripled in the last seven years. Last year, pedestrian deaths across the country were at their highest level since 1990. To counter this growing public safety concern, researchers at Columbia University (CU) in New York City are developing a ‘smart’ headphone system that can warn users of approaching vehicles and other dangers, helping to reduce pedestrian injuries and fatalities
The research and development of the smart headphones is complex, as it involves embedding multiple miniature microphones in the headset as well as developing a low-power data pipeline to process all the sounds near the pedestrian. It must also extract the correct cues that signal impending danger. The pipeline needs to contain an ultra-low power, custom-integrated circuit that extracts the relevant features from the sounds while using little battery power. The system’s intelligent signal processing must detect and isolate the sound of approaching vehicles, then send a clearly-heard audio alert to the pedestrian’s headphones.
The researchers are also using the most advanced data science techniques to design the smart headset. Machine-learning models on the user’s smartphone will classify hundreds of acoustical cues from city streets and nearby vehicles and warn users when they are in danger. The mechanism will be designed so that people will recognize the alert and respond quickly. The team is now testing its design both in the lab and on the streets of New York; a city known for its congestion and its cacophony of sounds. Research team member, Joshua New, a psychology professor from NYC’s Barnard College, will conduct perceptual and behavioural experiments with people to see how the alerts can be effectively provided to pedestrians who walk in cities wearing headphones.
“We are exploring a new area in developing an inexpensive and low-power technology that creates an audio-alert mechanism for pedestrians,” says Fred Jiang, a Data Science Institute member and an assistant professor at CU’s Electrical Engineering Department. “Our aim is to develop a prototype of the smart headphone system at Columbia and then transfer the technology to a commercial company. “We hope that once refined, the technology will be commercialised, and mass produced in a way that will help cities reduce pedestrian fatalities.”
The smart-headphone project was awarded a US$1.2m grant from the National Science Foundation in 2017, and the research team also includes Peter Kinget, chair of electrical engineering at CU; Shahriar Nirjon, a professor of computer science at UNC; with graduate students from both universities also working on the project.
