Sound Classifier

Introduction to the project

What is AI?

Artificial Intelligence is any task performed by a program or a machine that, if a human carried out the same activity, we would say the human had to apply intelligence to accomplish the task.
AI systems will typically demonstrate at least some of the following behaviors associated with human intelligence: planning, learning, reasoning, problem solving, knowledge representation, perception, motion, and manipulation and, to a lesser extent, social intelligence and creativity.


What is a Sound Classifier?

For this project, I chose to work on AI and particularly on sounds. The SoundClassifier allows you to teach a machine and make it learn different sounds for it to recognize it right after. It is a growing area of research with numerous real world application.

Real world applications

• Content-based multimedia indexing and retrieval
• Assisting deaf individuals in their daily activities
• Smart home use cases such as 360-degree safety and security capabilities
• Industrial uses such as predictive maintenance




Audio recognition

ml5 SoundClassifier

A ml5.js tool exists to be able to detect whether a certain noise was made (e.g. a clapping sound or a whistle) or a certain word was said (e.g. Up, Down, Yes, No). At this moment, with the ml5.soundClassifier(), you can use your own custom pre-trained speech commands or use the the "SpeechCommands18w" which can recognize "the ten digits from "zero" to "nine", "up", "down", "left", "right", "go", "stop", "yes", "no", as well as the additional categories of "unknown word" and "background noise"."
You can have more information directly on the ml5.js webpage.


Train you own SoundClassifier

Google created a website for you to train your own custom sound classification model. You can try to teach you own machine via this link.



This tool is useful to create your own sound classes and see how it works in real life. The first step here is to train the machine by first adding audio samples for the background noise so that the machine can differenciate the background noises from the ones that we want to record.
Once you've done that, you can add classes and different noises that you can teach to the machine by adding audio samples. Once it's done, the machine will automatically analyze your environment to recognize the different noises like you can see on the image below :


To go further

By watching this video you will see an example of a youtuber training the Sound Classifier machine. You will also understand how to import the machine learning model into a p5.js sketch with the ml5.js library.

Articles about Sound Classification :
- Sound Classification using Deep Learning
- Real-time Sound event classification