Build a Deep Audio Classifier with Python and Tensorflow 

This is exactly what I was looking for the past one month, and suddenly popped up on my recommendation! Can't thank you enough for this. You saved my semester!!

1 minute into the video, absolutely amazed by the high high quality of this video. You are my favorite programming youtuber along with FireShip and NomadCoders! Thanks so much Nicholas!

You’re the absolute best with these tutorials

Just discovered this channel on my recommended. Really awesome stuff man! Thanks for the great content.

Thank you so much for these tutorials, Nicolas. Will be great a tutorial about few shot learning. Grettings from México!

41 minutes into the video. Not even a second I was bored. Amazing

Thanks so much for all these great tutorials! I’ve discovered your channel a few days ago, your way of teaching makes it really easy to understand and learn. I was wondering if you’d be able to do a series or video around recommender systems: building a recommendation engine (content-based, collaborative filtering), rather Netflix (movie) recommendations, Spotify’s music recommendation (could include audio modeling) or Amazon (purchases) predictions. Many thanks! Keep up the amazing tutorials :)

Another great content from Nick. Thrilled to do a project on this

Man these tutorials are amazing. Congrats for creating such great content. And thank!!

Awesome sound classification project👍I need a capuchino break after hearing the capuchind bird sound😎

Super, super helpful getting my project off the ground!

This video is Awesome!!! I got to know from this video that we convert Audio data to image data, to approach audio related tasks in ML!!!

Mc Shubap is spinning the decks in your memory palace 😆 Great tutorial so far.

Great content, keep it up man!

Great tutorial! Thanks!!

I worked on similar project on Audio classification hooked up on raspberry with some sensors three months ago but using rcnn and librosa. A different approach from yours basically the same steps. Thumb up mate!!

What an amazing video. Definitely earned my sub.

Bro, great video and very good detailed explanation. 👍👍👍

Incredible video, much appreciated, on the side note, I love your face cam, also audio is excellent!

Thank you very much for your hard work. I really enjoy the video.

Thank you so much for these nice tutorials! They are quite helpful! I have a small question. I saw your process of building up models and training and testing them. If I want to spend less time in classifying the model, do you think it's possible to introduce some existing datasets such as esc-10 or esc-50 in your method?

What a great video is this. Thank you so much !!! 😍

Amazing content and explanations. You have a new subscriber and fan!

bro you are absolute gold, thank you so much for all the effort you put on your videos and teachings +1 subscriber

Great video. Would love to see a version of this done using the GPU, with multiple classifications, not just binary.

Nice tutorial. A few inaccuracies there though about stft() usage: "abs()" there is not for getting rid of negatives, but for complex values amplitude. frame_length would probably better be power of 2...

This was awesome 🔥

Nice one! Looking forward to a multi label text classification if you can! Thanks

Thanks for a great video. Could please explain why we need tf.abs in preprocess function?

Very Cool video Nick Bro!!

16:36 44.1KHz is the sampling rate. Meaning every 1/44.1K sec, the amplitude of the audio wave is sampled.

Amazing job

This is amazing

Thank you so much!!

If you're encountering a shape issue when calling model.fit and getting the error "cannot take length of shape with unknown rank," the solution might be to explicitly set the shape of your tensors during preprocessing.

Amazing tutorial. Really enjoyed that Nick 🙏🏼 I guess in case we wanted to detect multiple sounds from the same animal the procedure would be the same but we would need an equal number of samples to train the neural network? Furthermore in case we wanted to detect sounds from multiple animals and categorize them would we follow the same procedure just at the end we would put softmax instead of sigmoid ?

Thank you Eran

Amazing job Nicholas!! I have just a question, why didn't you calculate also the standard deviation of files' lenght so to have a more precise interval for your window?

Another great video. I love the setup and style of your videos now, have to love a good whiteboard! I'd be interested in seeing a drone video - I've recently got a Ryze tello. Enjoying the python package to control, and computer vision capabilities that come with it!

Thanks for uploading a great video and one question! You sliced an audio file from the very start of the file. I think the position where we start slicing can affect the model accuracy. For instance, if you skip 1s and start slicing, it may yield different wav data. Do you think slicing audio from the very start of the file is golden rule?

Thank you for the great comment. How do you classify multiple sounds in a file and get time information as well like a sound was made at time 5 seconds into the audio file and another was made at 8 seconds how do we get time and class?

VERY COOL!!

Awesome, kindly make a video on Speech Recognition model using Deep Learning.

good tutorial. Note that sample rate has nothing to do with the amplitude of an audio file but rather the number of times the audio file is sampled per seconds.

can I use this for classifying different guitar chords' audio?

Hey nicholas, this project is great but how do i classify multiple animal calls using this model? I can classify the audio as capuchin or not capuchin this way but if i included more audio classes, how could i classify the audio file as the animal as well as the number of counts ?

How to decide the values of frame_length and frame_step in tf.signal.stft(wav, frame_length=320, frame_step=32) ? Appreciate any help !

Awesome explanation. Please help me to create audio augmentation for health records without losing information. I worked with time_shift(-0.5 to 0.5 variation in the wav). But, model ACC is not up to the mark.

thanks for the great video, is it possible to use .mp3 files in python instead of .wav to save disk space?

Very smart! can you identify a person speaking in the midst of others speaking more quietly?

Maybe a basic question, but what does zero padding do when getting the frequency spectrum?

A much needed tutorial !! btw can you make a tutorial on tts synthesis? not with pyttsx3... train a model to speak from provided voice data of a human

Im not sure about the batch and prefetch part, for me i generates empty training sets afterwards and also it only takes 8 prefetched files for training?

23:30 What is the significance of that len(pos) or len(neg)? When len(pos) is replaced with 2 , I expect only first 2 sample data to have '1' label. However when I run -> positives.as_numpy_iterator().next(), I get '1' labelled not only for the first 2 samples but also for the rest.

Hello Nicholas, thanks for you video :) I have a question I am aiming to build a model for sound detection using deep learning algorithms ( I am thinking about using LSTM). for now I am done with preprocessing step. I have the spectrograms of the sounds (generated using Short time Fourier transform) also I have the labels (binary labels as arrays, 0s where there are no events and 1s where the events are present). I am now confused about who to fed this data to the model. The shape of each spectrogram is (257, 626) and the shape of each label is (626,). How should I give this data to the LSTM. Can I build a model that takes the spectrograms with their current shape and give the labels as sequence of ones and zeros or I have to segment the spectrograms and give each segment a label?

Why we need to calculate Mean , Min and Max in 30:20

58:00 - Another great one, thank you, already looking forward to applying. Quick question - why mix the audio signals on the MP3. I get that it gets us to one channel - is there a way to just process one channel at a time. Im imagining that would allow for some spatial awareness in the model? Or perhaps too many variables because we are just looking for the one sound? Thinking that it would be useful to associate density with directionality... but not sure that's accurate if the original recordings were not setup to actually be directional...

Great video thanks so much, I have a small question, In the line Spectogram = tf.signal.sftf Why you choose Fram_step =320 Fram_length=32 Can some one explain the method of choosing this please?

Thanks a lot

AMAZING

Thanks so much sir, one request sir can you explain about how to classify audio as positive, negative and neutral

Hi Nicholas, how much data would I need if I just wanted to train on a single type sound ?

great 🔥

Grazie!

It's a very interesting video. But can we do the test using sound sensor?

Hey, I have a question. I am working with the Mozilla Common Voice dataset and I converted the audio files to a wav file. Now there comes my problem. I want a mfcc of the files with the shape (128,) but it is not possible for me to get it to this shape. I always get a shape like (128, and here a random number) My Code is: y, sr = librosa.load(os.path.splitext(f"{base_name}\\{f[0]}")[0] + ".wav") y = librosa.to_mono(y) y = librosa.resample(y, orig_sr=sr, target_sr=16000) mfcc = librosa.feature.mfcc(y=y, sr=sr, n_mfcc=128) f is the filename extracted of the csv file.

Hey Nick, i try to do like you but a different dataset and i had an error called : Bad bytes per sample, expected 2 but got 4, I checked my data and it's a wav file with 16000 Hz. What should i do?

Hey Nicholas, I was wondering if there was a way to change the code to print the timestamps in the audio clip that each sound is recognized?

@NicholasRenotte The kernel appears to have died, It will restart automatically. Got this error while running model.fit. How can I solve this?

good video, new subscriber, but is there a way to export that training model, to be able to use it from a python file that can do sound detection through the use of said model

thanks for the great tutorial as always sir in case i want to make voice recognition to identify for which person is this voice is this steps will help me to reach that or do i need to look for something which is more specific to this task.

Thanks for such a great tutorial. I have a question: What happens when resampling is done to an audio file? Does its total time changes or its number of sample changes or both changes or it depends on specific algorithm?

LET'S GO, NICK :D This is actually pretty awesome and once again one of those things that i feel you can do TONS of different things with. Voice commands and animal calls are obvious examples, but maybe you could build a device that listen for a human's breath and heartrate and stuff like that and detects irregularities? This could be used for diagnostic purposes, but also as a warning device for for example elderly people. The moment it hears weird heart or breath sounds or something, it gives a warning and tells them to see a doctor. Same thing can be applied in a bunch of different fields, i think. Listening for weird engine sounds for example to help diagnose engine problems before internal parts suddenly and violently become external parts. Also, astronomy! Listening for gravitational wave events and things like that, though I'm pretty sure they're already using tons of AI for this anyway, so it's probably being done already. By the by, you posted about crowdsourcing labels/labeled data the other day, i think that's a great idea, especially if you're sharing that labeled data with the public! Doing it this way is a much more manageable way to get labeled data since hopefully the work is being done in parallel by distributed resources ("many humans") and sharing it online means that this project essentially helps everyone wanting to play around with ML and use the data for something cool, learn about things and maybe come up with awesome ideas to change our future. So, awesome idea and great way to leverage the outreach you have here on YT, I love it! ❤ What i really liked about this video in particular is the exploratory data analysis and the preprocessing alongside it. As we all know, it's very important to feed your data to your network in a way that makes it as easy to digest as possible, so learning more about this is absolutely essential and really fun aswell! Much appreciated! As always, Nick, thanks a ton for your videos and for doing all this for us, much much appreciated! Really love this video and looking forward to the next one! All the best, have a great week! :)

What happens when there are multiple birds in the dataset. Now how good is a CNN model on this kinda dataset? Again the source training audio samples can vary in length, sometimes in minutes. How can we do the pre processing in such cases?

data = data.map(preprocess) in this part im getting an error -----------------> TypeError: tf__decode_wav() got an unexpected keyword argument 'rate_in' although rate_in is the parameter of tfio.audio.resample

I've gotten through to the part where you start testing the predictions and my validation_data isn't showing up. The epochs run fine, but I have no val_precision or val_loss values. All I have are loss and precision_2. Any fixes?

if anyone get error at 22:14 for pos = tf.data.Dataset.list_files(POS+'\*.wav') neg = tf.data.Dataset.list_files(NEG+'\*.wav') just use the / instead of \. pos = tf.data.Dataset.list_files(POS+'/*.wav') neg = tf.data.Dataset.list_files(NEG+'/*.wav')

In the cell 'hist = model.fit(train, epochs=1, validation_data=test)' the Kernel crashes in my case. Seems to be a tensor flow problem. I tried to install different versions of tensor flow, it didnt work though. Does anyone have any advice?

How can we use Linear predictive coding in the preprocessing function of this code?

1k! ) //THANK YOU! 🤗

Thanks

Hi i am working with a new audio dataset which does not require audio slicing part? What should I modify to loop through the folder for the last part. Any help would be greatly appreciated

what computer specifications do you have? training the model for me took 8 hrs

what approach is the best to find a number of blows in the recording of a Jackhammer?

hey nicholas,trying to execute this but facing issue as you havent done any preprocessing on the training data,looking for help from you

Had a lot of issues getting this to work. You need python 3.8.18 for this to work. I had that version of python on my conda env.

Hello sir! I installed tensor flow as presented but I don't understand the reason of the error message,no module named tensor flow.

I like this a lot was curious if you can make a video on selenium and chromedriver or something to do with browser automation

Is it possible to run this while i have my mic always listening and to do live proccesing on that? Btw this will be my first project and i know its a lot.

Please make video on making very own speech recognition using tensor flow

Hi Nick ! Thank you for your extremely useful contribution to the AI learners' community

Tried to recreate the same. Somehow the program abruptly stops at hist = model.fit(train, epochs=4, validation_data=test) It has to do with using lot of RAM. Anyway for me? Thanks!

Whats the python version being used in the tutorials

hey nick, i was rebuilding this project of yours now on a local machine - i was struggling with the spectogram size and memory allocation for my gpu rtx3070.. i downscaled to frame_length=160, frame_step=64 with input_shape=(748, 129,1) so far so good, now regarding the .fit of the model my training time is 50ms per epoch, final results tend to yours...did the colab train on cpu for the sake of memory with "highscale" spectograms maybe? could you run that locally and give some info? im very insecure regarding the train time of 50ms to the 3mins in the vid.. thanks anyways for the great tutorial man!!!

It's him, he is back.

How did you uploaded the file as colab does not support folder upload until it is in zip file format

Hello Nicholas, I have been using this great video in my beginners courses and last year everything was fine. Unfortunately in today's lecture the code did not run on any of my machines or configurations ... The load_16k_ wav function is not able to resample the audio files, and much worse, the model.fit function complains about an input that could not be -1, 0 or 1. Are you willing to check and update your code ? (Spend 6 hours now to find the error😊) Thx jtm

Hey Nick. I'm getting a Unicode Decode error when I'm trying to grab a single batch of training data using numpy iterator. Can you please help?

How do you save the model and use it somewhere else?

I need help getting compatible version of tensorflow and tensorflow-io. The latest release of tensorflow io is 0.35.0, however my os is saying that only up to 0.31.0 is available. My tensorflow is updated to the latest version and I have Python 3.11.6.