Sunday, July 19, 2015

Emotion Recognition and Tone Characterization by DNN + ELM

Let's face it, emotions are tricky!

Method

After struggling with emotion recognition for long time, I decided to implement the method proposed by Microsoft Research last year in the Interspeech conference, an approach using a deep neural network(DNN) and extreme learning machine(ELM). In particular, a DNN was trained to extract segment-level (256 ms) features and ELM was trained to make decisions based on the statistics of these features on a utterance level.

The training was done using the Interactive Emotional Dyadic Motion Capture (IEMOCAP) database from here.

Experimental results in the paper demonstrated that this approach outperforms the state-of-the-art, including OpenEAR.

Here you can also find a video presentation by the author Kun Han himself.

My implementation

all code can be found here

Dependency:

python speech features library: used to extract MFCC features from audio files, which can be replaced by the common feature extraction component once the Red Hen Lab Audio Analysis pipeline is established. But to run the current code, this library must be included in the working folder.

PDNN: A Python Toolkit for Deep Learning, which is used to extract segment-level features by deep neural network. 

Python files:

energy.py
Takes a speech signal and returns the indices of frames with top 10% energy.

Given two audio folders (training and validation, see "folder structure" for the structures of these folders), extracts the segment-level features from audio files in these folders for DNN training.

Given one (testing) audio folder, extracts the segment-level features from audio files in the folder for DNN feature extraction.

Train ELM with the probability features extracted by DNN.

Annotate the recognition results of the test files into Results.txt

Folder Structure:

wav_train (training folder):
has as many subfolders as there are emotions(or tones) to be recognized, each subfolder corresponds to one emotion (or tone) and has one utterance as one separate .wav audio file inside it.

wav_valid (validation folder):
has as many subfolders as there are emotions(or tones) to be recognized, each subfolder corresponds to one emotion (or tone) and has one utterance as one separate .wav audio file inside.

wav_test (testing folder):
has one utterance as one separate .wav audio inside. These files will be annotated and the results are written in Results.txt

Shell Scripts:

Extract features and train DNN and ELM. You can specify the pointer to PDNN, the training folder, and validation folder in this file.

Extract features of test files and recognize the emotions of utterance files in test folder. You can specify the pointer to PDNN and the testing folder here.

Intermediate Files:

dnn.param
contains the parameters of the trained deep neural network

dnn.cfg
contains the configurations of the trained deep neural network

ELMWeights.pickle.gz
contains weight parameters of the extreme learning machine

LabelNumMap.pickle.gz
contains the mapping between the emotion string labels in training/validation folders and integer labels during DNN training.

Output File:

Results.txt
the recognition results

Usage Example:

To train a new emotion recognizer (after specifying the training and validation data):
./train.sh

To predict emotions on new audio files with trained model: 
./test.sh

65 comments:

  1. That was a great message in my carrier, and It's wonderful commands like mind relaxes with  understand words of knowledge by information's.
    Python Training in Chennai

    ReplyDelete
  2. It is amazing and wonderful to visit your site.Thanks for sharing this information,this is useful to me...
    Android Training in Chennai
    Ios Training in Chennai

    ReplyDelete
  3. I really enjoy simply reading all of your weblogs. Simply wanted to inform you that you have people like me who appreciate your work. Definitely a great post I would like to read this
    Click here:
    python training in marathahalli
    Click here:
    python training in btm

    ReplyDelete
  4. I was looking for this certain information for a long time. Thank you and good luck.
    Click here:
    python online training
    Click here:
    python training in usa

    ReplyDelete
  5. Well done! Pleasant post! This truly helps me to discover the solutions for my inquiry. Trusting, that you will keep posting articles having heaps of valuable data. You're the best! 
    Click here:
    python online training
    Click here:
    python training in usa

    ReplyDelete
  6. Awesome article. It is so detailed and well formatted that i enjoyed reading it as well as get some new information too.
    java training in chennai | java training in USA

    ReplyDelete
  7. Hmm, it seems like your site ate my first comment (it was extremely long) so I guess I’ll just sum it up what I had written and say, I’m thoroughly enjoying your blog. I as well as an aspiring blog writer, but I’m still new to the whole thing. Do you have any recommendations for newbie blog writers? I’d appreciate it.

    AWS Interview Questions And Answers


    AWS Training in Bangalore | Amazon Web Services Training in Bangalore

    AWS Training in Pune | Best Amazon Web Services Training in Pune

    Amazon Web Services Training in Pune | Best AWS Training in Pune

    AWS Online Training | Online AWS Certification Course

    ReplyDelete
  8. Whoa! I’m enjoying the template/theme of this website. It’s simple, yet effective. A lot of times it’s very hard to get that “perfect balance” between superb usability and visual appeal. I must say you’ve done a very good job with this.

    AWS Training in Toronto | Amazon Web Services Training in Toronto , Canada

    AWS Training in New york city | Amazon Web Services in New York City


    AWS Training in London | Amazon Web Services Training in London , UK

    ReplyDelete
  9. Thanks for posting this info. I just want to let you know that I just check out your site and I find it very interesting and informative. I can't wait to read lots of your posts

    angularjs online Training

    angularjs Training in marathahalli

    angularjs interview questions and answers

    angularjs Training in bangalore

    angularjs Training in bangalore

    ReplyDelete
  10. Great post! I am actually getting ready to across this information, It’s very helpful for this blog.Also great with all of the valuable information you have Keep up the good work you are doing well.
    advanced excel training in bangalore

    ReplyDelete
  11. I am a regular reader of your blog and being students it is great to read that your responsibilities have not prevented you from continuing your study and other activities. Love
    python training in chennai | python training in chennai | python training in bangalore

    ReplyDelete
  12. The knowledge of technology you have been sharing thorough this post is very much helpful to develop new idea. here by i also want to share this.
    Best Devops training in sholinganallur
    Devops training in velachery
    Devops training in annanagar
    Devops training in tambaram

    ReplyDelete
  13. I was recommended this web site by means of my cousin. I am now not certain whether this post is written through him as nobody else recognise such precise about my difficulty. You're amazing! Thank you!
    angularjs online Training

    angularjs Training in marathahalli

    angularjs interview questions and answers

    angularjs Training in bangalore

    angularjs Training in bangalore

    ReplyDelete
  14. I like your article very much. It has many useful ideas and suggestions. I thinks it will be more helpful for my research in an efficient manner. Please try to post some more topics as well as possible.
    Python Training Institute in Chennai
    Best Python Training in Chennai
    best german classes in chennai
    German language training in chennai
    Devops Training in Chennai
    RPA Training in Chennai

    ReplyDelete
  15. I read this post two times, I like it so much, please try to keep posting & Let me introduce other material that may be good for our community.
    angularjs online training

    apache spark online training

    informatica mdm online training

    devops online training

    aws online training

    ReplyDelete
  16. Great post! I am actually getting ready to across this information, It’s very helpful for this blog.Also great with all of the valuable information you have Keep up the good work you are doing well.
    Microsoft Azure online training
    Selenium online training
    Java online training
    Python online training
    uipath online training

    ReplyDelete
  17. This information is impressive; I am inspired with your post writing style & how continuously you describe this topic. After reading your post, thanks for taking the time to discuss this, I feel happy about it and I love learning more about this topic.
    apple service center chennai
    apple service center in chennai
    apple mobile service centre in chennai
    apple service center near me

    ReplyDelete
  18. Really very happy to say, your post is very interesting to read. I never stop myself to say something about it. You’re doing a great job. Keep it up…

    Looking for Best Training Institute in Bangalore , India. Softgen Infotech is the best one to offers 85+ computer training courses including IT Software Course in Bangalore , India. Also it provides placement assistance service in Bangalore for IT.

    ReplyDelete
  19. Whatever we gathered information from the blogs, we should implement that in practically then only we can understand that exact thing clearly, but it’s no need to do it, microsoft azure training because you have explained the concepts very well. It was crystal clear, keep sharing..

    ReplyDelete
  20. You should be a part of a contest for one of the highest quality websites on the internet. I am going to highly recommend this web site!
    Click Here for more
    Jio Information Available
    Check 2019-20 List
    Find Helpline Resources

    ReplyDelete
  21. Good Post! , it was so good to read and useful to improve my knowledge as an updated one, keep blogging.After seeing your article I want to say that also a well-written article with some very good information which is very useful for the readers....thanks for sharing it and do share more posts likethis. https://www.3ritechnologies.com/course/data-science-online-training/

    ReplyDelete
  22. Excellent post, it will be definitely helpful for many people. Keep posting more like this.
    DevOps Training in Chennai

    DevOps Course in Chennai

    ReplyDelete
  23. Sharing the same interest, Infycle feels so happy to share our detailed information about all these courses with you all! Python Training In Chennai & get to know everything you want to about software trainings.

    ReplyDelete
  24. Sharing the same interest, Infycle feels so happy to share our detailed information about all these courses with you all! Do check them out
    Big data training in chennai & get to know everything you want to about software trainings.

    ReplyDelete
  25. You need to participate in a challenge for probably the best webpage on the web. I will enthusiastically suggest this site!
    best interiors

    ReplyDelete
  26. Infycle Technologies, the No.1 software training institute in Chennai offers the No.1 Big Data course in Chennai for tech professionals and students at the best offers. In addition to the Big Data course, other in-demand courses such as Python, Selenium, Oracle, Java, Python, Power BI, Digital Marketing also will be trained with 100% practical classes. After the completion of training, the trainees will be sent for placement interviews in the top MNC's. Call 7504633633 to get more info and a free demo.Best Big Data Course in Chennai | Infycle Technologies

    ReplyDelete
  27. Yuk Gabung di NAGAQQ: AGEN BANDARQ BANDARQ ONLINE ADUQ ONLINE DOMINOQQ TERBAIK

    Yang Merupakan Agen Bandarq, Domino 99, Dan Bandar Poker Online Terpercaya di asia hadir untuk anda semua dengan permainan permainan menarik dan bonus menarik untuk anda semua

    Bonus yang diberikan NagaQQ :
    * Bonus rollingan 0.5%,setiap senin di bagikannya
    * Bonus Refferal 10% + 10%,seumur hidup
    * Bonus Jackpot, yang dapat anda dapatkan dengan mudah
    * Minimal Depo 15.000
    * Minimal WD 20.000

    Games Yang di Hadirkan NagaQQ :
    * Poker Online
    * BandarQ
    * Domino99
    * Bandar Poker
    * Bandar66
    * Sakong
    * Capsa Susun
    * AduQ
    * Perang Bacarrat
    * Perang Dadu
    * BD QQ (New Game)

    Menyediakan deposit Via E-Money
    Seperti : OVO, Dana , Gopay, Link aja ,dll

    Info Lebih lanjut Kunjungi :
    Website : NagaQQ
    Facebook : NagaQQ Official
    Kontakk : Info NagaQQ
    linktree : Agen Judi Online
    WHATSAPP 1 : +855977509035
    Line : Cs_nagaQQ
    TELEGRAM : +855967014811

    ReplyDelete
  28. Happy to visit your blog, I am by all accounts forward to more solid articles and I figure we as a whole wish to thank such countless great articles, blog to impart to us…

    Data Science Training in Hyderabad

    ReplyDelete
  29. Grab the Digital Marketing Training in Chennai from Infycle Technologies, the best software training institute, and Placement center in Chennai which is providing professional software courses such as Data Science, Artificial Intelligence, Cyber Security, Big Data, Java, Hadoop, Selenium, Android, and iOS Development, DevOps, Oracle, etc with 100% hands-on practical training. Dial 7502633633 to get more info and a free demo and to grab the certification for having a peak rise in your career.

    ReplyDelete
  30. Thanks for posting the best information and the blog is very helpful.Best MicroNutrients Company in India

    ReplyDelete
  31. Movavi Video Editor 22.3.1 Crack 2022 download activation key offers new and attractive effects, transitions, and other video editing tools. Movavi Video Editor Crack Only

    ReplyDelete
  32. 30 Romantic Wedding Anniversary Wishes for Wife · To the woman I married with love in my heart. To my wife of many years, it never gets old to say I love . 10th Wedding Anniversary Wishes To Wife

    ReplyDelete