Sunday, July 19, 2015

Emotion Recognition and Tone Characterization by DNN + ELM

Let's face it, emotions are tricky!

Method

After struggling with emotion recognition for long time, I decided to implement the method proposed by Microsoft Research last year in the Interspeech conference, an approach using a deep neural network(DNN) and extreme learning machine(ELM). In particular, a DNN was trained to extract segment-level (256 ms) features and ELM was trained to make decisions based on the statistics of these features on a utterance level.

The training was done using the Interactive Emotional Dyadic Motion Capture (IEMOCAP) database from here.

Experimental results in the paper demonstrated that this approach outperforms the state-of-the-art, including OpenEAR.

Here you can also find a video presentation by the author Kun Han himself.

My implementation

all code can be found here

Dependency:

python speech features library: used to extract MFCC features from audio files, which can be replaced by the common feature extraction component once the Red Hen Lab Audio Analysis pipeline is established. But to run the current code, this library must be included in the working folder.

PDNN: A Python Toolkit for Deep Learning, which is used to extract segment-level features by deep neural network. 

Python files:

energy.py
Takes a speech signal and returns the indices of frames with top 10% energy.

Given two audio folders (training and validation, see "folder structure" for the structures of these folders), extracts the segment-level features from audio files in these folders for DNN training.

Given one (testing) audio folder, extracts the segment-level features from audio files in the folder for DNN feature extraction.

Train ELM with the probability features extracted by DNN.

Annotate the recognition results of the test files into Results.txt

Folder Structure:

wav_train (training folder):
has as many subfolders as there are emotions(or tones) to be recognized, each subfolder corresponds to one emotion (or tone) and has one utterance as one separate .wav audio file inside it.

wav_valid (validation folder):
has as many subfolders as there are emotions(or tones) to be recognized, each subfolder corresponds to one emotion (or tone) and has one utterance as one separate .wav audio file inside.

wav_test (testing folder):
has one utterance as one separate .wav audio inside. These files will be annotated and the results are written in Results.txt

Shell Scripts:

Extract features and train DNN and ELM. You can specify the pointer to PDNN, the training folder, and validation folder in this file.

Extract features of test files and recognize the emotions of utterance files in test folder. You can specify the pointer to PDNN and the testing folder here.

Intermediate Files:

dnn.param
contains the parameters of the trained deep neural network

dnn.cfg
contains the configurations of the trained deep neural network

ELMWeights.pickle.gz
contains weight parameters of the extreme learning machine

LabelNumMap.pickle.gz
contains the mapping between the emotion string labels in training/validation folders and integer labels during DNN training.

Output File:

Results.txt
the recognition results

Usage Example:

To train a new emotion recognizer (after specifying the training and validation data):
./train.sh

To predict emotions on new audio files with trained model: 
./test.sh

95 comments:

  1. That was a great message in my carrier, and It's wonderful commands like mind relaxes with  understand words of knowledge by information's.
    Python Training in Chennai

    ReplyDelete
  2. It is amazing and wonderful to visit your site.Thanks for sharing this information,this is useful to me...
    Android Training in Chennai
    Ios Training in Chennai

    ReplyDelete
  3. it is really amazing...thanks for sharing....provide more useful information...
    Mobile app development company

    ReplyDelete
  4. I really enjoy simply reading all of your weblogs. Simply wanted to inform you that you have people like me who appreciate your work. Definitely a great post I would like to read this
    Click here:
    python training in marathahalli
    Click here:
    python training in btm

    ReplyDelete
  5. I was looking for this certain information for a long time. Thank you and good luck.
    Click here:
    python online training
    Click here:
    python training in usa

    ReplyDelete
  6. Well done! Pleasant post! This truly helps me to discover the solutions for my inquiry. Trusting, that you will keep posting articles having heaps of valuable data. You're the best! 
    Click here:
    python online training
    Click here:
    python training in usa

    ReplyDelete
  7. Good Post! Thank you so much for sharing this pretty post, it was so good to read and useful to improve my knowledge as updated one, keep blogging.

    DevOps online Training|DevOps Training in USA
    Devops Training in Chennai

    Devops Training in Bangalore

    ReplyDelete
  8. Awesome article. It is so detailed and well formatted that i enjoyed reading it as well as get some new information too.
    java training in chennai | java training in USA

    ReplyDelete
  9. Hmm, it seems like your site ate my first comment (it was extremely long) so I guess I’ll just sum it up what I had written and say, I’m thoroughly enjoying your blog. I as well as an aspiring blog writer, but I’m still new to the whole thing. Do you have any recommendations for newbie blog writers? I’d appreciate it.

    AWS Interview Questions And Answers


    AWS Training in Bangalore | Amazon Web Services Training in Bangalore

    AWS Training in Pune | Best Amazon Web Services Training in Pune

    Amazon Web Services Training in Pune | Best AWS Training in Pune

    AWS Online Training | Online AWS Certification Course

    ReplyDelete
  10. Whoa! I’m enjoying the template/theme of this website. It’s simple, yet effective. A lot of times it’s very hard to get that “perfect balance” between superb usability and visual appeal. I must say you’ve done a very good job with this.

    AWS Training in Toronto | Amazon Web Services Training in Toronto , Canada

    AWS Training in New york city | Amazon Web Services in New York City


    AWS Training in London | Amazon Web Services Training in London , UK

    ReplyDelete
  11. Whoa! I’m enjoying the template/theme of this website. It’s simple, yet effective. A lot of times it’s very hard to get that “perfect balance” between superb usability and visual appeal. I must say you’ve done a very good job with this.

    AWS Training in Toronto | Amazon Web Services Training in Toronto , Canada

    AWS Training in New york city | Amazon Web Services in New York City


    AWS Training in London | Amazon Web Services Training in London , UK

    ReplyDelete
  12. Thanks for posting this info. I just want to let you know that I just check out your site and I find it very interesting and informative. I can't wait to read lots of your posts

    angularjs online Training

    angularjs Training in marathahalli

    angularjs interview questions and answers

    angularjs Training in bangalore

    angularjs Training in bangalore

    ReplyDelete
  13. Great post! I am actually getting ready to across this information, It’s very helpful for this blog.Also great with all of the valuable information you have Keep up the good work you are doing well.
    advanced excel training in bangalore

    ReplyDelete
  14. This is most informative and also this post most user friendly and super navigation to all posts... Thank you so much for giving this information to me.. 

    rpa online training |
    rpa course in bangalore |
    rpa training in bangalore |
    rpa training institute in bangalore

    ReplyDelete
  15. I am a regular reader of your blog and being students it is great to read that your responsibilities have not prevented you from continuing your study and other activities. Love
    python training in chennai | python training in chennai | python training in bangalore

    ReplyDelete
  16. The knowledge of technology you have been sharing thorough this post is very much helpful to develop new idea. here by i also want to share this.
    Best Devops training in sholinganallur
    Devops training in velachery
    Devops training in annanagar
    Devops training in tambaram

    ReplyDelete
  17. I was recommended this web site by means of my cousin. I am now not certain whether this post is written through him as nobody else recognise such precise about my difficulty. You're amazing! Thank you!
    angularjs online Training

    angularjs Training in marathahalli

    angularjs interview questions and answers

    angularjs Training in bangalore

    angularjs Training in bangalore

    ReplyDelete
  18. Whoa! I’m enjoying the template/theme of this website. It’s simple, yet effective. A lot of times it’s very hard to get that “perfect balance” between superb usability and visual appeal. I must say you’ve done a very good job with this.


    AWS Training in Velachery | Best AWS Course in Velachery,Chennai

    Best AWS Training in Chennai | AWS Training Institutes |Chennai,Velachery

    Amazon Web Services Training in Anna Nagar, Chennai |Best AWS Training in Anna Nagar, Chennai

    Amazon Web Services Training in OMR , Chennai | Best AWS Training in OMR,Chennai

    ReplyDelete
  19. I like your article very much. It has many useful ideas and suggestions. I thinks it will be more helpful for my research in an efficient manner. Please try to post some more topics as well as possible.
    Python Training Institute in Chennai
    Best Python Training in Chennai
    best german classes in chennai
    German language training in chennai
    Devops Training in Chennai
    RPA Training in Chennai

    ReplyDelete
  20. I read this post two times, I like it so much, please try to keep posting & Let me introduce other material that may be good for our community.
    angularjs online training

    apache spark online training

    informatica mdm online training

    devops online training

    aws online training

    ReplyDelete
  21. Great post! I am actually getting ready to across this information, It’s very helpful for this blog.Also great with all of the valuable information you have Keep up the good work you are doing well.
    Microsoft Azure online training
    Selenium online training
    Java online training
    Python online training
    uipath online training

    ReplyDelete
  22. This information is impressive; I am inspired with your post writing style & how continuously you describe this topic. After reading your post, thanks for taking the time to discuss this, I feel happy about it and I love learning more about this topic.
    apple service center chennai
    apple service center in chennai
    apple mobile service centre in chennai
    apple service center near me

    ReplyDelete
  23. Alleyaaircool is the one of the best home appliances repair canter in all over Delhi we deals in repairing window ac, Split ac , fridge , microwave, washing machine, water cooler, RO and more other home appliances in cheap rates

    Window AC Repair in vaishali
    Split AC Repair in indirapuram
    Fridge Repair in kaushambi
    Microwave Repair in patparganj
    Washing Machine Repair in vasundhara
    Water Cooler Repair in indirapuram
    RO Service AMC in vasundhara
    Any Cooling System in vaishali
    Window AC Repair in indirapuram

    ReplyDelete
  24. such a informative information Great work! Keep going

    Thanks
    Cpa offers

    ReplyDelete
  25. Your info is really amazing with impressive content..Excellent blog with informative concept. Really I feel happy to see this useful blog, Thanks for sharing such a nice blog..
    If you are looking for any python Related information please visit our website python training institutes in Bangalore page!

    ReplyDelete
  26. thank you so much for this nice information Article, Digitahanks for sharing your post with us.web designing training in bangalore

    ReplyDelete
  27. Really very happy to say, your post is very interesting to read. I never stop myself to say something about it. You’re doing a great job. Keep it up…

    Looking for Best Training Institute in Bangalore , India. Softgen Infotech is the best one to offers 85+ computer training courses including IT Software Course in Bangalore , India. Also it provides placement assistance service in Bangalore for IT.

    ReplyDelete
  28. Whatever we gathered information from the blogs, we should implement that in practically then only we can understand that exact thing clearly, but it’s no need to do it, microsoft azure training because you have explained the concepts very well. It was crystal clear, keep sharing..

    ReplyDelete
  29. I do agree your blog for quiz programming concepts, which is very helpful to grow up your knowledge. keep sharing more
    AWS training in chennai | AWS training in annanagar | AWS training in omr | AWS training in porur | AWS training in tambaram | AWS training in velachery

    ReplyDelete
  30. You should be a part of a contest for one of the highest quality websites on the internet. I am going to highly recommend this web site!
    Click Here for more
    Jio Information Available
    Check 2019-20 List
    Find Helpline Resources

    ReplyDelete
  31. Good Post! , it was so good to read and useful to improve my knowledge as an updated one, keep blogging.After seeing your article I want to say that also a well-written article with some very good information which is very useful for the readers....thanks for sharing it and do share more posts likethis. https://www.3ritechnologies.com/course/data-science-online-training/

    ReplyDelete
  32. Excellent post, it will be definitely helpful for many people. Keep posting more like this.
    DevOps Training in Chennai

    DevOps Course in Chennai

    ReplyDelete
  33. Thank you for posting informative insights, I think we have got some more information to share with! Do check out
    oracle training in chennai and let us know your thoughts. Let’s have great learning!

    ReplyDelete
  34. Sharing the same interest, Infycle feels so happy to share our detailed information about all these courses with you all! Python Training In Chennai & get to know everything you want to about software trainings.

    ReplyDelete
  35. Learn Amazon Web Services for excellent job opportunities from Infycle Technologies, the best AWS training institute in Chennai. Infycle Technologies gives the most trustworthy AWS course in Chennai, with full hands-on practical training from professional trainers in the field. Along with that, the placement interviews will be arranged for the candidates, so that, they can meet the job interviews without missing them. To transform your career to the next level, call 7502633633 to Infycle Technologies and grab a free demo to know more.Best AWS Training in Chennai

    ReplyDelete
  36. Sharing the same interest, Infycle feels so happy to share our detailed information about all these courses with you all! Do check them out
    Big data training in chennai & get to know everything you want to about software trainings.

    ReplyDelete
  37. You need to participate in a challenge for probably the best webpage on the web. I will enthusiastically suggest this site!
    best interiors

    ReplyDelete
  38. Infycle Technologies, the No.1 software training institute in Chennai offers the No.1 Big Data course in Chennai for tech professionals and students at the best offers. In addition to the Big Data course, other in-demand courses such as Python, Selenium, Oracle, Java, Python, Power BI, Digital Marketing also will be trained with 100% practical classes. After the completion of training, the trainees will be sent for placement interviews in the top MNC's. Call 7504633633 to get more info and a free demo.Best Big Data Course in Chennai | Infycle Technologies

    ReplyDelete
  39. Yuk Gabung di NAGAQQ: AGEN BANDARQ BANDARQ ONLINE ADUQ ONLINE DOMINOQQ TERBAIK

    Yang Merupakan Agen Bandarq, Domino 99, Dan Bandar Poker Online Terpercaya di asia hadir untuk anda semua dengan permainan permainan menarik dan bonus menarik untuk anda semua

    Bonus yang diberikan NagaQQ :
    * Bonus rollingan 0.5%,setiap senin di bagikannya
    * Bonus Refferal 10% + 10%,seumur hidup
    * Bonus Jackpot, yang dapat anda dapatkan dengan mudah
    * Minimal Depo 15.000
    * Minimal WD 20.000

    Games Yang di Hadirkan NagaQQ :
    * Poker Online
    * BandarQ
    * Domino99
    * Bandar Poker
    * Bandar66
    * Sakong
    * Capsa Susun
    * AduQ
    * Perang Bacarrat
    * Perang Dadu
    * BD QQ (New Game)

    Menyediakan deposit Via E-Money
    Seperti : OVO, Dana , Gopay, Link aja ,dll

    Info Lebih lanjut Kunjungi :
    Website : NagaQQ
    Facebook : NagaQQ Official
    Kontakk : Info NagaQQ
    linktree : Agen Judi Online
    WHATSAPP 1 : +855977509035
    Line : Cs_nagaQQ
    TELEGRAM : +855967014811

    ReplyDelete