Computer Science Stack Exchange is a question and answer site for students, researchers and practitioners of computer science. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I want to determine which parts of an audio file contain speech respectively music.

I hope someone has a made something like this or can tell me where to start. Can you please suggest some method or tutorial for doing the same?

share|cite|improve this question

migrated from Jun 12 '12 at 21:07

This question came from our site for theoretical computer scientists and researchers in related fields.

There are a lot of papers/documentation online, just search for "music speech discrimination", or take a look at this: "Speech and Music Classification and Separation: A Review" – Vor Jun 7 '12 at 7:56
@Kaveh: “If you are looking for algorithms then the question might be OK here”: Please be careful to make a general statement like that. I think that from OP’s viewpoint, he or she is obviously looking for algorithms (which classify audio to speech and music), so according to what you said, “the question might be OK here.” But the issue with this question is that the problem to solve is not well-defined from the TCS point of view: “speech” and “music” are just what human interpret a sound as, and their meanings cannot be defined mathematically without referring to human. (more) – Tsuyoshi Ito Jun 9 '12 at 11:40
(cont’d) Exploring the characteristic differences of what human interprets as speech and what human interprets as music is not in the scope of TCS, and this is the main point of the question. – Tsuyoshi Ito Jun 9 '12 at 11:43
@TsuyoshiIto: IMO the difference between a BIG "$2^n$" and a small "$n^k$" is also a "human interpretation" :-) – Vor Jun 9 '12 at 13:26
@Vor: No. The difference between $2^n$ and $n^k$ is well-defined in mathematical sense. Interpreting this difference as important is a human interpretation, but the interpretation does not affect the definition of, say, P. – Tsuyoshi Ito Jun 9 '12 at 13:28

The appropriate technique is machine learning. Some keywords you could search for are "music speech discrimination", and you could look at this survey. (These pointers came from Vor's comment.)

share|cite|improve this answer

If you have many data ( large number of audio pieces with human labels), I would suggest you try Deep Convolutional Neural Networks. But I think there should be a much more direct way than that, since I believe the spectrum could be a very good feature for discriminate decisions.

share|cite|improve this answer
Do you have any evidence why those networks can be expected to work well for this problem? – Raphael Feb 4 '13 at 12:16
@Raphael, I think those deep networks learn very good feature representation of audio signals -- see this example…. Of course, theories to explain why these neural networks work well are still within the scope of active research. – Strin Feb 6 '13 at 8:23

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.