# Speech vs Music classification

I want to determine which parts of an audio file contain speech respectively music.

I hope someone has a made something like this or can tell me where to start. Can you please suggest some method or tutorial for doing the same?

• There are a lot of papers/documentation online, just search for "music speech discrimination", or take a look at this: "Speech and Music Classification and Separation: A Review"
– Vor
Jun 7, 2012 at 7:56
• @Kaveh: “If you are looking for algorithms then the question might be OK here”: Please be careful to make a general statement like that. I think that from OP’s viewpoint, he or she is obviously looking for algorithms (which classify audio to speech and music), so according to what you said, “the question might be OK here.” But the issue with this question is that the problem to solve is not well-defined from the TCS point of view: “speech” and “music” are just what human interpret a sound as, and their meanings cannot be defined mathematically without referring to human. (more) Jun 9, 2012 at 11:40
• (cont’d) Exploring the characteristic differences of what human interprets as speech and what human interprets as music is not in the scope of TCS, and this is the main point of the question. Jun 9, 2012 at 11:43
• @TsuyoshiIto: IMO the difference between a BIG "$2^n$" and a small "$n^k$" is also a "human interpretation" :-)
– Vor
Jun 9, 2012 at 13:26
• @Vor: No. The difference between $2^n$ and $n^k$ is well-defined in mathematical sense. Interpreting this difference as important is a human interpretation, but the interpretation does not affect the definition of, say, P. Jun 9, 2012 at 13:28

## 2 Answers

The appropriate technique is machine learning. Some keywords you could search for are "music speech discrimination", and you could look at this survey. (These pointers came from Vor's comment.)

If you have many data ( large number of audio pieces with human labels), I would suggest you try Deep Convolutional Neural Networks. But I think there should be a much more direct way than that, since I believe the spectrum could be a very good feature for discriminate decisions.

• Do you have any evidence why those networks can be expected to work well for this problem? Feb 4, 2013 at 12:16
• @Raphael, I think those deep networks learn very good feature representation of audio signals -- see this example google.com.hk/…. Of course, theories to explain why these neural networks work well are still within the scope of active research. Feb 6, 2013 at 8:23