# Speech vs Music classification

I want to determine which parts of an audio file contain speech respectively music.

I hope someone has a made something like this or can tell me where to start. Can you please suggest some method or tutorial for doing the same?

-

## migrated from cstheory.stackexchange.comJun 12 '12 at 21:07

This question came from our site for theoretical computer scientists and researchers in related fields.

There are a lot of papers/documentation online, just search for "music speech discrimination", or take a look at this: "Speech and Music Classification and Separation: A Review" –  Vor Jun 7 '12 at 7:56
@Kaveh: “If you are looking for algorithms then the question might be OK here”: Please be careful to make a general statement like that. I think that from OP’s viewpoint, he or she is obviously looking for algorithms (which classify audio to speech and music), so according to what you said, “the question might be OK here.” But the issue with this question is that the problem to solve is not well-defined from the TCS point of view: “speech” and “music” are just what human interpret a sound as, and their meanings cannot be defined mathematically without referring to human. (more) –  Tsuyoshi Ito Jun 9 '12 at 11:40
(cont’d) Exploring the characteristic differences of what human interprets as speech and what human interprets as music is not in the scope of TCS, and this is the main point of the question. –  Tsuyoshi Ito Jun 9 '12 at 11:43
@TsuyoshiIto: IMO the difference between a BIG "$2^n$" and a small "$n^k$" is also a "human interpretation" :-) –  Vor Jun 9 '12 at 13:26
@Vor: No. The difference between $2^n$ and $n^k$ is well-defined in mathematical sense. Interpreting this difference as important is a human interpretation, but the interpretation does not affect the definition of, say, P. –  Tsuyoshi Ito Jun 9 '12 at 13:28

## 2 Answers

The appropriate technique is machine learning. Some keywords you could search for are "music speech discrimination", and you could look at this survey. (These pointers came from Vor's comment.)

-

If you have many data ( large number of audio pieces with human labels), I would suggest you try Deep Convolutional Neural Networks. But I think there should be a much more direct way than that, since I believe the spectrum could be a very good feature for discriminate decisions.

-
Do you have any evidence why those networks can be expected to work well for this problem? –  Raphael Feb 4 '13 at 12:16
@Raphael, I think those deep networks learn very good feature representation of audio signals -- see this example google.com.hk/…. Of course, theories to explain why these neural networks work well are still within the scope of active research. –  Strin Feb 6 '13 at 8:23