2
$\begingroup$

The backstory

I have spectrograms of certain repeating sounds. The background of the spectrograms are black and could easily be made transparent if that would help. The sounds are easily distinct from the background. As shown below, each sound is a series of connected pixels of various colors.

example output from program second example output

What I want to do

I would like to separate each of the sounds into separate images or slices. That is, from the above example I would like to isolate the sounds into something like the following example.

sliced image produced by algorithm

If you are familiar with Unity 3D, they do something similar with their "Sprite Slicer" that creates a rectangle around objects on a transparent image. image courtesy Unity3D docs

My attempt of an algorithm

Select non-black pixel
For each neighboring pixel
  If pixel is black
    Try each non-visited neighboring pixel again, traveling a maximum of three pixels away ( a sort of threshold )
  If pixel is non-black
    Add to a list of significant pixels and travel to neighboring pixels
Mark selected pixels as visited and select a new non-black pixel

This would result in a list of lists containing pixels where each list is the extracted sound or cluster of pixels that could be compiled into images.

My concerns

I know this has been done before and is called something, but for the life of me I cannot find the words to describe it (hence the poor title).

My algorithm feels non-standard and potentially slow and poorly written. It should work, but it's not it.

Questions

What is this called, how can I find more about it?

How do I succeed in extracting the sub-images as shown by the example?

$\endgroup$
4
$\begingroup$

It seems that you're just trying to find the connected components of a graph. In this case, the vertices are the non-black pixels and there's an edge between two pixels if they're at distance at most $d$, for some appropriate choice of $d$ and some appropriate metric (Euclidean distance, Manhattan distance, whatever works best).

(Note that, in terms of implementation, you don't need to construct an explicit data structure for the graph; you can calculate whether there's an edge between any two vertices whenever you need to.)

$\endgroup$
3
$\begingroup$

You are probably looking for something like image segmentation.

$\endgroup$
2
$\begingroup$

This is otherwise known as connected component labeling.

Or rather, kind of. You can extend CC labeling into image segmentation easily enough by keeping track of the bounding rectangle of each label and then afterwards copying any correctly labeled pixel to the corresponding pixel on the rectangle.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.