Computer Vision - What is spatial histogram/pyramid feature?

Question

I am trying to re-implement the model in this paper https://arxiv.org/pdf/1511.02917v2.pdf. But the paper omitted some details in feature extraction (Section 4.1). I am new to computer vision so I am confused when the paper just said they got the spatial feature (1440 dimensional) that corresponds to a 32*32 spatial histogram, combined with a spatial pyramid, to indicate the bounding box location at multiple scales.

I guess a 32*32 spatial histogram is to divide the image into 32*32 regions and concatenate histogram for each small region together? But what does it mean to combine with a spatial pyramid, and how to end up with a 1440 dimensional spatial features.

Answer 1

Spatial histogram. A simple histogram is obtained by taking a region of an image, assigning a label to each pixel (somehow; via some mapping function), and then computing a histogram of the labels. The histogram counts, for each possible label, how many pixels received that label. You obtain a feature vector: a vector of counts, one count per possible label.

Such a histogram doesn't capture any information about where the labels appear. A spatial histogram addresses that shortcoming by dividing the image up into several smaller patches, computing a histogram for each patch, and concatenating those histograms. For instance, you could take a 32x32 image and break it up into sixteen 8x8 patches, compute a histogram for each patch, and concatenate them to get a feature vector.

You still have to choose an appropriate matching. You might choose a texture descriptor, or just the pixel RGB value (quantized appropriately), or something about the direction of the gradient (quantized), or something else. For more about spatial histograms, see also http://www.answers.opencv.org/question/7207/spatial-histogram/ and the following paper:

Hongming Zhang, Wen Gao, Xilin Chen, Debin Zhao. Object detection using spatial histogram features. Image and Vision Computing, vol 24 no 4, April 2006.

Pyramid. A pyramid allows you to do something at multiple scales. You start with a 32x32 image and compute a feature vector for it. Then, you rescale the image down to a 16x16 image and compute a feature vector for it. Then you rescale down to a 8x8 image and compute a feature vector for that. And so on. Finally, you concatenate all of those feature vectors. This allows you to capture both coarse features (from the highly downscaled images) as well as fine features (from the large images).

For more about pyramids, see https://en.wikipedia.org/wiki/Pyramid_%28image_processing%29.