How do I apply patch sized features to larger images?

Question

I've been trying to teach myself some machine learning, and I wanted to ask what seems a simple question, but I've not been able to find any resources that explain the next step.

Let's say I am doing semi-supervised learning, and have a few hundred sparsely distributed feature extractors, trained on random 32x32 regions (the unsupervised part of the process).

I now want to take the larger images in my training set, and do some supervised learning based on the feature extractors I now have. In this case, multi-label classification.

The bit I'm not clear on is what I do with the full sized image from my training set:

• Take random samples from it? -- seems like it would be pot luck if it picks an area needed to identify appropriate labels
• Take overlapping tiles with a sliding window? -- seems like I'd end up with absurd dimensionality, since for each tile, I get a whole vector of features
• Take adjacent tiles? -- dimensionality still nonsensical, and probably translation sensitive as well

It's a hypothetical example, but let's say my inputs are 800x600 photographs, i.e. the input is about 100-1000 times bigger than the samples I used in the unsupervised learning stage.

Answer 1

The options are various.

If you want a naive and easy workaround simply resize your image (shrink it) and trust that the interpolation would preserve information.

If not, you might want to use Gaussian Pyramids or Spatial Pyramid Match Kernels:

http://www.ifp.illinois.edu/~jyang29/papers/CVPR09-ScSPM.pdf http://www.vision.caltech.edu/Image_Datasets/Caltech101/cvpr06b_lana.pdf

Other options are a bit untidy. For example, you might take overlapping samples from it, apply some sort of feature extraction and if those features are still big for you, apply a PCA or LDA for dimensionality reduction. (There are more clever sampling strategies such as farthest point sampling : http://webee.technion.ac.il/people/zeevi/papers/17-00623193.pdf

Also taking a look at the covariance matrices might provide you relevant information. Why don't you also check:

http://static.googleusercontent.com/external_content/untrusted_dlcp/research.google.com/en/us/pubs/archive/36416.pdf

Cheers,

Answer 2

I understand that you are trying to build a scene analysis algorithm based on patch based feature extractors. Apart from tbirdal's suggestions, take a look at literature on feature pooling, that is crucial for building higher level information from the output of low level features.

Take this paper for example:

A Theoretical Analysis of Feature Pooling in Visual Recognition

http://www.icml2010.org/papers/638.pdf