# How to cluster images based on meta-information in tags

Context and Motivation

I have researched online for an algorithm (independent of a programming language AND in the context of Machine Learning) that accepts images as inputs with the expectation that individual images contain tags in the form of semi-structured information (for example, in the JSON format) that identify physical objects inside each image.

My goal is to find a clustering algorithm that will cluster objects based on common occurences of strings (i.e. physical objects described by a word, for example 'tree', 'mountain', 'airport' etc) contained in the tags about the images AND NOT on the input features of individual images contained in the pixels.

Question

My question is a binary question (Yes / No) - has such an algorithim been studied in published papers? If yes, can you refer me to a source or a list of sources that explores, explains, discusses, codes AND/OR contains instructions on how to implement the algorithm?

Input:

A set of images with semi-structured tags identifying physical objects in each indiviual image.

Output:

A set of clusters where images are grouped based on commonly occuring objects in the images.

• You mention that "images contain tags" e.g., in JSON format. Are you talking about the metadata associated with the image? If so, it seems the image itself is irrelevant and you're really asking how to cluster image files based on their metadata. Is that correct?
– D.W.
Apr 12, 2019 at 23:48
• I confirm you are correct. Apr 13, 2019 at 0:05

What you have is a set of texts, and you want to cluster them based on similarities. You simply want to do text clustering. A very simple way to start is to create a vector that represent each of the words. Suppose that you have 1000 words. Sort them, and you have a 1000 dimensional vector. Now, each text (metadata) is represented by a vector with a 1 in position $$i$$ if the $$i$$th word is in the text and 0 otherwise.
Do clustering on these vectors as you normally would, you can easily start of with $$k$$-means clustering.