I have an NLP problem and a potential solution, but I’m a bit green here, so I’m looking for some validation or alternative suggestions.
I have two types of documents: one is a set of short statements of an organization's goals and objectives (“Goals”, from here on. ~500 docs, ~100 words/doc. New documents monthly-annually). The other is a larger set of things they've actually done (statements of work, contracts, etc. “Work”, from here on. ~10M docs, ~300 words/doc. New documents daily-weekly.).
My objective is to assign each of these Work statements to one or more Goals, if possible with a weight indicating of how closely they fit.
The hypothesis I'm working from is that each Work document is created by someone with knowledge of the Goals, and therefore there's a hidden relationship between the two that should show up in a probabilistic approach.
There's no existing data of this sort, so that seems to eliminate most supervised approaches.
I've done a lot of reading about ML/NLP, but this is the first time I've tried to do something beyond basic examples and use of pre-existing libraries. Most of my knowledge comes from Introduction to Information Retrieval, so I've been going through that trying to find something that fits.
Here's my current idea:
- Build a term-document matrix of Goals.
- Use singular-value decomposition to construct a low-rank approximation. The intention here is to narrow out both stopwords and terms common across all Goals. This part seems to be the most “magical” to me right now, so I might be misunderstanding its capabilities here.
- Construct a term-document matrix of Work.
- Compare the Goal matrix to the Work matrix (or the selection of the relevant terms from the Work matrix). Compute the distance from each Work document to each Goal document. The final result will be a matrix of Work x Goals with some kind of weight between each. My linear algebra is a bit shaky, so I can’t remember if there’s an obviously good distance function that I’m missing the intuition on. Or should it be something more like cosine similarity?
I came up with this after reading the Language models for information retrieval and Matrix decompositions and latent semantic indexing chapters in IIR. I don’t think what I’m describing falls exactly into any of those techniques like LSI, but I may be confused because they’re mostly talking about matching queries to documents, rather than other documents. Maybe my restricted-term Goal martix is actually a set of query vectors then? Or maybe this is a different technique that I picked up elsewhere?
Or maybe I’m way off and this won’t work at all? :-) I’d love some feedback. Thanks.