Questions tagged [natural-language-processing]

Natural language processing (NLP)

Filter by
Sorted by
Tagged with
2
votes
1answer
108 views

Solving the part-of-speech tagging problem with HMM

There is a famous part-of-speech tagging problem in Natural Language Processing. The popular solution is to use Hidden Markov Models. So that, given the sentence $x_1 \dots x_n$ we want to find the ...
2
votes
1answer
37 views

Trying to tag parts of a word and keep track of any changes that happen to those parts

I'm a researcher working with a language that has gone through phonological changes through time. I would like to tag parts of a word (i.e. prefix, stem, suffix) and then apply those phonological ...
2
votes
1answer
80 views

Using interval graphs to find authorship disputes

The first chapter of the book "Graphs and their uses" by Oystein Ore says that interval graphs can be used to resolve authorship disputes, but I couldn't find any details. How does this work? What ...
5
votes
1answer
537 views

Determining interests from Twitter text using Latent Dirichlet Allocation

I would like to determine interests or hobbies that people have given their Twitter timeline data. Their timeline is their historical collection of tweets. The result I am trying to achieve is to ...
4
votes
0answers
2k views

Next-Word Prediction, Language Models, N-grams

I was looking into how a next-word prediction engine like swift key or XT9 can be implemented. Here's what I did. I read about n-grams here - en.wikipedia.org/wiki/N-gram and aicat.inf.ed.ac.uk/...
1
vote
0answers
2k views

Next Word Prediction using n-gram & Tries

I am studying the following paper for understanding next-word prediction using n-gram & trie: - http://nlp.cs.berkeley.edu/pubs/Pauls-Klein_2011_LM_paper.pdf Before this, I did some brief study on ...
2
votes
2answers
144 views

Hidden Markov Model in Tagging Problem

I try to understand the details regarding using Hidden Markov Model in Tagging Problem. The best concise description that I found is the Course notes by Michal Collins. The goal is to find a ...
0
votes
1answer
393 views

short text categorization with spelling correction

I am building a classifier for short texts in a chat system. My features are words and pairs of words. Naturally, the sentences contain spelling mistakes. If a particular wrong spelling of a certain ...
0
votes
0answers
181 views

Advantages of knowing foreign languages for natural language processing

I wonder about cases in which knowing several languages can lead a researcher to interesting results in natural language processing. For example, knowledge of foreign languages can without doubt ...
2
votes
1answer
121 views

Analyzing rules of articles in languages

I'm interested in finding a solution for the following problem: problem space: any language that has more than 1 article let's take German language as example. So the articles in German are "der", "...
2
votes
0answers
185 views

The grammar of the GeoQuery language

GeoQuery is a dataset used for benchmarking semantic parsers. It contains 880 queries about USA geography. The queries are in Prolog format, for example: answer(A,longest(A,(river(A),traverse(A,B),...
3
votes
1answer
75 views

Evaluation metric for an ordering algorithm

There is a sentence with N words. The words are randomly shuffled. I have a heuristic algorithm that tries to restore the original order. I want to evaluate my algorithm on a dataset of several ...
5
votes
2answers
490 views

Algorithm to find the probability of a given text to be about a large topic

I want the conditional probability for each topic (being the word that we give as input). For example, the text being have seen and reviewed your requirements you posted here. If you can give me ...
1
vote
0answers
108 views

Help understanding an audio processing algorithm

I'm reading “Speech segmentation without speech recognition” by Dong Wang, Lie Lu and Hong-Jiang Zhang. The algorithm I'm looking at is a V/C/P (Vowel/Consonant/Pause) classification algorithm on a ...
2
votes
4answers
2k views

Optimal algorithm for finding all ngrams from a pre-defined set in a text

I'm thinking about the optimal algorithm for the following problem: Input data: a text, say it's an article about 5-50 pages. a set of ngrams (ngram strings, n>2), of arbitrary length, could be more ...
3
votes
0answers
267 views

Has someone seen this structure before?

I am working 1 with a certain structure, and I wonder if someone has seen it before. I am no mathematician, so all I can say is that I will do my best to describe this structure. It is actually very ...
4
votes
1answer
202 views

Semantic similarity in text

Is there a relatively simple way of telling if two pieces of text are semantically similar? Some assumptions that are valid: It is all english I have a list of all the important nouns Are there any ...
5
votes
2answers
154 views

When did commercial Speech Recognition first begin using grammar (sentence structure) for prediction?

It seems as though modern speech recognition (e.g., through Android, iOS phones) make use of grammar or sentence structure. (e.g., it might have a tough time distinguishing between "grammar" and "...
13
votes
2answers
729 views

Identifying events related to dates in a paragraph

Is there an algorithmic approach to identify that dates given in a paragraph correlate to particular events (phrases) in the paragraph? Example, consider the following paragraph: In June 1970, the ...

1 2 3
4