Podcast #128: We chat with Kent C Dodds about why he loves React and discuss what life was like in the dark days before Git. Listen now.

Questions tagged [natural-language-processing]

Natural language processing (NLP)

Filter by
Sorted by
Tagged with
1
vote
1answer
306 views

NLP Classification: rule-based approach providing high precision but low recall

In NLP, rule-based/linguistic approaches are usually said to provide relatively high precision but low recall compared to for example machine learning approaches. I think this is often true for almost ...
0
votes
0answers
37 views

How can feature structures be applied to text summarization?

I'm looking for innovative ways to create text summaries and I was wondering whether feature structures can be used for such a task. Is it possible to create subsume and unification methods that ...
1
vote
1answer
39 views

How to detect plurals in English sentences?

I'm wondering how to detect English plurals in natural language sentences. Such as: fish computers houses ... Using Earley parsing I'm able to get to the s or no-s, but I have difficulties in ...
0
votes
1answer
492 views

Where can I find a reasonable NLP CFG or CSG for English?

I'm looking to do natural language parsing and looking for how the CFG (or CSG) should be defined for English. Surely one'd expect to find one from the internet already, but do you know where? It ...
3
votes
0answers
452 views

Is English Recursively enumerable? [closed]

The title says it all. I've tried digging up debate on this issue to see a proof one way or the other but it doesn't look like anyone is able to say whether or not it is. Clearly there are recursive ...
0
votes
1answer
170 views

In NLP, does the lexer have to tag the tokens before the parser?

In NLP, does the lexer have to tag the tokens before the parser? I.e. does the lexer have to classify the tokens to morphological categories before the parser? I'm thinking yes, but is this also the ...
0
votes
1answer
61 views

How to extract best illustrative sentences from documents for a given word? [closed]

I want to extract "best" sentences from the given (html) documents. Let me illustrate this with an example. I have a word "determine". Now I have hundreds of html files having paragraphs and sentences ...
0
votes
0answers
26 views

Common models for inferring semantics (truth/factuality of statements)?

What are common models for creating rules for inferring meanings (e.g. truth values) of natural language statements such as if one wanted to infer the truth value of the input statement John Lennon ...
-1
votes
1answer
53 views

Context-free grammar for “member of”/set membership?

I'm wondering whether context-free grammars (or what else) can be used to implement "member of" structures, which are structures for describing that something is a member of something. E.g. I want to ...
1
vote
0answers
73 views

Parsing based on logical connectives and quantifiers?

What techniques exist for parsing sentences based on logical connectives and quantifiers? That is, for parsing sentences that are structured "around" logical connectives and quantifiers. E.g. [A] ...
2
votes
2answers
579 views

Generative Machine Learning algorithms on tree structure

I'm looking into PCFG sentence grammar dependency structure parsing using StanfordNLP PCFG parser. It generates tree structures represented as a string like this: ...
1
vote
0answers
41 views

Algorithm to change article reading level

I was wondering, are there any famous algorithms in NLP or NLG that can take some text (say a news article online) and transform it to a lower reading level?
1
vote
1answer
108 views

NLP: Automatically categorize words semantically into non-predefined classes

I am looking for a method to categorize a given word into semantically into classes. A simple example would be: chair -> furniture The most reasonable way to ...
5
votes
3answers
4k views

Word classification algorithms

I am looking for algorithms to classify words in a paragraph of text. I am particularly interested in a classification to determine if a certain word is noun, verb, etc., but also looking for any kind ...
2
votes
1answer
126 views

Scientific soundness of computer science papers [closed]

First of all, this question relates to an issue that affects several areas of science, but since StackExchange doesn't have a meta science section I'll make it specific to computer science, which is ...
1
vote
2answers
4k views

Computing probability of sentence using N-grams

I have implemented N-grams by constructing a tree (or a trie, technically) that stores frequencies of each N-gram. Each path in the tree represents an N-gram and its frequency: the path consists of N ...
2
votes
0answers
22 views

Standard training and test corpuses for POS tagging

I'm wondering if there is a standard corpus that people use to evaluate their part-of-speech taggers. I am implementing my own tagger, and would like a meaningful way to measure its performance, ...
4
votes
0answers
50 views

Infer probabilities, for concatenation of words

Fix an alphabet $\Sigma$, and a set of words, $W = \{w_1,\dots,w_n\} \subseteq \Sigma^*$. I have a randomized model that works like this: Alice generates a random sequence of words, using some ...
6
votes
2answers
683 views

Are syntax and semantic just 2 structures such that one is a model of the other?

The syntax of a language is a structure. The semantic of a language is a structure. The semantic of a language is a model of its syntax. And that's all ? The duality syntax/semantic is just model ...
3
votes
1answer
175 views

Choice of machine learning algorithms frequency of parts of speech

I'm new to machine learning. I have text, and I tag the text according to their parts of speech tag ie walk is tagged as verb, etc. I tag entire sentences, and then convert them into a vector based on ...
2
votes
0answers
52 views

How to store linguistic data?

I'm both interested in practical and theoretical aspects of the problem. How do you store linguistic data. Suppose you have stemmed, parsed, tagged and, perhaps even did some preliminary analysis, ...
3
votes
1answer
121 views

Natural language query processing

I am trying to implement a natural language query preprocessing module which would, given a query formulated in natural language, extract the keywords from that query and submit it to an IR (...
2
votes
1answer
76 views

Most frequently tools or programming language for implementation text processing and nlp algorithms in academic papers and journals [closed]

I want to prototype and try some idea (some algorithm) in the field of text processing and nlp and if the results was good I want to publish some paper or journal article about that. I am familiar ...
2
votes
1answer
75 views

Looking for a paper comparing NLP methods with Data Mining techniques

I've recently attended a conference, where one of the participants mentioned a recent paper published by a Google employee, which showed that using data mining techniques in application to NLP might ...
1
vote
1answer
68 views

NLP - Deep belief neural networks or logic?

The NLP progresses seems to me that has split in two big group of thoughts: Deep learning have a neural network with multiple layers that has been trained and learned to represent knowledge and ...
1
vote
1answer
24 views

Where can I find a study on the amount of different word frequencies in a corpus? [closed]

I need to know how many different values the frequency of a certain word in a corpus can there be for a natural language processing problem. Is there any study or site that has such estimation?
2
votes
1answer
173 views

Natural Language Parser that can handle syntactic and lexical errors

I have some background in natural language processing and I know that all parsers (top down or bottom up, or mix), at least when I studied just about a few years ago, cannot handle any error. A small ...
2
votes
1answer
35 views

How is wordnet curated?

I use Princeton's WordNet in NLP applications. I always read that it is "human-curated." But how is it curated? Who decides that words belong in a synset? How does the process work? Wikipedia does not ...
2
votes
1answer
278 views

Automatic speech correction

When we are trying to learn a new language without any help from native speakers, the first thing we need is something/someone to correct our grammatical mistakes/improper word selection. So is there ...
2
votes
0answers
60 views

What are recent, high-quality surveys on NLP topics?

In particular POS tagging, dependency and constituent parsing. This is not really my field of study but I would really like to be able to make informed claims on what precisions current top systems ...
2
votes
3answers
643 views

Does programming language detection need more input than natural language detection?

I wonder which one of the two needs a larger input to achieve a decent accuracy: programming language detection or natural language detection? More details: Definition of Language detection: In ...
2
votes
2answers
2k views

What are the definitions of syntax and semantics?

For a formal language $L \subseteq \Sigma^*$ over an alphabet $\Sigma$. From https://proofwiki.org/wiki/Definition:Syntax The syntax of a formal language is its structure, and is specified by a ...
2
votes
0answers
51 views

Two classes of documents. Find weighted relations between them

I have an NLP problem and a potential solution, but I’m a bit green here, so I’m looking for some validation or alternative suggestions. Background I have two types of documents: one is a set of ...
3
votes
2answers
71 views

How do learn the most important nodes in a tree?

I have a list of 20000 words and how often they appeared in a set of 500 newspaper articles. I am trying to build a stemmer which chops off suffuxes from each words, so ...
1
vote
0answers
124 views

Translation of natural language to logic [closed]

Given a statement in natural language, what can be said about how many possible translations there are in first-order-logic? What happen if we take a more complex logic like second-order logic and ...
3
votes
0answers
74 views

What are some of the methods that NLP practitioners use to automatically learn linguistic features from text? [closed]

I am learning about NLP, with an eye to starting some practical NLP projects. I see that many of the algorithms for relation extraction and named entity recognition require you to identify linguistic ...
2
votes
0answers
20 views

What are ways that you might analyze relations extracted from text [closed]

In the coursera NLP course, Dan Jurofsky says you might want to extract relations from text to serve as input to other algorithms. What are examples of ways that researchers have used or might use ...
3
votes
0answers
76 views

Document clustering for summarization

I am curious as to what steps one would reasonably need to take to perform an extraction-based text summarizer. I've taken a look at some papers I've found on Google such as this one, which explains ...
0
votes
0answers
30 views

Is there a paper/implementation/tool on automatically finding/suggesting similar questions for a given question?

I would like to know whether there is a method to find similar questions for a given question, just like this stackexchange.com does. Is there any paper or tools? I tried keywords such as "...
0
votes
0answers
427 views

NLP: in practice, what should I choose as Start and End symbol?

I'm writing a small program that implements a trigram HMM model and apply it by using Viterbi Algorithm. It's very standardized. However, I have never written anything like this before, and I get ...
2
votes
4answers
1k views

What aspects of linguistics are necessary or good for natural language processing? [closed]

What aspects of linguistics are necessary or good to know for natural language processing? What references do you recommend for studying those aspects? Thanks!
2
votes
1answer
106 views

Solving the part-of-speech tagging problem with HMM

There is a famous part-of-speech tagging problem in Natural Language Processing. The popular solution is to use Hidden Markov Models. So that, given the sentence $x_1 \dots x_n$ we want to find the ...
2
votes
1answer
35 views

Trying to tag parts of a word and keep track of any changes that happen to those parts

I'm a researcher working with a language that has gone through phonological changes through time. I would like to tag parts of a word (i.e. prefix, stem, suffix) and then apply those phonological ...
2
votes
1answer
80 views

Using interval graphs to find authorship disputes

The first chapter of the book "Graphs and their uses" by Oystein Ore says that interval graphs can be used to resolve authorship disputes, but I couldn't find any details. How does this work? What ...
5
votes
1answer
530 views

Determining interests from Twitter text using Latent Dirichlet Allocation

I would like to determine interests or hobbies that people have given their Twitter timeline data. Their timeline is their historical collection of tweets. The result I am trying to achieve is to ...
4
votes
0answers
2k views

Next-Word Prediction, Language Models, N-grams

I was looking into how a next-word prediction engine like swift key or XT9 can be implemented. Here's what I did. I read about n-grams here - en.wikipedia.org/wiki/N-gram and aicat.inf.ed.ac.uk/...
1
vote
0answers
2k views

Next Word Prediction using n-gram & Tries

I am studying the following paper for understanding next-word prediction using n-gram & trie: - http://nlp.cs.berkeley.edu/pubs/Pauls-Klein_2011_LM_paper.pdf Before this, I did some brief study on ...
2
votes
2answers
142 views

Hidden Markov Model in Tagging Problem

I try to understand the details regarding using Hidden Markov Model in Tagging Problem. The best concise description that I found is the Course notes by Michal Collins. The goal is to find a ...
0
votes
1answer
369 views

short text categorization with spelling correction

I am building a classifier for short texts in a chat system. My features are words and pairs of words. Naturally, the sentences contain spelling mistakes. If a particular wrong spelling of a certain ...
0
votes
0answers
160 views

Advantages of knowing foreign languages for natural language processing

I wonder about cases in which knowing several languages can lead a researcher to interesting results in natural language processing. For example, knowledge of foreign languages can without doubt ...