Questions tagged [natural-language-processing]

Natural language processing (NLP)

68 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
19
votes
1answer
792 views

Compression of domain names

I am curious as to how one might very compactly compress the domain of an arbitrary IDN hostname (as defined by RFC5890) and suspect this could become an interesting challenge. A Unicode host or ...
4
votes
0answers
20 views

How to represent sentences with their dependency parses as input to an RNN?

I am working on a task embedding sentences into a lower-dimensional space according to style, both grammatical and lexical. As such, I want to have as input the linear ordering of tokens in each ...
4
votes
0answers
49 views

Infer probabilities, for concatenation of words

Fix an alphabet $\Sigma$, and a set of words, $W = \{w_1,\dots,w_n\} \subseteq \Sigma^*$. I have a randomized model that works like this: Alice generates a random sequence of words, using some ...
4
votes
0answers
2k views

Next-Word Prediction, Language Models, N-grams

I was looking into how a next-word prediction engine like swift key or XT9 can be implemented. Here's what I did. I read about n-grams here - en.wikipedia.org/wiki/N-gram and aicat.inf.ed.ac.uk/...
3
votes
0answers
31 views

How to compute the loss and backprop of word2vec skip-gram using hierarchical softmax?

So we are calculating the loss $$J(\theta) = -\frac{1}{T}\sum_{t=1}^T\sum_{-m \leq j \leq m} \log P(w_{t+j}|w_t;\theta)$$ and to do this we need to calculate $$P(o|c) = \frac{\exp(u_o^Tv_c)}{\sum \...
3
votes
0answers
117 views

Semantic parsing with Grammatical Framework - is this possible?

So far I have learned about categorial grammars, type logical grammars and formal semantics of natural language, the relevant tools are Cornell Semantic Parsing Framework https://github.com/clic-lab/...
3
votes
0answers
86 views

How to translate lambda calculus into (first-order, modal) logic, is it possible at all?

It is possible (using formal semantics) to translate natural language sentences into lambda expressions. So, is it possible to translate those lambda expressions into some logic, e.g. into first-order ...
3
votes
0answers
76 views

Document clustering for summarization

I am curious as to what steps one would reasonably need to take to perform an extraction-based text summarizer. I've taken a look at some papers I've found on Google such as this one, which explains ...
3
votes
0answers
266 views

Has someone seen this structure before?

I am working 1 with a certain structure, and I wonder if someone has seen it before. I am no mathematician, so all I can say is that I will do my best to describe this structure. It is actually very ...
2
votes
0answers
39 views

Explain the process of formation of word senses in WordNet

Regarding the word sense disambiguation problem, read the following fact written on Knowledge-based Word Sense Disambiguation using Topic Models: Note that although WordNet is the most widely used ...
2
votes
0answers
6 views

Are there any neural NLG systems which don't generate in left-to-right order?

For a while, all classification tasks in natural language processing were based on simple RNN's, which operate in a very word-by-word order. Adding gating mechanisms increased ability to "look back", ...
2
votes
0answers
15 views

How does the Earley parser accept the input string as syntactically correct?

https://en.wikipedia.org/wiki/Earley_parser An input string of length $n$ is syntactically correct if at least one $(S \rightarrow X_1 ... X_m • 0)$ is in $E_n$. Why is this the case ? The part ...
2
votes
0answers
48 views

Proving the probability of zero occurrences in training using Good-Turing maximum likelihood estimate

Background Good-Turing (GT) smoothing is used in language models to estimate the counts of words in the test set that have not been seen in the training set. In GT smoothing, $N_c$ is the count of ...
2
votes
0answers
51 views

Automatic learning/discovery of logics

Are there efforts to automatically discover new logics? Logics are simple structures - they have formal language, deduction rules, semantics and certain properties that are proved or discarded for ...
2
votes
0answers
49 views

Semantic/DRT methods for conversational agents / chatbots / dialogue systems - reference request?

The wiki pages about chabots mention that statistical methods, keyword search and precompiled answers are used for the chatbots. But I feel that there should exist different - semantical approach for ...
2
votes
0answers
22 views

Where to cut a category tree

Since I don't have CS background I will most probably ask this question the wrong way. I need to choose a node from a tree, where I include all beneath this node leafs in a validation. I have a data ...
2
votes
0answers
474 views

What are the inputs to an LSTM for Slot Filling Task

I am confused on the inputs of a Long-Short Term Memory (LSTM) for the slot filling task in Spoken Language Understanding. Before I worked on this, I implemented a language model with a Recurrent ...
2
votes
0answers
48 views

Phrase generation approaches

In generating reports, sometimes there is a need to produce quite involved phrases in one of natural languages given numerical or boolean parameters. To get a feel of it, it is enough to take a look ...
2
votes
0answers
63 views

How to determine agreement between two sentences?

A common Natural Language Processing (NLP) task is to determine semantic similarity between two sentences. Has the question of agreement/disagreement between two sentences been covered in NLP or other ...
2
votes
0answers
22 views

Standard training and test corpuses for POS tagging

I'm wondering if there is a standard corpus that people use to evaluate their part-of-speech taggers. I am implementing my own tagger, and would like a meaningful way to measure its performance, ...
2
votes
0answers
52 views

How to store linguistic data?

I'm both interested in practical and theoretical aspects of the problem. How do you store linguistic data. Suppose you have stemmed, parsed, tagged and, perhaps even did some preliminary analysis, ...
2
votes
0answers
60 views

What are recent, high-quality surveys on NLP topics?

In particular POS tagging, dependency and constituent parsing. This is not really my field of study but I would really like to be able to make informed claims on what precisions current top systems ...
2
votes
0answers
51 views

Two classes of documents. Find weighted relations between them

I have an NLP problem and a potential solution, but I’m a bit green here, so I’m looking for some validation or alternative suggestions. Background I have two types of documents: one is a set of ...
2
votes
0answers
179 views

The grammar of the GeoQuery language

GeoQuery is a dataset used for benchmarking semantic parsers. It contains 880 queries about USA geography. The queries are in Prolog format, for example: answer(A,longest(A,(river(A),traverse(A,B),...
1
vote
0answers
23 views

CYK algorithm - how to handle unknown terminals given in a sentence to parse?

There is a given treebank which we derive the Probabilistic context free grammar. I wonder how do one handles with a given sentence which includes terminals that don't exist in the derived rules? Is ...
1
vote
0answers
24 views

How can I find the perplexity of a text by the perplexity of its sentences?

For a bigram language model, I can calculate the perplexity of sentences of a test document. However, I'm not sure what would be the perplexity of the whole document. Should I get the average of the ...
1
vote
0answers
38 views

Text Segmentation Problem give Word Frequencies in a Universe

Given a dictionary of words and their frequencies (how many times they appear in a universe and given a string(no spaces, punctuation, etc.). What is the best way to segment into individual words? I ...
1
vote
0answers
22 views

Generalization of formal grammars - production rules with more general functions?

Usually formal grammars have production rules in the format N=tNt where simple concatenation function is used for the expansion of the nonterminal. https://www....
1
vote
0answers
188 views

How can node2vec help find similar “roles” within a graph (nodes whose connections have similar structure within the graph)?

I have a question on the node2vec algorithm described in this paper. Node2vec is a deep learning algorithm that word2vec to graphs to learn embeddings. The authors claim that it can help find nodes ...
1
vote
0answers
77 views

HMM Baum-Welch training and pruning

I am working on a HMM tagger that should be initialized with some small data and then supposedly improved with Baum-Welch algorithm on the data. However, the number of states is huge, almost $459^2$, ...
1
vote
0answers
388 views

How much training data for speech recognition?

How much training data is needed to build a speech-to-text engine based on machine learning? (To within an order of magnitude or so.) Big companies like Google, Facebook have a massive amount of data....
1
vote
0answers
47 views

Training algorithm for Manual error correction in text

I want to work on a model where I make some manual corrections in my clinical notes data and want the neural network to learn those corrections.The algorithm will be supplied with both corrected and ...
1
vote
0answers
36 views

Outline detection from patterns in a list of textual articles

Are there NLP algorithms dealing with detecting the repeating patterns in a a list of texts from which a topic keywords and other associative keywords can be derived? I will show it as an example: ...
1
vote
0answers
36 views

How to store topic-to-topic relationships efficiently and accurately with two huge tables?

I am scanning Wikipedia's DB and retrieving topics (page names, mostly) and their relationships. I want to use it later to create a visual map. A topic table stores the id and the label of each topic....
1
vote
0answers
25 views

Is there a publicly available informer tagger or dataset?

I am working on a question answering system. I've learned that informer spans are valuable features for question classification. However from what I've read I wasn't able to find any publicly ...
1
vote
0answers
13 views

Evaluation of annotation

I'm doing a research on NLG systems. I need to annotate my corpus (~6 million words) automatically. My algorithm works well and I want to calculate Cohen's Kappa. What I cannot understand is the ...
1
vote
0answers
67 views

Parsing based on logical connectives and quantifiers?

What techniques exist for parsing sentences based on logical connectives and quantifiers? That is, for parsing sentences that are structured "around" logical connectives and quantifiers. E.g. [A] ...
1
vote
0answers
41 views

Algorithm to change article reading level

I was wondering, are there any famous algorithms in NLP or NLG that can take some text (say a news article online) and transform it to a lower reading level?
1
vote
0answers
39 views

In NLP, tokens not seen in training sample, but you know or don't know what they are

In NLP, do you distinguish tokens that you don't observe in a training sample and still expect that they may occur in a test sample, between those you know what they are, and those you don't know ...
1
vote
0answers
2k views

Next Word Prediction using n-gram & Tries

I am studying the following paper for understanding next-word prediction using n-gram & trie: - http://nlp.cs.berkeley.edu/pubs/Pauls-Klein_2011_LM_paper.pdf Before this, I did some brief study on ...
1
vote
0answers
106 views

Help understanding an audio processing algorithm

I'm reading “Speech segmentation without speech recognition” by Dong Wang, Lie Lu and Hong-Jiang Zhang. The algorithm I'm looking at is a V/C/P (Vowel/Consonant/Pause) classification algorithm on a ...
1
vote
1answer
108 views

NLP: Automatically categorize words semantically into non-predefined classes

I am looking for a method to categorize a given word into semantically into classes. A simple example would be: chair -> furniture The most reasonable way to ...
0
votes
0answers
19 views

More information about the 10 types of information that can be useful for WSD (Word Sense Disambiguation)

Agirre and Martinez (Knowledge Sources for Word Sense Disambiguation) distinguish ten different types of information that can be useful for WSD (Word Sense Disambiguation): Part of speech (POS) ...
0
votes
1answer
11 views

What does Bootstrapping approach means in terms of information extraction/ontology population?

What does Bootstrapping approach means in terms of information extraction/ontology population?
0
votes
0answers
7 views

Query about an equation in GAN-NMT paper

So i was studing the paper Adversarial Neural Machine Translation by Lijun Wu1, Yingce Xia2, Li Zhao3, Fei Tian3, Tao Qin3, Jianhuang Lai1,4 and Tie-Yan Liu. The link to the paper is : https://arxiv....
0
votes
0answers
13 views

Understanding Multi-headed attention

Recently I came across the concepts like Attention and Transformer architecture. After studying the papers and many blogs, I understood the concepts, but recently one doubt is coming in my mind and ...
0
votes
1answer
33 views

CYK result de-transformation

Suppose we have a rules derived from a treebank. And in order to get a syntax tree of a given sentence we use the cyk algorithm. In order to use the cyk we should convert the rules into chomsky normal ...
0
votes
0answers
103 views

How does Google's Superpod work?

I was looking for the best natural language quenstion answering system and chatbots when I found Google's Superpod described here. I googled for how Superpod works but I have found any important ...
0
votes
1answer
69 views

N-gram language model question

I have this question I found regarding n-gram modelling in the Speech and language processing text book: ...
0
votes
0answers
27 views

Find pattern in string of unkown language

Hello I am looking for an algorithm which solves following problem for me. Given a text/string = 'BIRDBIRDBIRDBIRDDEVILDEVILDEVILDEVILEASYEASYEASY ...' of an unkown language. For the sake of ...