Questions tagged [natural-language-processing]

Natural language processing (NLP)

69 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
22
votes
1answer
913 views

Compression of domain names

I am curious as to how one might very compactly compress the domain of an arbitrary IDN hostname (as defined by RFC5890) and suspect this could become an interesting challenge. A Unicode host or ...
4
votes
0answers
34 views

How to identify measure words in Chinese text?

Measure words (aka classifiers) are used in Chinese to "measure" things, e.g. 三杯牛奶 Three glasses of milk 那个人 That person 一只乌鸦 One crow 一公斤豆腐 A kilo of tofu We don't have an equivalent in ...
4
votes
0answers
22 views

How to represent sentences with their dependency parses as input to an RNN?

I am working on a task embedding sentences into a lower-dimensional space according to style, both grammatical and lexical. As such, I want to have as input the linear ordering of tokens in each ...
4
votes
0answers
53 views

Infer probabilities, for concatenation of words

Fix an alphabet $\Sigma$, and a set of words, $W = \{w_1,\dots,w_n\} \subseteq \Sigma^*$. I have a randomized model that works like this: Alice generates a random sequence of words, using some ...
4
votes
0answers
2k views

Next-Word Prediction, Language Models, N-grams

I was looking into how a next-word prediction engine like swift key or XT9 can be implemented. Here's what I did. I read about n-grams here - en.wikipedia.org/wiki/N-gram and aicat.inf.ed.ac.uk/...
3
votes
0answers
48 views

Explain the process of formation of word senses in WordNet

Regarding the word sense disambiguation problem, read the following fact written on Knowledge-based Word Sense Disambiguation using Topic Models: Note that although WordNet is the most widely used ...
3
votes
0answers
40 views

How to compute the loss and backprop of word2vec skip-gram using hierarchical softmax?

So we are calculating the loss $$J(\theta) = -\frac{1}{T}\sum_{t=1}^T\sum_{-m \leq j \leq m} \log P(w_{t+j}|w_t;\theta)$$ and to do this we need to calculate $$P(o|c) = \frac{\exp(u_o^Tv_c)}{\sum \...
3
votes
0answers
66 views

Proving the probability of zero occurrences in training using Good-Turing maximum likelihood estimate

Background Good-Turing (GT) smoothing is used in language models to estimate the counts of words in the test set that have not been seen in the training set. In GT smoothing, $N_c$ is the count of ...
3
votes
0answers
142 views

Semantic parsing with Grammatical Framework - is this possible?

So far I have learned about categorial grammars, type logical grammars and formal semantics of natural language, the relevant tools are Cornell Semantic Parsing Framework https://github.com/clic-lab/...
3
votes
0answers
97 views

How to translate lambda calculus into (first-order, modal) logic, is it possible at all?

It is possible (using formal semantics) to translate natural language sentences into lambda expressions. So, is it possible to translate those lambda expressions into some logic, e.g. into first-order ...
3
votes
0answers
592 views

What are the inputs to an LSTM for Slot Filling Task

I am confused on the inputs of a Long-Short Term Memory (LSTM) for the slot filling task in Spoken Language Understanding. Before I worked on this, I implemented a language model with a Recurrent ...
3
votes
0answers
79 views

Document clustering for summarization

I am curious as to what steps one would reasonably need to take to perform an extraction-based text summarizer. I've taken a look at some papers I've found on Google such as this one, which explains ...
3
votes
0answers
268 views

Has someone seen this structure before?

I am working 1 with a certain structure, and I wonder if someone has seen it before. I am no mathematician, so all I can say is that I will do my best to describe this structure. It is actually very ...
2
votes
0answers
31 views

Question about word embeddings in a specific language model - GPT-2

How were the GPT-2 token embeddings constructed? The authors mention that they used Byte Pair Encoding to construct their vocabulary. But BPE is a compression algorithm that returns a list of ...
2
votes
0answers
13 views

Are there any neural NLG systems which don't generate in left-to-right order?

For a while, all classification tasks in natural language processing were based on simple RNN's, which operate in a very word-by-word order. Adding gating mechanisms increased ability to "look back", ...
2
votes
0answers
16 views

How does the Earley parser accept the input string as syntactically correct?

https://en.wikipedia.org/wiki/Earley_parser An input string of length $n$ is syntactically correct if at least one $(S \rightarrow X_1 ... X_m • 0)$ is in $E_n$. Why is this the case ? The part ...
2
votes
0answers
54 views

Automatic learning/discovery of logics

Are there efforts to automatically discover new logics? Logics are simple structures - they have formal language, deduction rules, semantics and certain properties that are proved or discarded for ...
2
votes
0answers
54 views

Semantic/DRT methods for conversational agents / chatbots / dialogue systems - reference request?

The wiki pages about chabots mention that statistical methods, keyword search and precompiled answers are used for the chatbots. But I feel that there should exist different - semantical approach for ...
2
votes
0answers
22 views

Where to cut a category tree

Since I don't have CS background I will most probably ask this question the wrong way. I need to choose a node from a tree, where I include all beneath this node leafs in a validation. I have a data ...
2
votes
0answers
71 views

How to determine agreement between two sentences?

A common Natural Language Processing (NLP) task is to determine semantic similarity between two sentences. Has the question of agreement/disagreement between two sentences been covered in NLP or other ...
2
votes
0answers
22 views

Standard training and test corpuses for POS tagging

I'm wondering if there is a standard corpus that people use to evaluate their part-of-speech taggers. I am implementing my own tagger, and would like a meaningful way to measure its performance, ...
2
votes
0answers
54 views

How to store linguistic data?

I'm both interested in practical and theoretical aspects of the problem. How do you store linguistic data. Suppose you have stemmed, parsed, tagged and, perhaps even did some preliminary analysis, ...
2
votes
0answers
60 views

What are recent, high-quality surveys on NLP topics?

In particular POS tagging, dependency and constituent parsing. This is not really my field of study but I would really like to be able to make informed claims on what precisions current top systems ...
2
votes
0answers
52 views

Two classes of documents. Find weighted relations between them

I have an NLP problem and a potential solution, but I’m a bit green here, so I’m looking for some validation or alternative suggestions. Background I have two types of documents: one is a set of ...
2
votes
0answers
186 views

The grammar of the GeoQuery language

GeoQuery is a dataset used for benchmarking semantic parsers. It contains 880 queries about USA geography. The queries are in Prolog format, for example: answer(A,longest(A,(river(A),traverse(A,B),...
1
vote
0answers
33 views

How can Kneser-Ney Smoothing be integrated into a neural language model?

I found a paper titled Multimodal representation: Kneser-Ney Smoothing/Skip-Gram based neural language model. I am curious about how the Kneser-Ney Smoothing technique can be integrated into a feed-...
1
vote
1answer
53 views

What kind of bigram probability smoothing is this?

I hope it isn't off topic but I need to understand this example. Given the corpus 12 1 13 12 15 234 2526 and smoothing factor of ...
1
vote
0answers
35 views

CYK algorithm - how to handle unknown terminals given in a sentence to parse?

There is a given treebank which we derive the Probabilistic context free grammar. I wonder how do one handles with a given sentence which includes terminals that don't exist in the derived rules? Is ...
1
vote
0answers
31 views

How can I find the perplexity of a text by the perplexity of its sentences?

For a bigram language model, I can calculate the perplexity of sentences of a test document. However, I'm not sure what would be the perplexity of the whole document. Should I get the average of the ...
1
vote
0answers
40 views

Text Segmentation Problem give Word Frequencies in a Universe

Given a dictionary of words and their frequencies (how many times they appear in a universe and given a string(no spaces, punctuation, etc.). What is the best way to segment into individual words? I ...
1
vote
0answers
32 views

Generalization of formal grammars - production rules with more general functions?

Usually formal grammars have production rules in the format N=tNt where simple concatenation function is used for the expansion of the nonterminal. https://www....
1
vote
0answers
335 views

How can node2vec help find similar “roles” within a graph (nodes whose connections have similar structure within the graph)?

I have a question on the node2vec algorithm described in this paper. Node2vec is a deep learning algorithm that word2vec to graphs to learn embeddings. The authors claim that it can help find nodes ...
1
vote
0answers
95 views

HMM Baum-Welch training and pruning

I am working on a HMM tagger that should be initialized with some small data and then supposedly improved with Baum-Welch algorithm on the data. However, the number of states is huge, almost $459^2$, ...
1
vote
0answers
445 views

How much training data for speech recognition?

How much training data is needed to build a speech-to-text engine based on machine learning? (To within an order of magnitude or so.) Big companies like Google, Facebook have a massive amount of data....
1
vote
0answers
54 views

Training algorithm for Manual error correction in text

I want to work on a model where I make some manual corrections in my clinical notes data and want the neural network to learn those corrections.The algorithm will be supplied with both corrected and ...
1
vote
0answers
37 views

Outline detection from patterns in a list of textual articles

Are there NLP algorithms dealing with detecting the repeating patterns in a a list of texts from which a topic keywords and other associative keywords can be derived? I will show it as an example: ...
1
vote
0answers
36 views

How to store topic-to-topic relationships efficiently and accurately with two huge tables?

I am scanning Wikipedia's DB and retrieving topics (page names, mostly) and their relationships. I want to use it later to create a visual map. A topic table stores the id and the label of each topic....
1
vote
0answers
27 views

Is there a publicly available informer tagger or dataset?

I am working on a question answering system. I've learned that informer spans are valuable features for question classification. However from what I've read I wasn't able to find any publicly ...
1
vote
0answers
13 views

Evaluation of annotation

I'm doing a research on NLG systems. I need to annotate my corpus (~6 million words) automatically. My algorithm works well and I want to calculate Cohen's Kappa. What I cannot understand is the ...
1
vote
0answers
74 views

Parsing based on logical connectives and quantifiers?

What techniques exist for parsing sentences based on logical connectives and quantifiers? That is, for parsing sentences that are structured "around" logical connectives and quantifiers. E.g. [A] ...
1
vote
0answers
42 views

Algorithm to change article reading level

I was wondering, are there any famous algorithms in NLP or NLG that can take some text (say a news article online) and transform it to a lower reading level?
1
vote
0answers
41 views

In NLP, tokens not seen in training sample, but you know or don't know what they are

In NLP, do you distinguish tokens that you don't observe in a training sample and still expect that they may occur in a test sample, between those you know what they are, and those you don't know ...
1
vote
0answers
2k views

Next Word Prediction using n-gram & Tries

I am studying the following paper for understanding next-word prediction using n-gram & trie: - http://nlp.cs.berkeley.edu/pubs/Pauls-Klein_2011_LM_paper.pdf Before this, I did some brief study on ...
1
vote
0answers
109 views

Help understanding an audio processing algorithm

I'm reading “Speech segmentation without speech recognition” by Dong Wang, Lie Lu and Hong-Jiang Zhang. The algorithm I'm looking at is a V/C/P (Vowel/Consonant/Pause) classification algorithm on a ...
1
vote
1answer
110 views

NLP: Automatically categorize words semantically into non-predefined classes

I am looking for a method to categorize a given word into semantically into classes. A simple example would be: chair -> furniture The most reasonable way to ...
0
votes
1answer
17 views

sentiment analysis by sentence similarity

How can sentence similarity be used for sentiment analysis? I know we have sentence Bert which can use cosine sim to measure the distance between to vectors (the sentence embeddings), but has anyone ...
0
votes
1answer
23 views

What Happens if I swap the forget gate and update gate in LSTM model?

Consider the following eqautions used in LSTM ( taken from Andrew ng's course on Sequential model) In an LSTM model, LSTM Cell has three inputs at any time step t Input($X_t , a^{(t-1)}, C^{(t-1)})$, ...
0
votes
0answers
129 views

More Information about these Question Answering Systems

My question is about the multi-hop question answering systems described in the page https://hotpotqa.github.io/ and reproduced in the following picture: I am interested in the models BFR-Graph, GSAN-...
0
votes
0answers
31 views

Interpolation for maximum likelihood estimate of unigram and bigram

We are given the following corpus, modified from the one in the chapter: ...
0
votes
0answers
29 views

n-grams textbook question

I have this question I found regarding n-gram modeling in the Speech and Language Processing textbook by Daniel Jurafsky: Suppose we didn’t use the end-symbol ...