Questions tagged [natural-language-processing]
Natural language processing (NLP)
160
questions
0
votes
1answer
25 views
How can I recognize source from a text sample?
I'm supposed to write a (short) program that recognizes whether a given sample of text (10-200 words) comes from "Uncle Tom's Cabin," "Moby Dick," or "The Scarlet Letter". I cannot access the source ...
0
votes
1answer
21 views
Which algorithm for predicting the next word(s) based on previous words, given a sentence?
I want to input some words, and out comes the next word(s). Neural nets are really hot at the moment, and I'm afraid of throwing a neural net at something, when one is not really needed. Or... maybe ...
0
votes
0answers
17 views
What classifier can recognize differences in two text strings immediately?
I'm playing around with the TextBlob library for python. It has in it a NaiveBayesClassifier as well as a ...
0
votes
0answers
32 views
More information about the 10 types of information that can be useful for WSD (Word Sense Disambiguation)
Agirre and Martinez (Knowledge Sources for Word Sense Disambiguation) distinguish ten different types of information that can be useful for WSD (Word Sense Disambiguation):
Part of speech (POS) ...
0
votes
1answer
106 views
N-gram language model question
I have this question I found regarding n-gram modelling in the Speech and language processing text book:
...
19
votes
1answer
801 views
Compression of domain names
I am curious as to how one might very compactly compress the domain of an arbitrary IDN hostname (as defined by RFC5890) and suspect this could become an interesting challenge. A Unicode host or ...
3
votes
0answers
43 views
Explain the process of formation of word senses in WordNet
Regarding the word sense disambiguation problem, read the following fact written on Knowledge-based Word Sense Disambiguation using Topic Models:
Note that although WordNet is the most widely used ...
1
vote
0answers
40 views
What are open problems in computer science? [closed]
I should prepare some paper for a colloquium (kinda student-task) and it should cover the following points:
(1) at least one notable discovery in theoretical informatics (or computer science)
(2) at ...
0
votes
1answer
33 views
CYK result de-transformation
Suppose we have a rules derived from a treebank.
And in order to get a syntax tree of a given sentence we use the cyk algorithm.
In order to use the cyk we should convert the rules into chomsky normal ...
0
votes
0answers
7 views
Query about an equation in GAN-NMT paper
So i was studing the paper Adversarial Neural Machine Translation by Lijun Wu1, Yingce Xia2, Li Zhao3, Fei Tian3, Tao Qin3, Jianhuang Lai1,4 and Tie-Yan Liu.
The link to the paper is : https://arxiv....
0
votes
0answers
15 views
Understanding Multi-headed attention
Recently I came across the concepts like Attention and Transformer architecture. After studying the papers and many blogs, I understood the concepts, but recently one doubt is coming in my mind and ...
1
vote
0answers
24 views
CYK algorithm - how to handle unknown terminals given in a sentence to parse?
There is a given treebank which we derive the Probabilistic context free grammar.
I wonder how do one handles with a given sentence which includes terminals that don't exist in the derived rules?
Is ...
0
votes
0answers
104 views
How does Google's Superpod work?
I was looking for the best natural language quenstion answering system and chatbots when I found Google's Superpod described here. I googled for how Superpod works but I have found any important ...
1
vote
1answer
211 views
How does SpaCy make its dependency tree?
I discovered that SpaCy had the ability to make dependency trees. For instance given a question “To whom did the Virgin Mary allegedly appear in 1858 in Lourdes France?”
We can create its dependency ...
1
vote
0answers
25 views
How can I find the perplexity of a text by the perplexity of its sentences?
For a bigram language model, I can calculate the perplexity of sentences of a test document. However, I'm not sure what would be the perplexity of the whole document. Should I get the average of the ...
3
votes
1answer
139 views
Bottom up chart parser adding active arc step
I am following Bottom up chart parsing algorithm from Natural Language Understanding book by James Allen. It is
I couldn't understand the 3rd step. I thought that when active arc is added the dot ...
1
vote
0answers
38 views
Text Segmentation Problem give Word Frequencies in a Universe
Given a dictionary of words and their frequencies (how many times they appear in a universe and given a string(no spaces, punctuation, etc.). What is the best way to segment into individual words? I ...
0
votes
0answers
27 views
Find pattern in string of unkown language
Hello I am looking for an algorithm which solves following problem for me.
Given a text/string = 'BIRDBIRDBIRDBIRDDEVILDEVILDEVILDEVILEASYEASYEASY ...'
of an unkown language. For the sake of ...
2
votes
0answers
7 views
Are there any neural NLG systems which don't generate in left-to-right order?
For a while, all classification tasks in natural language processing were based on simple RNN's, which operate in a very word-by-word order. Adding gating mechanisms increased ability to "look back", ...
1
vote
0answers
23 views
Generalization of formal grammars - production rules with more general functions?
Usually formal grammars have production rules in the format N=tNt where simple concatenation function is used for the expansion of the nonterminal. https://www....
1
vote
1answer
72 views
Show that the Laplace smoothing for bigrams is a valid probability distribution
If we consider any smoothing technique like laplace or delta smoothing. Intuitively we can see that the we are stealing from sequences with non zero probablity and re distribute to sequences with zero ...
0
votes
0answers
56 views
How to prove that katz backing off smoothing technique is a valid probability distribution?
How to prove that katz backing off smoothing technique is a valid probability distribution? Take the example of bigram.
You can go through this link for katz back off model
https://en.wikipedia.org/...
2
votes
1answer
99 views
How to represent symbolic knowledge using real numbers - theory about neural networks and natural/analog computing?
One can define the semantics of one definite word using the references to real world entities, relationships with the other words and other concepts and represent all this knowledge about this one ...
3
votes
2answers
113 views
Measuring the information of a document?
I'd like to measure how much information a document $D$ contains.
Clearly, the New York Times published yesterday contains more information than my diary wrote on the same day. But, I do not know ...
4
votes
0answers
20 views
How to represent sentences with their dependency parses as input to an RNN?
I am working on a task embedding sentences into a lower-dimensional space according to style, both grammatical and lexical. As such, I want to have as input the linear ordering of tokens in each ...
3
votes
0answers
32 views
How to compute the loss and backprop of word2vec skip-gram using hierarchical softmax?
So we are calculating the loss
$$J(\theta) = -\frac{1}{T}\sum_{t=1}^T\sum_{-m \leq j \leq m} \log P(w_{t+j}|w_t;\theta)$$
and to do this we need to calculate
$$P(o|c) = \frac{\exp(u_o^Tv_c)}{\sum \...
1
vote
1answer
94 views
How to transform lambda function to multi-argument lambda function and how to rewrite or approximate terms?
I am trying to do the formal semantics (Montague grammar, abstract categorial grammar) of natural language and encode the sentence John is boss. The type system has ...
2
votes
1answer
22 views
Marginalise edge weights on graph
I have a directed acyclic graph with a score on each edge. The score of a path is defined to be the sum of the scores on the edges along this path. The probability of a path is the score of such a ...
5
votes
2answers
710 views
Estimate entropy, based upon observed frequency counts
Suppose I have $n$ independent observations $x_1,\dots,x_n$ from some unknown distribution over a known alphabet $\Sigma$, and I want to estimate the entropy of the distribution. I can count the ...
1
vote
0answers
205 views
How can node2vec help find similar “roles” within a graph (nodes whose connections have similar structure within the graph)?
I have a question on the node2vec algorithm described in this paper.
Node2vec is a deep learning algorithm that word2vec to graphs to learn embeddings. The authors claim that it can help find nodes ...
2
votes
1answer
820 views
Subsampling of Frequent Words in Word2Vec
I am reading through the following paper: https://papers.nips.cc/paper/5021-distributed-representations-of-words-and-phrases-and-their-compositionality.pdf
Under section 2.3 on page 4 the authors ...
1
vote
1answer
76 views
Question on word probability for hierarchical softmax used in natural language processing
I am reading the following paper: https://papers.nips.cc/paper/5021-distributed-representations-of-words-and-phrases-and-their-compositionality.pdf
On page 4 of the paper they describe the ...
0
votes
1answer
38 views
Naive Bayes' Classification and Using the Entire Vocabulary in the Denominator
I am working through the NLP notes for Naive Bayes' classification here: https://web.stanford.edu/~jurafsky/slp3/6.pdf
Below $c$ is the class of the observation and $w_i$ is the $i$th word of a text ...
2
votes
0answers
15 views
How does the Earley parser accept the input string as syntactically correct?
https://en.wikipedia.org/wiki/Earley_parser
An input string of length $n$ is syntactically correct if at least one $(S \rightarrow X_1 ... X_m • 0)$ is in $E_n$. Why is this the case ? The part ...
2
votes
0answers
50 views
Proving the probability of zero occurrences in training using Good-Turing maximum likelihood estimate
Background
Good-Turing (GT) smoothing is used in language models to estimate the counts of words in the test set that have not been seen in the training set.
In GT smoothing, $N_c$ is the count of ...
1
vote
1answer
30 views
How to figure out whether two texts refer to the same object or event
Let's assume there is something happen in the world - Football world cup final. And team-1 beat team-2 with the score 3:2. So there is whole bunch of articles on every website about it, each contains ...
0
votes
0answers
28 views
Any references/current work on anomaly detection in conversations?
Suppose I have a lot of data on conversations between humans and chatbots (human text, chatbot text, times, media used for chat, etc), and I want to be able to detect anomalies in these conversations. ...
2
votes
1answer
774 views
Semantic natural language processing - from texts to logical expressions? Universal knowledge base?
My question is - is there a semantic natural language processing that tries to understand the meaning of the texts and that tries to derive the consequences of the understood meaning? Is there a ...
1
vote
0answers
39 views
In NLP, tokens not seen in training sample, but you know or don't know what they are
In NLP, do you distinguish tokens that you don't observe in a training sample and still expect that they may occur in a test sample, between
those you know what they are, and
those you don't know ...
2
votes
0answers
52 views
Automatic learning/discovery of logics
Are there efforts to automatically discover new logics? Logics are simple structures - they have formal language, deduction rules, semantics and certain properties that are proved or discarded for ...
3
votes
0answers
120 views
Semantic parsing with Grammatical Framework - is this possible?
So far I have learned about categorial grammars, type logical grammars and formal semantics of natural language, the relevant tools are Cornell Semantic Parsing Framework https://github.com/clic-lab/...
4
votes
1answer
4k views
Semantic clustering
I have a very specific question about semantic clustering.
I have a list of words/phrases. I want to run an intelligent semantic clustering algorithm on this list. Please let me know what the ...
0
votes
1answer
167 views
Converting CFG to CNF [closed]
need some help with the following question:
I've watched a few youtube tutorials but I'm struggling to convert this specific CFG
5
votes
1answer
599 views
nlp: phonetic edit distance between a word and the closest of a set of words
Let's say someone is using Dragon Dictation, Google Speech, or some other free form dictation software (it will recognize anything they say to the best of its ability). I have some reasonably large ...
3
votes
0answers
88 views
How to translate lambda calculus into (first-order, modal) logic, is it possible at all?
It is possible (using formal semantics) to translate natural language sentences into lambda expressions. So, is it possible to translate those lambda expressions into some logic, e.g. into first-order ...
2
votes
0answers
51 views
Semantic/DRT methods for conversational agents / chatbots / dialogue systems - reference request?
The wiki pages about chabots mention that statistical methods, keyword search and precompiled answers are used for the chatbots. But I feel that there should exist different - semantical approach for ...
1
vote
1answer
54 views
Understanding the geometrical interpretation of word2vec
I'm trying to understand how the $word2vec$ method actually nudges word vectors of similar semantic/syntactic content closer together in the word vector space.
I've read here (Quora answer) that it's ...
1
vote
0answers
82 views
HMM Baum-Welch training and pruning
I am working on a HMM tagger that should be initialized with some small data and then supposedly improved with Baum-Welch algorithm on the data.
However, the number of states is huge, almost $459^2$, ...
3
votes
1answer
27 views
HMM tagger - Baum Welch training
I am trying to implement a trigram HMM tagger for a language that has over 1000 tags. In my training data I have 459 tags. Now if we consider that states of the HMM are all possible bigrams of tags, ...
1
vote
1answer
25 views
How to use a logistic regression classifier to estimate the confidence of a rule?
The following is an excerpt from the Estimating LAT Confidence section of this paper:
... Since some LAT detection rules are more reliable than others, we
would like to have a confidence value in ...
