Questions tagged [natural-language-processing]
Natural language processing (NLP)
177
questions
0
votes
1answer
34 views
What Happens if I swap the forget gate and update gate in LSTM model?
Consider the following eqautions used in LSTM ( taken from Andrew ng's course on Sequential model)
In an LSTM model, LSTM Cell has three inputs at any time step t
Input($X_t , a^{(t-1)}, C^{(t-1)})$, ...
0
votes
0answers
23 views
Is this problem decideable programatically?
Consider the following 3 strings:
1tb hdd 256gb ssd
hdd 256gb
1tb hdd
If you know, that all ...
0
votes
2answers
18 views
Compute the edit distance between two words in which substitution is not allowed
How do I compute the edit distance between two words in which substitution is not allowed?
The allowed operations include insertion (with cost 1) and deletion (with cost 1), but not substitution.
How ...
1
vote
0answers
6 views
Calculate coherence for non-gensim topic model [closed]
I've built a topic model, with:
Input: list of tokenized lists
Output: a m x t matrix (with each cell indicating the probability of word i appearing in topic k).
Output: a k x n matrix (with each ...
0
votes
0answers
12 views
Question about RNN detecting toxic spans
How can an RNN be used for detecting toxic spans (spans of words containing toxic language) in a social media comment? Specifically, what should be the input to the RNN at each time step t? How many ...
0
votes
1answer
67 views
CYK result de-transformation
Suppose we have a rules derived from a treebank.
And in order to get a syntax tree of a given sentence we use the cyk algorithm.
In order to use the cyk we should convert the rules into chomsky normal ...
2
votes
0answers
21 views
Generating an “approximate” grammar
This is my first time posting here, so I hope I'm on topic. I have a table of natural-language data of the form
...
0
votes
1answer
20 views
sentiment analysis by sentence similarity
How can sentence similarity be used for sentiment analysis? I know we have sentence Bert which can use cosine sim to measure the distance between to vectors (the sentence embeddings), but has anyone ...
1
vote
1answer
56 views
What kind of bigram probability smoothing is this?
I hope it isn't off topic but I need to understand this example. Given the corpus 12 1 13 12 15 234 2526 and smoothing factor of ...
23
votes
1answer
999 views
Compression of domain names
I am curious as to how one might very compactly compress the domain of an arbitrary IDN hostname (as defined by RFC5890) and suspect this could become an interesting challenge. A Unicode host or ...
0
votes
0answers
9 views
Is there an accurate way to convert frequencies of the Stupid Backoff language model to probabilities?
We introduce a similar but simpler scheme, named Stupid Backoff, that does not generate normalized probabilities. The main difference is that we don't apply any discounting and instead directly use ...
1
vote
0answers
13 views
Is relation extraction considered a subtask of information extraction?
I'm currently trying to investigate the relationship between relation extraction (RE) and event extraction (EE). Doing more reading on the two tasks has caused me to question my initial belief that RE ...
0
votes
3answers
712 views
N-gram language model question
I have this question I found regarding n-gram modelling in the textbook Speech and Language Processing:
Suppose we didn't use the end symbol </s>. Train an unsmoothed bigram grammar on the ...
0
votes
0answers
10 views
should you take question mark as a separate word in bigram language modelling when finding probability?
the corpus
what is the shape ?
what is the colour ?
what colour is it ?
it is what ?
is it red ?
what is it ?
what shape is it ?
it is red
it is green
the colour is red
the shape ...
4
votes
2answers
2k views
Examples for CFG that cannot be expressed by regular language
There are nice examples for context free grammars which cannot be expressed with regular language, for example the palindrome and a similar contrived example here, but they are very intuitively ...
0
votes
1answer
46 views
Which algorithm for predicting the next word(s) based on previous words, given a sentence?
I want to input some words, and out comes the next word(s). Neural nets are really hot at the moment, and I'm afraid of throwing a neural net at something, when one is not really needed. Or... maybe ...
-1
votes
1answer
39 views
Natural Languages
Can one imply that natural languages can be described by regular grammar? Is that what happens through NPL? Trying to understand the subject of how spoken language can be converted to data and how.
2
votes
1answer
37 views
Bible Verse division
I'm working on a project compiling various versions of the Bible into a dataset. For the most part versions separate verses discreetly. In some versions, however, verses are combined. Instead of verse ...
1
vote
0answers
34 views
How can Kneser-Ney Smoothing be integrated into a neural language model?
I found a paper titled Multimodal representation: Kneser-Ney Smoothing/Skip-Gram based neural language model. I am curious about how the Kneser-Ney Smoothing technique can be integrated into a feed-...
0
votes
0answers
170 views
More Information about these Question Answering Systems
My question is about the multi-hop question answering systems described in the page https://hotpotqa.github.io/ and reproduced in the following picture:
I am interested in the models BFR-Graph, GSAN-...
0
votes
0answers
91 views
n-grams textbook question
I have this question I found regarding n-gram modeling in the Speech and Language Processing textbook by Daniel Jurafsky:
Suppose we didn’t use the end-symbol ...
0
votes
0answers
83 views
Interpolation for maximum likelihood estimate of unigram and bigram
We are given the following corpus, modified from the one in the chapter:
...
0
votes
1answer
273 views
More Information about the Question Answering System called LUKE
LUKE is a new state-of-art in question answering system and after googled keywords LUKE Studio Ousia NAIST and RIKEN AIP (I suppose LUKE is a colaboration between several research centers) I couldn't ...
1
vote
1answer
90 views
Explain better this new inventory for Word Sense Disambiguation
My question is about CSI (Coarse Sense Inventory), described in the paper CSI A Coarse Sense Inventory for 85% Word Sense Disambiguation (C Lacerra, M Bevilacqua, T Pasini, R Navigli).
Before the ...
0
votes
0answers
26 views
Optimal algorithm for making queries to a database
There is a database of, let's say, 500k English two-word combinations (e.g. "clover arc", "minister horse"). I can search for an arbitrary string and I will get a list of the alphabetically first 1000 ...
1
vote
1answer
100 views
Is there a meta text processing concept in CS?
I understand that text processing could be done in various ways on top of operating systems:
Shell utilities for processing a file (and/or a file name): tr, ...
2
votes
1answer
149 views
Similar news detection
How do top news portals detect similar news? For example https://www.bbc.com/news/world-asia-china-51431087, if you go to this webpage, you can see the "More on this story" section at the bottom of ...
4
votes
0answers
36 views
How to identify measure words in Chinese text?
Measure words (aka classifiers) are used in Chinese to "measure" things, e.g.
三杯牛奶
Three glasses of milk
那个人
That person
一只乌鸦
One crow
一公斤豆腐
A kilo of tofu
We don't have an equivalent in ...
3
votes
1answer
61 views
Phrase generation approaches
In generating reports, sometimes there is a need to produce quite involved phrases in one of natural languages given numerical or boolean parameters.
To get a feel of it, it is enough to take a look ...
0
votes
0answers
18 views
What is the main concept of using lexical,linguistic, semantic or syntactic approach in NLP for cyberbullying
Am really in need of some explanation, am working on a nlp cyberbullying detection tool which i will deploy to the web using django framework, however, am stuck on some idea, can someone explain to me....
4
votes
1answer
6k views
How to calculate an accurate estimated reading time of text?
I suppose the calculation should not be done by only two factors (average reading speed/words per minute, and word count). But at least by a third parameter, that in my opinion should measure the ...
2
votes
0answers
44 views
Question about word embeddings in a specific language model - GPT-2
How were the GPT-2 token embeddings constructed?
The authors mention that they used Byte Pair Encoding to construct their vocabulary. But BPE is a compression algorithm that returns a list of ...
3
votes
0answers
50 views
Explain the process of formation of word senses in WordNet
Regarding the word sense disambiguation problem, read the following fact written on Knowledge-based Word Sense Disambiguation using Topic Models:
Note that although WordNet is the most widely used ...
1
vote
0answers
44 views
What are open problems in computer science? [closed]
I should prepare some paper for a colloquium (kinda student-task) and it should cover the following points:
(1) at least one notable discovery in theoretical informatics (or computer science)
(2) at ...
1
vote
0answers
37 views
CYK algorithm - how to handle unknown terminals given in a sentence to parse?
There is a given treebank which we derive the Probabilistic context free grammar.
I wonder how do one handles with a given sentence which includes terminals that don't exist in the derived rules?
Is ...
1
vote
1answer
375 views
How does SpaCy make its dependency tree?
I discovered that SpaCy had the ability to make dependency trees. For instance given a question “To whom did the Virgin Mary allegedly appear in 1858 in Lourdes France?”
We can create its dependency ...
1
vote
0answers
33 views
How can I find the perplexity of a text by the perplexity of its sentences?
For a bigram language model, I can calculate the perplexity of sentences of a test document. However, I'm not sure what would be the perplexity of the whole document. Should I get the average of the ...
3
votes
1answer
168 views
Bottom up chart parser adding active arc step
I am following Bottom up chart parsing algorithm from Natural Language Understanding book by James Allen. It is
I couldn't understand the 3rd step. I thought that when active arc is added the dot ...
1
vote
0answers
42 views
Text Segmentation Problem give Word Frequencies in a Universe
Given a dictionary of words and their frequencies (how many times they appear in a universe and given a string(no spaces, punctuation, etc.). What is the best way to segment into individual words? I ...
2
votes
0answers
15 views
Are there any neural NLG systems which don't generate in left-to-right order?
For a while, all classification tasks in natural language processing were based on simple RNN's, which operate in a very word-by-word order. Adding gating mechanisms increased ability to "look back", ...
1
vote
0answers
33 views
Generalization of formal grammars - production rules with more general functions?
Usually formal grammars have production rules in the format N=tNt where simple concatenation function is used for the expansion of the nonterminal. https://www....
1
vote
1answer
109 views
Show that the Laplace smoothing for bigrams is a valid probability distribution
If we consider any smoothing technique like laplace or delta smoothing. Intuitively we can see that the we are stealing from sequences with non zero probablity and re distribute to sequences with zero ...
2
votes
1answer
125 views
How to represent symbolic knowledge using real numbers - theory about neural networks and natural/analog computing?
One can define the semantics of one definite word using the references to real world entities, relationships with the other words and other concepts and represent all this knowledge about this one ...
4
votes
2answers
163 views
Measuring the information of a document?
I'd like to measure how much information a document $D$ contains.
Clearly, the New York Times published yesterday contains more information than my diary wrote on the same day. But, I do not know ...
4
votes
0answers
22 views
How to represent sentences with their dependency parses as input to an RNN?
I am working on a task embedding sentences into a lower-dimensional space according to style, both grammatical and lexical. As such, I want to have as input the linear ordering of tokens in each ...
3
votes
0answers
49 views
How to compute the loss and backprop of word2vec skip-gram using hierarchical softmax?
So we are calculating the loss
$$J(\theta) = -\frac{1}{T}\sum_{t=1}^T\sum_{-m \leq j \leq m} \log P(w_{t+j}|w_t;\theta)$$
and to do this we need to calculate
$$P(o|c) = \frac{\exp(u_o^Tv_c)}{\sum \...
1
vote
1answer
121 views
How to transform lambda function to multi-argument lambda function and how to rewrite or approximate terms?
I am trying to do the formal semantics (Montague grammar, abstract categorial grammar) of natural language and encode the sentence John is boss. The type system has ...
2
votes
1answer
25 views
Marginalise edge weights on graph
I have a directed acyclic graph with a score on each edge. The score of a path is defined to be the sum of the scores on the edges along this path. The probability of a path is the score of such a ...
5
votes
2answers
808 views
Estimate entropy, based upon observed frequency counts
Suppose I have $n$ independent observations $x_1,\dots,x_n$ from some unknown distribution over a known alphabet $\Sigma$, and I want to estimate the entropy of the distribution. I can count the ...
1
vote
0answers
386 views
How can node2vec help find similar “roles” within a graph (nodes whose connections have similar structure within the graph)?
I have a question on the node2vec algorithm described in this paper.
Node2vec is a deep learning algorithm that word2vec to graphs to learn embeddings. The authors claim that it can help find nodes ...
