Questions tagged [natural-language-processing]

Natural language processing (NLP)

Filter by
Sorted by
Tagged with
0
votes
1answer
12 views

sentiment analysis by sentence similarity

How can sentence similarity be used for sentiment analysis? I know we have sentence Bert which can use cosine sim to measure the distance between to vectors (the sentence embeddings), but has anyone ...
2
votes
1answer
35 views

Bible Verse division

I'm working on a project compiling various versions of the Bible into a dataset. For the most part versions separate verses discreetly. In some versions, however, verses are combined. Instead of verse ...
0
votes
1answer
23 views

What Happens if I swap the forget gate and update gate in LSTM model?

Consider the following eqautions used in LSTM ( taken from Andrew ng's course on Sequential model) In an LSTM model, LSTM Cell has three inputs at any time step t Input($X_t , a^{(t-1)}, C^{(t-1)})$, ...
1
vote
0answers
32 views

How can Kneser-Ney Smoothing be integrated into a neural language model?

I found a paper titled Multimodal representation: Kneser-Ney Smoothing/Skip-Gram based neural language model. I am curious about how the Kneser-Ney Smoothing technique can be integrated into a feed-...
0
votes
0answers
119 views

More Information about these Question Answering Systems

My question is about the multi-hop question answering systems described in the page https://hotpotqa.github.io/ and reproduced in the following picture: I am interested in the models BFR-Graph, GSAN-...
0
votes
0answers
21 views

Interpolation for maximum likelihood estimate of unigram and bigram

We are given the following corpus, modified from the one in the chapter: ...
0
votes
0answers
20 views

n-grams textbook question

I have this question I found regarding n-gram modeling in the Speech and Language Processing textbook by Daniel Jurafsky: Suppose we didn’t use the end-symbol ...
-1
votes
1answer
36 views

Natural Languages

Can one imply that natural languages can be described by regular grammar? Is that what happens through NPL? Trying to understand the subject of how spoken language can be converted to data and how.
0
votes
1answer
175 views

More Information about the Question Answering System called LUKE

LUKE is a new state-of-art in question answering system and after googled keywords LUKE Studio Ousia NAIST and RIKEN AIP (I suppose LUKE is a colaboration between several research centers) I couldn't ...
1
vote
1answer
80 views

Explain better this new inventory for Word Sense Disambiguation

My question is about CSI (Coarse Sense Inventory), described in the paper CSI A Coarse Sense Inventory for 85% Word Sense Disambiguation (C Lacerra, M Bevilacqua, T Pasini, R Navigli). Before the ...
0
votes
0answers
24 views

Optimal algorithm for making queries to a database

There is a database of, let's say, 500k English two-word combinations (e.g. "clover arc", "minister horse"). I can search for an arbitrary string and I will get a list of the alphabetically first 1000 ...
2
votes
1answer
137 views

Similar news detection

How do top news portals detect similar news? For example https://www.bbc.com/news/world-asia-china-51431087, if you go to this webpage, you can see the "More on this story" section at the bottom of ...
4
votes
0answers
34 views

How to identify measure words in Chinese text?

Measure words (aka classifiers) are used in Chinese to "measure" things, e.g. 三杯牛奶 Three glasses of milk 那个人 That person 一只乌鸦 One crow 一公斤豆腐 A kilo of tofu We don't have an equivalent in ...
1
vote
1answer
51 views

What kind of bigram probability smoothing is this?

I hope it isn't off topic but I need to understand this example. Given the corpus 12 1 13 12 15 234 2526 and smoothing factor of ...
0
votes
0answers
18 views

What is the main concept of using lexical,linguistic, semantic or syntactic approach in NLP for cyberbullying

Am really in need of some explanation, am working on a nlp cyberbullying detection tool which i will deploy to the web using django framework, however, am stuck on some idea, can someone explain to me....
1
vote
1answer
98 views

Is there a meta text processing concept in CS?

I understand that text processing could be done in various ways on top of operating systems: Shell utilities for processing a file (and/or a file name): tr, ...
0
votes
0answers
10 views

Contingency Table Confusion NLP

Hello for the contingency table: [true positive, false negative, false positive, true negative]. I am having a hard time remembering the difference between these terms because all the terms are ...
2
votes
0answers
31 views

Question about word embeddings in a specific language model - GPT-2

How were the GPT-2 token embeddings constructed? The authors mention that they used Byte Pair Encoding to construct their vocabulary. But BPE is a compression algorithm that returns a list of ...
0
votes
1answer
40 views

Which algorithm for predicting the next word(s) based on previous words, given a sentence?

I want to input some words, and out comes the next word(s). Neural nets are really hot at the moment, and I'm afraid of throwing a neural net at something, when one is not really needed. Or... maybe ...
3
votes
0answers
46 views

Explain the process of formation of word senses in WordNet

Regarding the word sense disambiguation problem, read the following fact written on Knowledge-based Word Sense Disambiguation using Topic Models: Note that although WordNet is the most widely used ...
1
vote
0answers
43 views

What are open problems in computer science? [closed]

I should prepare some paper for a colloquium (kinda student-task) and it should cover the following points: (1) at least one notable discovery in theoretical informatics (or computer science) (2) at ...
0
votes
1answer
55 views

CYK result de-transformation

Suppose we have a rules derived from a treebank. And in order to get a syntax tree of a given sentence we use the cyk algorithm. In order to use the cyk we should convert the rules into chomsky normal ...
1
vote
0answers
34 views

CYK algorithm - how to handle unknown terminals given in a sentence to parse?

There is a given treebank which we derive the Probabilistic context free grammar. I wonder how do one handles with a given sentence which includes terminals that don't exist in the derived rules? Is ...
0
votes
2answers
366 views

N-gram language model question

I have this question I found regarding n-gram modelling in the Speech and language processing text book: ...
1
vote
0answers
31 views

How can I find the perplexity of a text by the perplexity of its sentences?

For a bigram language model, I can calculate the perplexity of sentences of a test document. However, I'm not sure what would be the perplexity of the whole document. Should I get the average of the ...
3
votes
1answer
155 views

Bottom up chart parser adding active arc step

I am following Bottom up chart parsing algorithm from Natural Language Understanding book by James Allen. It is I couldn't understand the 3rd step. I thought that when active arc is added the dot ...
1
vote
0answers
39 views

Text Segmentation Problem give Word Frequencies in a Universe

Given a dictionary of words and their frequencies (how many times they appear in a universe and given a string(no spaces, punctuation, etc.). What is the best way to segment into individual words? I ...
2
votes
0answers
13 views

Are there any neural NLG systems which don't generate in left-to-right order?

For a while, all classification tasks in natural language processing were based on simple RNN's, which operate in a very word-by-word order. Adding gating mechanisms increased ability to "look back", ...
1
vote
0answers
31 views

Generalization of formal grammars - production rules with more general functions?

Usually formal grammars have production rules in the format N=tNt where simple concatenation function is used for the expansion of the nonterminal. https://www....
1
vote
1answer
90 views

Show that the Laplace smoothing for bigrams is a valid probability distribution

If we consider any smoothing technique like laplace or delta smoothing. Intuitively we can see that the we are stealing from sequences with non zero probablity and re distribute to sequences with zero ...
2
votes
1answer
114 views

How to represent symbolic knowledge using real numbers - theory about neural networks and natural/analog computing?

One can define the semantics of one definite word using the references to real world entities, relationships with the other words and other concepts and represent all this knowledge about this one ...
4
votes
0answers
22 views

How to represent sentences with their dependency parses as input to an RNN?

I am working on a task embedding sentences into a lower-dimensional space according to style, both grammatical and lexical. As such, I want to have as input the linear ordering of tokens in each ...
3
votes
0answers
40 views

How to compute the loss and backprop of word2vec skip-gram using hierarchical softmax?

So we are calculating the loss $$J(\theta) = -\frac{1}{T}\sum_{t=1}^T\sum_{-m \leq j \leq m} \log P(w_{t+j}|w_t;\theta)$$ and to do this we need to calculate $$P(o|c) = \frac{\exp(u_o^Tv_c)}{\sum \...
1
vote
1answer
109 views

How to transform lambda function to multi-argument lambda function and how to rewrite or approximate terms?

I am trying to do the formal semantics (Montague grammar, abstract categorial grammar) of natural language and encode the sentence John is boss. The type system has ...
1
vote
1answer
317 views

How does SpaCy make its dependency tree?

I discovered that SpaCy had the ability to make dependency trees. For instance given a question “To whom did the Virgin Mary allegedly appear in 1858 in Lourdes France?” We can create its dependency ...
1
vote
0answers
329 views

How can node2vec help find similar “roles” within a graph (nodes whose connections have similar structure within the graph)?

I have a question on the node2vec algorithm described in this paper. Node2vec is a deep learning algorithm that word2vec to graphs to learn embeddings. The authors claim that it can help find nodes ...
2
votes
1answer
24 views

Marginalise edge weights on graph

I have a directed acyclic graph with a score on each edge. The score of a path is defined to be the sum of the scores on the edges along this path. The probability of a path is the score of such a ...
2
votes
1answer
1k views

Subsampling of Frequent Words in Word2Vec

I am reading through the following paper: https://papers.nips.cc/paper/5021-distributed-representations-of-words-and-phrases-and-their-compositionality.pdf Under section 2.3 on page 4 the authors ...
1
vote
1answer
100 views

Question on word probability for hierarchical softmax used in natural language processing

I am reading the following paper: https://papers.nips.cc/paper/5021-distributed-representations-of-words-and-phrases-and-their-compositionality.pdf On page 4 of the paper they describe the ...
0
votes
1answer
58 views

Naive Bayes' Classification and Using the Entire Vocabulary in the Denominator

I am working through the NLP notes for Naive Bayes' classification here: https://web.stanford.edu/~jurafsky/slp3/6.pdf Below $c$ is the class of the observation and $w_i$ is the $i$th word of a text ...
2
votes
0answers
16 views

How does the Earley parser accept the input string as syntactically correct?

https://en.wikipedia.org/wiki/Earley_parser An input string of length $n$ is syntactically correct if at least one $(S \rightarrow X_1 ... X_m • 0)$ is in $E_n$. Why is this the case ? The part ...
3
votes
0answers
65 views

Proving the probability of zero occurrences in training using Good-Turing maximum likelihood estimate

Background Good-Turing (GT) smoothing is used in language models to estimate the counts of words in the test set that have not been seen in the training set. In GT smoothing, $N_c$ is the count of ...
0
votes
0answers
32 views

Any references/current work on anomaly detection in conversations?

Suppose I have a lot of data on conversations between humans and chatbots (human text, chatbot text, times, media used for chat, etc), and I want to be able to detect anomalies in these conversations. ...
2
votes
0answers
54 views

Automatic learning/discovery of logics

Are there efforts to automatically discover new logics? Logics are simple structures - they have formal language, deduction rules, semantics and certain properties that are proved or discarded for ...
3
votes
0answers
141 views

Semantic parsing with Grammatical Framework - is this possible?

So far I have learned about categorial grammars, type logical grammars and formal semantics of natural language, the relevant tools are Cornell Semantic Parsing Framework https://github.com/clic-lab/...
1
vote
1answer
31 views

How to figure out whether two texts refer to the same object or event

Let's assume there is something happen in the world - Football world cup final. And team-1 beat team-2 with the score 3:2. So there is whole bunch of articles on every website about it, each contains ...
0
votes
1answer
210 views

Converting CFG to CNF [closed]

need some help with the following question: I've watched a few youtube tutorials but I'm struggling to convert this specific CFG
3
votes
0answers
95 views

How to translate lambda calculus into (first-order, modal) logic, is it possible at all?

It is possible (using formal semantics) to translate natural language sentences into lambda expressions. So, is it possible to translate those lambda expressions into some logic, e.g. into first-order ...
2
votes
0answers
54 views

Semantic/DRT methods for conversational agents / chatbots / dialogue systems - reference request?

The wiki pages about chabots mention that statistical methods, keyword search and precompiled answers are used for the chatbots. But I feel that there should exist different - semantical approach for ...
6
votes
1answer
897 views

nlp: phonetic edit distance between a word and the closest of a set of words

Let's say someone is using Dragon Dictation, Google Speech, or some other free form dictation software (it will recognize anything they say to the best of its ability). I have some reasonably large ...