Questions tagged [natural-language-processing]
Natural language processing (NLP)
197 questions
1
vote
1
answer
43
views
bijection map of LLM
I just want to know: could we built a bridge or bijection mapping for AI agent between Prompt and Response? In other words, AI ...
- 11
1
vote
0
answers
19
views
Effect of using signed vectors for LSH Random Projection (Simhash)
Simhash is an text similarity algorithm proposed by Moses Charikar in his paper "Similarity Estimation Techniques from Rounding Algorithms". However, in his original paper, he proposed to ...
- 111
0
votes
1
answer
40
views
Forming Sentences
I have a bunch of transcripts from online videos but I suspect the transcripts aren't formatted well. There are no punctuations, sentences are broken abruptly, some words aren't complete (for example ...
- 1
1
vote
1
answer
40
views
Generate product description from product specifications
I am looking for a python NLP library that can generate a proper product description based on product features provided to it.
Till now, i have tried transformers library and this is the code:
...
1
vote
1
answer
58
views
Is the fact true that GPT-3 wasn't trained on any explicitly gathered code dataset?
The paper mentions a line about GPT-3's training dataset -
...
- 111
0
votes
1
answer
37
views
Strange notation in the article about sparse self-attention
While reading an article devoted to the sparse self-attention, I came across a notation that was not very clear:
$$
Attend(X, S) =
\Big(
a(x_i, S_i)
\Big)_{i∈{\{1,...,n}\}}
$$
What means $\Big( \space ...
- 3
1
vote
0
answers
49
views
Are there known algorithms for generating abbreviations from a (sequence) of words (similar to hyphenation)
I'm currently developing a custom UI layout library. At the moment a user of that library should supply strings for UI labels in a format that allow for subsequent shortening of those labels, so that ...
- 111
0
votes
0
answers
41
views
n-gram model: why conditioning on the start symbol <s>?
from the book, Speech and Language Processing (3rd ed. draft)
I understand conditioning on <s> will give context about the first word.
From the formula above, ...
- 131
0
votes
2
answers
149
views
ML model to pick to classify data with inputs as an array of strings?
I have a medical dataset, which is something like this
...
- 1
1
vote
1
answer
49
views
How can a Machine Learning model predict this classification problem?
I am pretty new to ML and have a basic but fundamental question.
Let’s imagine we want to create a simple Sentiment Analysis model using Machine Learning not Deep Learning algorithms, so we need to ...
- 13
0
votes
1
answer
115
views
If Large Language Models like ChatGPT grow, do also their problems grow?
In a video Noam Chomsky said, that if these LLM get bigger, than also the things they are not good at get bigger too. He doesn't explain more details about this. So is this true? In what way do their ...
- 117
1
vote
0
answers
58
views
How should one interpret the minhash error rate?
On Wikipedia the error rate for the minhash algorithm vs baseline jaccard similarity is stated as 1/sqrt(1/hashes).
But, how should one interpret this number in a ...
- 111
1
vote
1
answer
78
views
Resources to learn NLP
I am an undergraduate student in mathematics. I have a fair bit of experience with deep learning in computer vision research and am willing to dabble into Natural Language Processing (NLP). I hope ...
0
votes
0
answers
66
views
Constraint satisfaction problems in Natural language processing
I have just started learning about CSP and NLP, for which I have to write a review paper of some research articles.
The problem is that when a searched for research articles on some trusted digital ...
- 1
2
votes
3
answers
187
views
Does ChatGPT use specific sub-programs
I have had a somewhat hard time trying to understand how ChatGPT can "solve" some tasks that cannot be entirely cast as language-model-based rephrasing of textual subsets of the internet ...
- 187
1
vote
1
answer
5k
views
What are the 175 billion parameters used in the GPT-3 language model?
I am currently working my way through Language Models are Few-Shot Learners
, the initial 75-page paper about GPT-3, the language learning model spawning off into ChatGTP.
In it, they mention several ...
- 2,295
1
vote
2
answers
82
views
Correctly compare names with different romanization?
I am running into a problem in which I have names with different romanization methods, and I need to correctly compare them to see which names match.
An example is romanized Korean variants: Seong vs ...
- 111
1
vote
1
answer
260
views
Efficient search of a large set of documents to find documents that only contain a particular set of words
Say I have a set of documents $D = \{d_1, d_2, \dots, d_n\}$ in some natural language.
Each document $d_i$ consists of a subset of words from a word pool $W = \{w_1, w_2, \dots, w_k\}$. For example, $\...
- 111
1
vote
0
answers
18
views
What is the draw back of Bag of Words model?
Bag of words model rank words in a particular document in such a way that
If it is occurring into that particular document multiple times and it's frequency is low for other documents then it's weight ...
- 123
3
votes
2
answers
1k
views
Why word embeddings are compared with cosine distance and not euclidean?
In most articles that compare word embeddings they use cosine distance to determine if words are similar. Why?
I guess that euclidean distance should work too. So, my question is: it doesn't?
And why ...
- 152
1
vote
0
answers
58
views
How do you detect names that sound similar to English words or phrases?
High school teachers use this Website called Kahoot to post quizzes that are answered live by the students. Students sign in to the quiz and are invited to input a name for themselves, and as you ...
- 185
1
vote
0
answers
222
views
How should I create a word-embedding an NLP model recognizing HTML elements?
I'm currently doing a small-time project where I have to deliver a model which can classify specific elements on a web-page using the HTML code.
For this, I have considered using the HTML tags for ...
1
vote
2
answers
968
views
An integer (a string of digits) is $\text{/[0-9][0-9]$\ast$/}$. (Why isn't it just $\text{/[0-9]$\ast$/}$?)
I am currently studying the textbook Speech and Language Processing, 3rd edition (draft), by Jurafsky and Martin. Chapter 2.1.1 Basic Regular Expression Patterns says the following:
The regular ...
- 217
0
votes
0
answers
25
views
Is this problem decideable programatically?
Consider the following 3 strings:
1tb hdd 256gb ssd
hdd 256gb
1tb hdd
If you know, that all ...
- 103
1
vote
3
answers
1k
views
Compute the edit distance between two words in which substitution is not allowed
How do I compute the edit distance between two words in which substitution is not allowed?
The allowed operations include insertion (with cost 1) and deletion (with cost 1), but not substitution.
How ...
- 121
1
vote
0
answers
35
views
Calculate coherence for non-gensim topic model [closed]
I've built a topic model, with:
Input: list of tokenized lists
Output: a m x t matrix (with each cell indicating the probability of word i appearing in topic k).
Output: a k x n matrix (with each ...
- 121
2
votes
0
answers
39
views
Generating an "approximate" grammar
This is my first time posting here, so I hope I'm on topic. I have a table of natural-language data of the form
...
- 121
1
vote
0
answers
25
views
Is relation extraction considered a subtask of information extraction?
I'm currently trying to investigate the relationship between relation extraction (RE) and event extraction (EE). Doing more reading on the two tasks has caused me to question my initial belief that RE ...
- 580
0
votes
0
answers
24
views
should you take question mark as a separate word in bigram language modelling when finding probability?
the corpus
what is the shape ?
what is the colour ?
what colour is it ?
it is what ?
is it red ?
what is it ?
what shape is it ?
it is red
it is green
the colour is red
the shape ...
- 1
0
votes
2
answers
97
views
sentiment analysis by sentence similarity
How can sentence similarity be used for sentiment analysis? I know we have sentence Bert which can use cosine sim to measure the distance between to vectors (the sentence embeddings), but has anyone ...
- 1
2
votes
1
answer
53
views
Bible Verse division
I'm working on a project compiling various versions of the Bible into a dataset. For the most part versions separate verses discreetly. In some versions, however, verses are combined. Instead of verse ...
- 121
0
votes
1
answer
63
views
What Happens if I swap the forget gate and update gate in LSTM model?
Consider the following eqautions used in LSTM ( taken from Andrew ng's course on Sequential model)
In an LSTM model, LSTM Cell has three inputs at any time step t
Input($X_t , a^{(t-1)}, C^{(t-1)})$, ...
- 101
1
vote
0
answers
47
views
How can Kneser-Ney Smoothing be integrated into a neural language model?
I found a paper titled Multimodal representation: Kneser-Ney Smoothing/Skip-Gram based neural language model. I am curious about how the Kneser-Ney Smoothing technique can be integrated into a feed-...
0
votes
0
answers
514
views
Interpolation for maximum likelihood estimate of unigram and bigram
We are given the following corpus, modified from the one in the chapter:
...
- 1
0
votes
0
answers
196
views
n-grams textbook question
I have this question I found regarding n-gram modeling in the Speech and Language Processing textbook by Daniel Jurafsky:
Suppose we didn’t use the end-symbol ...
- 1
-1
votes
1
answer
46
views
Natural Languages
Can one imply that natural languages can be described by regular grammar? Is that what happens through NPL? Trying to understand the subject of how spoken language can be converted to data and how.
- 13
0
votes
1
answer
331
views
More Information about the Question Answering System called LUKE
LUKE is a new state-of-art in question answering system and after googled keywords LUKE Studio Ousia NAIST and RIKEN AIP (I suppose LUKE is a colaboration between several research centers) I couldn't ...
- 179
1
vote
1
answer
101
views
Explain better this new inventory for Word Sense Disambiguation
My question is about CSI (Coarse Sense Inventory), described in the paper CSI A Coarse Sense Inventory for 85% Word Sense Disambiguation (C Lacerra, M Bevilacqua, T Pasini, R Navigli).
Before the ...
- 179
0
votes
0
answers
31
views
Optimal algorithm for making queries to a database
There is a database of, let's say, 500k English two-word combinations (e.g. "clover arc", "minister horse"). I can search for an arbitrary string and I will get a list of the alphabetically first 1000 ...
- 101
2
votes
1
answer
195
views
Similar news detection
How do top news portals detect similar news? For example https://www.bbc.com/news/world-asia-china-51431087, if you go to this webpage, you can see the "More on this story" section at the bottom of ...
- 121
4
votes
0
answers
42
views
How to identify measure words in Chinese text?
Measure words (aka classifiers) are used in Chinese to "measure" things, e.g.
三杯牛奶
Three glasses of milk
那个人
That person
一只乌鸦
One crow
一公斤豆腐
A kilo of tofu
We don't have an equivalent in ...
1
vote
1
answer
117
views
What kind of bigram probability smoothing is this?
I hope it isn't off topic but I need to understand this example. Given the corpus 12 1 13 12 15 234 2526 and smoothing factor of ...
- 111
0
votes
0
answers
18
views
What is the main concept of using lexical,linguistic, semantic or syntactic approach in NLP for cyberbullying
Am really in need of some explanation, am working on a nlp cyberbullying detection tool which i will deploy to the web using django framework, however, am stuck on some idea, can someone explain to me....
- 1
1
vote
1
answer
106
views
Is there a meta text processing concept in CS?
I understand that text processing could be done in various ways on top of operating systems:
Shell utilities for processing a file (and/or a file name): tr, ...
user109446
2
votes
0
answers
85
views
Question about word embeddings in a specific language model - GPT-2
How were the GPT-2 token embeddings constructed?
The authors mention that they used Byte Pair Encoding to construct their vocabulary. But BPE is a compression algorithm that returns a list of ...
- 121
0
votes
1
answer
70
views
Which algorithm for predicting the next word(s) based on previous words, given a sentence?
I want to input some words, and out comes the next word(s). Neural nets are really hot at the moment, and I'm afraid of throwing a neural net at something, when one is not really needed. Or... maybe ...
- 133
3
votes
0
answers
54
views
Explain the process of formation of word senses in WordNet
Regarding the word sense disambiguation problem, read the following fact written on Knowledge-based Word Sense Disambiguation using Topic Models:
Note that although WordNet is the most widely used ...
- 179
1
vote
0
answers
117
views
What are open problems in computer science? [closed]
I should prepare some paper for a colloquium (kinda student-task) and it should cover the following points:
(1) at least one notable discovery in theoretical informatics (or computer science)
(2) at ...
- 11
0
votes
2
answers
185
views
CYK result de-transformation
Suppose we have a rules derived from a treebank.
And in order to get a syntax tree of a given sentence we use the cyk algorithm.
In order to use the cyk we should convert the rules into chomsky normal ...
- 181
2
votes
0
answers
49
views
CYK algorithm - how to handle unknown terminals given in a sentence to parse?
There is a given treebank which we derive the Probabilistic context free grammar.
I wonder how do one handles with a given sentence which includes terminals that don't exist in the derived rules?
Is ...
- 181