Questions tagged [natural-language-processing]

Natural language processing (NLP)

Filter by
Sorted by
Tagged with
1 vote
1 answer
52 views

Is the fact true that GPT-3 wasn't trained on any explicitly gathered code dataset?

The paper mentions a line about GPT-3's training dataset - ...
Deepak Ahire's user avatar
0 votes
1 answer
30 views

Strange notation in the article about sparse self-attention

While reading an article devoted to the sparse self-attention, I came across a notation that was not very clear: $$ Attend(X, S) = \Big( a(x_i, S_i) \Big)_{i∈{\{1,...,n}\}} $$ What means $\Big( \space ...
b1ackf0x's user avatar
1 vote
0 answers
27 views

Are there known algorithms for generating abbreviations from a (sequence) of words (similar to hyphenation)

I'm currently developing a custom UI layout library. At the moment a user of that library should supply strings for UI labels in a format that allow for subsequent shortening of those labels, so that ...
datenwolf's user avatar
  • 111
0 votes
0 answers
28 views

n-gram model: why conditioning on the start symbol <s>?

from the book, Speech and Language Processing (3rd ed. draft) I understand conditioning on <s> will give context about the first word. From the formula above, ...
Izana's user avatar
  • 131
0 votes
0 answers
28 views

Dilated LSTM vs LSTM

I'm currently reading a paper for a literature review related to NLP and its research employs a dilated LSTM model. What is the difference between this and a regular LSTM, or more specifically what ...
paul lacher's user avatar
0 votes
0 answers
7 views

Does T5 or another embedding embed every possible length tokenization?

I’m curious about an embedding technique where every possible “tokenization” of a text gets an embedding - not just individuals words, but every single 2-gram, 3-gram, and n-gram. Does this exist? Or ...
Julius H.'s user avatar
  • 123
0 votes
2 answers
70 views

ML model to pick to classify data with inputs as an array of strings?

I have a medical dataset, which is something like this ...
Ashrjz's user avatar
  • 1
1 vote
1 answer
46 views

How can a Machine Learning model predict this classification problem?

I am pretty new to ML and have a basic but fundamental question. Let’s imagine we want to create a simple Sentiment Analysis model using Machine Learning not Deep Learning algorithms, so we need to ...
Z Bokaee's user avatar
-1 votes
2 answers
99 views

If Large Language Models like ChatGPT grow, do also their problems grow?

In a video Noam Chomsky said, that if these LLM get bigger, than also the things they are not good at get bigger too. He doesn't explain more details about this. So is this true? In what way do their ...
user117640's user avatar
0 votes
0 answers
37 views

How to avoid anthropomorphic language when talking about AI?

In my opinion we should not talk about chatgtp or similar AIs as if they have agency, because that will lead to people thinking these programs are real intelligent beings. So I think we should avoid ...
user117640's user avatar
1 vote
0 answers
45 views

How should one interpret the minhash error rate?

On Wikipedia the error rate for the minhash algorithm vs baseline jaccard similarity is stated as 1/sqrt(1/hashes). But, how should one interpret this number in a ...
micimize's user avatar
  • 111
1 vote
1 answer
46 views

Resources to learn NLP

I am an undergraduate student in mathematics. I have a fair bit of experience with deep learning in computer vision research and am willing to dabble into NLP. I hope that things won't be very ...
The Limit Does Not Exist's user avatar
0 votes
0 answers
57 views

Constraint satisfaction problems in Natural language processing

I have just started learning about CSP and NLP, for which I have to write a review paper of some research articles. The problem is that when a searched for research articles on some trusted digital ...
Yllka's user avatar
  • 1
2 votes
3 answers
170 views

Does ChatGPT use specific sub-programs

I have had a somewhat hard time trying to understand how ChatGPT can "solve" some tasks that cannot be entirely cast as language-model-based rephrasing of textual subsets of the internet ...
Radio Controlled's user avatar
0 votes
1 answer
4k views

What are the 175 billion parameters used in the GPT-3 language model?

I am currently working my way through Language Models are Few-Shot Learners , the initial 75-page paper about GPT-3, the language learning model spawning off into ChatGTP. In it, they mention several ...
Lance's user avatar
  • 2,193
1 vote
2 answers
65 views

Correctly compare names with different romanization?

I am running into a problem in which I have names with different romanization methods, and I need to correctly compare them to see which names match. An example is romanized Korean variants: Seong vs ...
hologram's user avatar
  • 111
1 vote
1 answer
113 views

Efficient search of a large set of documents to find documents that only contain a particular set of words

Say I have a set of documents $D = \{d_1, d_2, \dots, d_n\}$ in some natural language. Each document $d_i$ consists of a subset of words from a word pool $W = \{w_1, w_2, \dots, w_k\}$. For example, $\...
Xavier Taylor's user avatar
1 vote
0 answers
18 views

What is the draw back of Bag of Words model?

Bag of words model rank words in a particular document in such a way that If it is occurring into that particular document multiple times and it's frequency is low for other documents then it's weight ...
XYZ's user avatar
  • 123
3 votes
2 answers
865 views

Why word embeddings are compared with cosine distance and not euclidean?

In most articles that compare word embeddings they use cosine distance to determine if words are similar. Why? I guess that euclidean distance should work too. So, my question is: it doesn't? And why ...
estebarb's user avatar
  • 152
1 vote
0 answers
49 views

How do you detect names that sound similar to English words or phrases?

High school teachers use this Website called Kahoot to post quizzes that are answered live by the students. Students sign in to the quiz and are invited to input a name for themselves, and as you ...
moonman239's user avatar
1 vote
0 answers
192 views

How should I create a word-embedding an NLP model recognizing HTML elements?

I'm currently doing a small-time project where I have to deliver a model which can classify specific elements on a web-page using the HTML code. For this, I have considered using the HTML tags for ...
GoldenChicken's user avatar
1 vote
2 answers
527 views

An integer (a string of digits) is $\text{/[0-9][0-9]$\ast$/}$. (Why isn't it just $\text{/[0-9]$\ast$/}$?)

I am currently studying the textbook Speech and Language Processing, 3rd edition (draft), by Jurafsky and Martin. Chapter 2.1.1 Basic Regular Expression Patterns says the following: The regular ...
The Pointer's user avatar
0 votes
0 answers
25 views

Is this problem decideable programatically?

Consider the following 3 strings: 1tb hdd 256gb ssd hdd 256gb 1tb hdd If you know, that all ...
Adam Baranyai's user avatar
1 vote
3 answers
947 views

Compute the edit distance between two words in which substitution is not allowed

How do I compute the edit distance between two words in which substitution is not allowed? The allowed operations include insertion (with cost 1) and deletion (with cost 1), but not substitution. How ...
human's user avatar
  • 121
1 vote
0 answers
24 views

Calculate coherence for non-gensim topic model [closed]

I've built a topic model, with: Input: list of tokenized lists Output: a m x t matrix (with each cell indicating the probability of word i appearing in topic k). Output: a k x n matrix (with each ...
Emil's user avatar
  • 121
2 votes
0 answers
36 views

Generating an "approximate" grammar

This is my first time posting here, so I hope I'm on topic. I have a table of natural-language data of the form ...
marcelgoh's user avatar
  • 121
1 vote
0 answers
25 views

Is relation extraction considered a subtask of information extraction?

I'm currently trying to investigate the relationship between relation extraction (RE) and event extraction (EE). Doing more reading on the two tasks has caused me to question my initial belief that RE ...
Sean's user avatar
  • 570
0 votes
0 answers
21 views

should you take question mark as a separate word in bigram language modelling when finding probability?

the corpus what is the shape ? what is the colour ? what colour is it ? it is what ? is it red ? what is it ? what shape is it ? it is red it is green the colour is red the shape ...
ap1711's user avatar
  • 1
0 votes
2 answers
89 views

sentiment analysis by sentence similarity

How can sentence similarity be used for sentiment analysis? I know we have sentence Bert which can use cosine sim to measure the distance between to vectors (the sentence embeddings), but has anyone ...
Kwaku's user avatar
  • 1
2 votes
1 answer
52 views

Bible Verse division

I'm working on a project compiling various versions of the Bible into a dataset. For the most part versions separate verses discreetly. In some versions, however, verses are combined. Instead of verse ...
rwreed's user avatar
  • 121
0 votes
1 answer
59 views

What Happens if I swap the forget gate and update gate in LSTM model?

Consider the following eqautions used in LSTM ( taken from Andrew ng's course on Sequential model) In an LSTM model, LSTM Cell has three inputs at any time step t Input($X_t , a^{(t-1)}, C^{(t-1)})$, ...
black sheep 369's user avatar
1 vote
0 answers
46 views

How can Kneser-Ney Smoothing be integrated into a neural language model?

I found a paper titled Multimodal representation: Kneser-Ney Smoothing/Skip-Gram based neural language model. I am curious about how the Kneser-Ney Smoothing technique can be integrated into a feed-...
Dimitris Dimitriadis's user avatar
0 votes
0 answers
447 views

Interpolation for maximum likelihood estimate of unigram and bigram

We are given the following corpus, modified from the one in the chapter: ...
Ohanzee's user avatar
0 votes
0 answers
146 views

n-grams textbook question

I have this question I found regarding n-gram modeling in the Speech and Language Processing textbook by Daniel Jurafsky: Suppose we didn’t use the end-symbol ...
Ohanzee's user avatar
-1 votes
1 answer
46 views

Natural Languages

Can one imply that natural languages can be described by regular grammar? Is that what happens through NPL? Trying to understand the subject of how spoken language can be converted to data and how.
Dusan Biga's user avatar
0 votes
1 answer
328 views

More Information about the Question Answering System called LUKE

LUKE is a new state-of-art in question answering system and after googled keywords LUKE Studio Ousia NAIST and RIKEN AIP (I suppose LUKE is a colaboration between several research centers) I couldn't ...
R. S.'s user avatar
  • 167
1 vote
1 answer
99 views

Explain better this new inventory for Word Sense Disambiguation

My question is about CSI (Coarse Sense Inventory), described in the paper CSI A Coarse Sense Inventory for 85% Word Sense Disambiguation (C Lacerra, M Bevilacqua, T Pasini, R Navigli). Before the ...
R. S.'s user avatar
  • 167
0 votes
0 answers
30 views

Optimal algorithm for making queries to a database

There is a database of, let's say, 500k English two-word combinations (e.g. "clover arc", "minister horse"). I can search for an arbitrary string and I will get a list of the alphabetically first 1000 ...
Dariush's user avatar
  • 101
2 votes
1 answer
187 views

Similar news detection

How do top news portals detect similar news? For example https://www.bbc.com/news/world-asia-china-51431087, if you go to this webpage, you can see the "More on this story" section at the bottom of ...
Nabin's user avatar
  • 121
4 votes
0 answers
41 views

How to identify measure words in Chinese text?

Measure words (aka classifiers) are used in Chinese to "measure" things, e.g. 三杯牛奶 Three glasses of milk 那个人 That person 一只乌鸦 One crow 一公斤豆腐 A kilo of tofu We don't have an equivalent in ...
Rebecca J. Stones's user avatar
1 vote
1 answer
101 views

What kind of bigram probability smoothing is this?

I hope it isn't off topic but I need to understand this example. Given the corpus 12 1 13 12 15 234 2526 and smoothing factor of ...
user2495207's user avatar
0 votes
0 answers
18 views

What is the main concept of using lexical,linguistic, semantic or syntactic approach in NLP for cyberbullying

Am really in need of some explanation, am working on a nlp cyberbullying detection tool which i will deploy to the web using django framework, however, am stuck on some idea, can someone explain to me....
junoirt's user avatar
1 vote
1 answer
106 views

Is there a meta text processing concept in CS?

I understand that text processing could be done in various ways on top of operating systems: Shell utilities for processing a file (and/or a file name): tr, ...
user avatar
2 votes
0 answers
80 views

Question about word embeddings in a specific language model - GPT-2

How were the GPT-2 token embeddings constructed? The authors mention that they used Byte Pair Encoding to construct their vocabulary. But BPE is a compression algorithm that returns a list of ...
Sam's user avatar
  • 121
0 votes
1 answer
69 views

Which algorithm for predicting the next word(s) based on previous words, given a sentence?

I want to input some words, and out comes the next word(s). Neural nets are really hot at the moment, and I'm afraid of throwing a neural net at something, when one is not really needed. Or... maybe ...
HelloWorld's user avatar
3 votes
0 answers
54 views

Explain the process of formation of word senses in WordNet

Regarding the word sense disambiguation problem, read the following fact written on Knowledge-based Word Sense Disambiguation using Topic Models: Note that although WordNet is the most widely used ...
R. S.'s user avatar
  • 167
1 vote
0 answers
109 views

What are open problems in computer science? [closed]

I should prepare some paper for a colloquium (kinda student-task) and it should cover the following points: (1) at least one notable discovery in theoretical informatics (or computer science) (2) at ...
Constantin's user avatar
0 votes
1 answer
154 views

CYK result de-transformation

Suppose we have a rules derived from a treebank. And in order to get a syntax tree of a given sentence we use the cyk algorithm. In order to use the cyk we should convert the rules into chomsky normal ...
KaliTheGreat's user avatar
2 votes
0 answers
49 views

CYK algorithm - how to handle unknown terminals given in a sentence to parse?

There is a given treebank which we derive the Probabilistic context free grammar. I wonder how do one handles with a given sentence which includes terminals that don't exist in the derived rules? Is ...
KaliTheGreat's user avatar
1 vote
3 answers
2k views

N-gram language model question

I have this question I found regarding n-gram modelling in the textbook Speech and Language Processing: Suppose we didn't use the end symbol </s>. Train an unsmoothed bigram grammar on the ...
rert588's user avatar
  • 261