Skip to main content
New
Stack Overflow Jobs powered by Indeed: A job site that puts thousands of tech jobs at your fingertips (U.S. only). Search jobs
14 votes

Implementation of Naive Bayes

The usual trick to avoid this underflow is to compute with logarithms, using the identity $$ \log \prod_{i=1}^n p_i = \sum_{i=1}^n \log p_i. $$ That is, instead of using probabilities, you use their ...
Yuval Filmus's user avatar
7 votes

Semantic readings of the Lambek sequent calculus

What constitutes a proof in a system like this is a derivation which is a tree of rule applications. The above translation function is defined by (structural) recursion over that tree. Note, this is ...
Derek Elkins left SE's user avatar
6 votes
Accepted

How to express modalities in lambda calculus - are some extensions required?

Are extensions required? Not really. You can take an axiomatic description of a modal logic and simply provide a "primitive" lambda term for each. The modal operators would become type constructors. ...
Derek Elkins left SE's user avatar
5 votes
Accepted

nlp: phonetic edit distance between a word and the closest of a set of words

Building on the comments, there's something called the Needleman–Wunsch algorithm, for which you can find the lowest 'cost' alignment of two sequences (phonemes). You can get a good 'confusion cost ...
Brent Allard's user avatar
5 votes
Accepted

An integer (a string of digits) is $\text{/[0-9][0-9]$\ast$/}$. (Why isn't it just $\text{/[0-9]$\ast$/}$?)

The empty word $\epsilon$ isn't an integer. Integers are $0, 1, 2, 3, \dots, 10, 11, \dots$ Clearly none of them have $0$ letters when you write them, and hence they are not $\epsilon$. Hence, you ...
nir shahar's user avatar
  • 11.6k
4 votes

Examples for CFG that cannot be expressed by regular language

If no bracket language examples suffice, your quest is futile. Every context-free-language is a reduction of a bracket/parenthesis language. Reduction comprises homomorphism; intersection with a ...
Thumbnail's user avatar
  • 626
4 votes

Examples for CFG that cannot be expressed by regular language

Theoretically, you can nest sentences arbitrarily depth, by using subclauses and this excluded any finite state mechanism. The inventor of the phrase structure grammars himself looked at finite state ...
StefanH's user avatar
  • 1,449
4 votes

Measuring the information of a document?

Assuming that your data comes from a Markovian source, you can estimate the entropy of the source using an optimal compression algorithm such as Lempel–Ziv, whose theoretical version (without limiting ...
Yuval Filmus's user avatar
4 votes

How does SpaCy make its dependency tree?

Matthew Honnibal, who created Spacy, answered me : The full answer is rather complicated :). I've written a few blog posts over the years to make it more accessible. The best place to start is ...
Revolucion for Monica's user avatar
4 votes

How to transform lambda function to multi-argument lambda function and how to rewrite or approximate terms?

You are looking for currying and uncurrying which transform functions of type $A \times B \to C$ to functions of type $A \to (B \to C)$, and vice versa. Currying takes ...
Andrej Bauer's user avatar
  • 30.5k
3 votes

Specific case of a ranking model

Build a boolean classifier that classifies a message as either "task" or "not a task". Choose a classifier that can output a confidence score (many classifiers can, including ensembles like random ...
D.W.'s user avatar
  • 160k
3 votes

Advice needed. NLP and ML. Where to start?

Spell checking, grammar checking, proofreading with ML Norvig: How to Write a Spelling Corrector "Artistic Style Transfer" for articles (if it is even possible), i.e. "transfer" Shakespeare'...
Martin Thoma's user avatar
  • 2,360
3 votes
Accepted

Advice needed. NLP and ML. Where to start?

As far as I can see, your reading list lacks specific NLP introductions. A really good starting point is Dan Jurafsky and Chris Manning's Coursera course (for example here https://www.youtube.com/...
Sven Büchel's user avatar
3 votes

What is the Best and easiest way to create a Classifer for Sentiment Analysis

This isn't the best way but it's simple and fast: preprocess the input with the Porter Stemmming algorithm (it's a process for removing the commoner morphological and inflexional endings from words ...
manlio's user avatar
  • 2,062
3 votes

Are there repositories of automatically generated (spam) webpages?

Question: Where can I access automatically generated (spam) webpages? You could use: a parody generator program. They're usually based on Markov chains (e.g. SubredditSimulator) or context-free ...
manlio's user avatar
  • 2,062
3 votes
Accepted

Subsampling of Frequent Words in Word2Vec

My guess is that we should regard $f(w_i)$ as a number between 0 and 1, i.e., the number of times word $w_i$ appears divided by the number of words. This makes it an estimate of the probability of ...
D.W.'s user avatar
  • 160k
2 votes

Finding interesting anagrams

It doesn't cover the exact algorithm you had in mind (which Tsuyoshi Ito's answer does), but trying to get at the underlying problem of finding "interesting" anagrams... My first thought was to use ...
wren romano's user avatar
2 votes

How to compute the TF-IDF scores for a handful of documents, but without my own corpus?

Answer What do you value in a ranking function? If you don't value TF or IDF, then it'll be difficult for you to achieve what you want to achieve. Without a corpus however, you could use a frequency ...
aces's user avatar
  • 251
2 votes

Applying Machine learning in biological data

A simple starting point would be to apply a bag-of-words feature vector and try using a naive Bayes or logistic regression classifier. You'll probably want to apply stemming and lemmatization and ...
D.W.'s user avatar
  • 160k
2 votes
Accepted

How to calculate an accurate estimated reading time of text?

There is a similar question, so I will add a bit over the answer. There are too many factors independent of language but solely based on individual reading abilities that such estimate would not be ...
Evil's user avatar
  • 9,465
2 votes
Accepted

How to represent text for a program to add punctuation to a block of text?

I am making two assumptions here about things that, to me, are not quite clear from your question When you say you removed punctuation, you actually meant whitespaces as well. You tried neural ...
Sven Büchel's user avatar
2 votes
Accepted

Understanding the geometrical interpretation of word2vec

If two words tend to appear in the same context, then they will tend to receive a similar word vector. For instance, a large corpus might contain sentences like "I really love Fuji apples" and "I ...
D.W.'s user avatar
  • 160k
2 votes
Accepted

Question on word probability for hierarchical softmax used in natural language processing

I have figured out my confusion. The indicator function is $\lbrack x \rbrack$ is set to output a $-1$ or a $1$ based on the argument being false or true, respectively. When the authors say "let $...
ClownInTheMoon's user avatar
2 votes

How to represent symbolic knowledge using real numbers - theory about neural networks and natural/analog computing?

No, probably not. I think you're expecting too much from the current state of the art in word embeddings. Word embeddings don't magically capture all semantic knowledge. They don't reflect perfect ...
D.W.'s user avatar
  • 160k
2 votes
Accepted

Bottom up chart parser adding active arc step

You have spotted a typo in that book. The dot $\circ$ in step 3 of figure 3.11 should be after constituency C. To be explicit and precise, step 3 should be stated as the following. For each ...
John L.'s user avatar
  • 39k
2 votes

Similar news detection

This is a proprietary system so we have no way of knowing. In general, you will find a lot of work and potential solutions by searching for text similarity or document similarity. For starters, you ...
Juho's user avatar
  • 22.6k
2 votes

Bible Verse division

In my opinion, this may be best approached as a sequence tagging problem, similar to part-of-speech tagging or named entity recognition. (So, this would be the seq2seq option, rather than regular ...
Sven Büchel's user avatar
2 votes

An integer (a string of digits) is $\text{/[0-9][0-9]$\ast$/}$. (Why isn't it just $\text{/[0-9]$\ast$/}$?)

$[0-9]*$ includes all finite strings of the digits including the empty string $\epsilon$ (Some books denote the empty string by $\lambda$). An integer, by definition, has at least one digit. Hence, $[...
Pratik Haware's user avatar
2 votes

Efficient search of a large set of documents to find documents that only contain a particular set of words

You are trying to compute the set $$E = \{e \in D \mid e \subseteq K \land e \cap S \ne \emptyset\}.$$ There is a naive algorithm. Store $K,S$ in a hashtable (using a hash function that maps each ...
D.W.'s user avatar
  • 160k
2 votes

What are the 175 billion parameters used in the GPT-3 language model?

The 175 billion parameters in the GPT-3 language model are values that are used by the model to make predictions about the next word or words in a sentence or piece of text. These parameters are ...
AlekseyHoffman's user avatar

Only top scored, non community-wiki answers of a minimum length are eligible