14
votes
Implementation of Naive Bayes
The usual trick to avoid this underflow is to compute with logarithms, using the identity $$ \log \prod_{i=1}^n p_i = \sum_{i=1}^n \log p_i. $$
That is, instead of using probabilities, you use their ...
- 275k
7
votes
Semantic readings of the Lambek sequent calculus
What constitutes a proof in a system like this is a derivation which is a tree of rule applications. The above translation function is defined by (structural) recursion over that tree. Note, this is ...
6
votes
Accepted
How to express modalities in lambda calculus - are some extensions required?
Are extensions required? Not really. You can take an axiomatic description of a modal logic and simply provide a "primitive" lambda term for each. The modal operators would become type constructors. ...
5
votes
Accepted
nlp: phonetic edit distance between a word and the closest of a set of words
Building on the comments, there's something called the Needleman–Wunsch algorithm, for which you can find the lowest 'cost' alignment of two sequences (phonemes). You can get a good 'confusion cost ...
- 211
5
votes
Accepted
An integer (a string of digits) is $\text{/[0-9][0-9]$\ast$/}$. (Why isn't it just $\text{/[0-9]$\ast$/}$?)
The empty word $\epsilon$ isn't an integer. Integers are $0, 1, 2, 3, \dots, 10, 11, \dots$
Clearly none of them have $0$ letters when you write them, and hence they are not $\epsilon$.
Hence, you ...
- 11.5k
4
votes
How to detect plurals in English sentences?
Morphological parsing requires a lexicon (stems and their part of speech) , morphotactics (ordering of morpheme classes), and orthographic rules (e.g. fox + PL = foxes rather than foxs).
The ...
- 829
4
votes
Examples for CFG that cannot be expressed by regular language
Theoretically, you can nest sentences arbitrarily depth, by using subclauses and this excluded any finite state mechanism. The inventor of the phrase structure grammars himself looked at finite state ...
- 1,439
4
votes
How does SpaCy make its dependency tree?
Matthew Honnibal, who created Spacy, answered me :
The full answer is rather complicated :). I've written a few blog
posts over the years to make it more accessible. The best place to
start is ...
4
votes
How to transform lambda function to multi-argument lambda function and how to rewrite or approximate terms?
You are looking for currying and uncurrying which transform functions of type $A \times B \to C$ to functions of type $A \to (B \to C)$, and vice versa. Currying takes ...
- 30k
3
votes
Advice needed. NLP and ML. Where to start?
Spell checking, grammar checking, proofreading with ML
Norvig: How to Write a Spelling Corrector
"Artistic Style Transfer" for articles (if it is even possible), i.e. "transfer" Shakespeare'...
- 2,350
3
votes
Accepted
Advice needed. NLP and ML. Where to start?
As far as I can see, your reading list lacks specific NLP introductions. A really good starting point is Dan Jurafsky and Chris Manning's Coursera course (for example here https://www.youtube.com/...
- 226
3
votes
What is the Best and easiest way to create a Classifer for Sentiment Analysis
This isn't the best way but it's simple and fast:
preprocess the input with the Porter Stemmming algorithm (it's a process for removing the commoner morphological and inflexional endings from words ...
- 2,007
3
votes
Measuring the information of a document?
Assuming that your data comes from a Markovian source, you can estimate the entropy of the source using an optimal compression algorithm such as Lempel–Ziv, whose theoretical version (without limiting ...
- 275k
3
votes
Specific case of a ranking model
Build a boolean classifier that classifies a message as either "task" or "not a task". Choose a classifier that can output a confidence score (many classifiers can, including ensembles like random ...
D.W.♦
- 156k
3
votes
In NLP, does the lexer have to tag the tokens before the parser?
Morphological tags can help the parser. On the other hand, the complete sentence structure, maybe even the paragraph context may help to finally disambiguate possible tags for a token. So there is no ...
- 245
3
votes
Are there repositories of automatically generated (spam) webpages?
Question: Where can I access automatically generated (spam) webpages?
You could use:
a parody generator program.
They're usually based on Markov chains (e.g. SubredditSimulator) or context-free ...
- 2,007
3
votes
Accepted
Computing probability of sentence using N-grams
The $N$-gram model assumes a generative model in which the next word generated depends only on the preceding $N-1$ words. Using Bayes' law, we get that the probability of a sentence $w_1,\ldots,w_n$ ...
- 275k
3
votes
Examples for CFG that cannot be expressed by regular language
If no bracket language examples suffice, your quest is futile. Every context-free-language is a reduction of a bracket/parenthesis language. Reduction comprises
homomorphism;
intersection with a ...
- 616
3
votes
Accepted
Subsampling of Frequent Words in Word2Vec
My guess is that we should regard $f(w_i)$ as a number between 0 and 1, i.e., the number of times word $w_i$ appears divided by the number of words. This makes it an estimate of the probability of ...
D.W.♦
- 156k
2
votes
How to compute the TF-IDF scores for a handful of documents, but without my own corpus?
Answer
What do you value in a ranking function? If you don't value TF or IDF, then it'll be difficult for you to achieve what you want to achieve.
Without a corpus however, you could use a frequency ...
- 251
2
votes
Relation and difference between information retrieval and information extraction?
From a modeling standpoint, information retrieval is a deep field predicated on several disciplines, including statistics, math, linguistics, artificial intelligence and now data science. In practice, ...
- 21
2
votes
Finding interesting anagrams
It doesn't cover the exact algorithm you had in mind (which Tsuyoshi Ito's answer does), but trying to get at the underlying problem of finding "interesting" anagrams...
My first thought was to use ...
- 236
2
votes
Accepted
Scientific soundness of computer science papers
This is a complex issue that affects all areas of science but has been getting higher visibility as the mainstream media has reported some cases in headlines. An answer seems to be better review ...
- 11k
2
votes
Accepted
Proving that English is not a regular language
It is not easy to prove the complexities of natural languages. See Complexity of natural languages for some hint that English language is not regular.
In fact in Evidence Against the Context-Freeness ...
- 4,807
2
votes
Accepted
How to calculate an accurate estimated reading time of text?
There is a similar question, so I will add a bit over the answer.
There are too many factors independent of language but solely based on individual reading abilities that such estimate would not be ...
- 9,425
2
votes
Applying Machine learning in biological data
A simple starting point would be to apply a bag-of-words feature vector and try using a naive Bayes or logistic regression classifier. You'll probably want to apply stemming and lemmatization and ...
D.W.♦
- 156k
2
votes
Accepted
How to represent text for a program to add punctuation to a block of text?
I am making two assumptions here about things that, to me, are not quite clear from your question
When you say you removed punctuation, you actually meant whitespaces as well.
You tried neural ...
- 226
2
votes
Accepted
Understanding the geometrical interpretation of word2vec
If two words tend to appear in the same context, then they will tend to receive a similar word vector.
For instance, a large corpus might contain sentences like "I really love Fuji apples" and "I ...
D.W.♦
- 156k
2
votes
Accepted
Question on word probability for hierarchical softmax used in natural language processing
I have figured out my confusion.
The indicator function is $\lbrack x \rbrack$ is set to output a $-1$ or a $1$ based on the argument being false or true, respectively.
When the authors say "let $...
- 323
2
votes
How to represent symbolic knowledge using real numbers - theory about neural networks and natural/analog computing?
No, probably not. I think you're expecting too much from the current state of the art in word embeddings. Word embeddings don't magically capture all semantic knowledge. They don't reflect perfect ...
D.W.♦
- 156k
Only top scored, non community-wiki answers of a minimum length are eligible