Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [natural-language-processing]

Natural language processing (NLP)

0
votes
0answers
23 views

Find pattern in string of unkown language

Hello I am looking for an algorithm which solves following problem for me. Given a text/string = 'BIRDBIRDBIRDBIRDDEVILDEVILDEVILDEVILEASYEASYEASY ...' of an unkown language. For the sake of ...
2
votes
0answers
6 views

Are there any neural NLG systems which don't generate in left-to-right order?

For a while, all classification tasks in natural language processing were based on simple RNN's, which operate in a very word-by-word order. Adding gating mechanisms increased ability to "look back", ...
1
vote
0answers
18 views

Generalization of formal grammars - production rules with more general functions?

Usually formal grammars have production rules in the format N=tNt where simple concatenation function is used for the expansion of the nonterminal. https://www....
0
votes
0answers
36 views

How to prove that katz backing off smoothing technique is a valid probability distribution?

How to prove that katz backing off smoothing technique is a valid probability distribution? Take the example of bigram. You can go through this link for katz back off model https://en.wikipedia.org/...
1
vote
1answer
46 views

Show that the Laplace smoothing for bigrams is a valid probability distribution

If we consider any smoothing technique like laplace or delta smoothing. Intuitively we can see that the we are stealing from sequences with non zero probablity and re distribute to sequences with zero ...
0
votes
0answers
13 views

Semantic Question Answering system vs Cross Language Information Retrieval

I've been trying to understand Bilingual Question Answering (QA) systems and Cross-Language Information retrieval systems (CLIR). I understand that QA systems give a precise response instead of a set ...
2
votes
1answer
85 views

How to represent symbolic knowledge using real numbers - theory about neural networks and natural/analog computing?

One can define the semantics of one definite word using the references to real world entities, relationships with the other words and other concepts and represent all this knowledge about this one ...
0
votes
0answers
28 views

FastText how it works?

So my question is how Supervised FastText works for the most part. I understood in the original paper they use bag of n-grams for features, but then they released a paper with enriching the word ...
4
votes
0answers
18 views

How to represent sentences with their dependency parses as input to an RNN?

I am working on a task embedding sentences into a lower-dimensional space according to style, both grammatical and lexical. As such, I want to have as input the linear ordering of tokens in each ...
0
votes
0answers
18 views

How to solve question answer questions using trees and graph theory?

Good evening! I'm doing an internship at Ecole Normale Supérieure of France, where they create their scholars, and I encountered a dataset I wanted to solve using graphs... I. The challenge ? ...
3
votes
0answers
26 views

How to compute the loss and backprop of word2vec skip-gram using hierarchical softmax?

So we are calculating the loss $$J(\theta) = -\frac{1}{T}\sum_{t=1}^T\sum_{-m \leq j \leq m} \log P(w_{t+j}|w_t;\theta)$$ and to do this we need to calculate $$P(o|c) = \frac{\exp(u_o^Tv_c)}{\sum \...
1
vote
1answer
82 views

How to transform lambda function to multi-argument lambda function and how to rewrite or approximate terms?

I am trying to do the formal semantics (Montague grammar, abstract categorial grammar) of natural language and encode the sentence John is boss. The type system has ...
1
vote
1answer
67 views

How does SpaCy make its dependency tree?

I discovered that SpaCy had the ability to make dependency trees. For instance given a question “To whom did the Virgin Mary allegedly appear in 1858 in Lourdes France?” We can create its dependency ...
1
vote
0answers
80 views

How can node2vec help find similar “roles” within a graph (nodes whose connections have similar structure within the graph)?

I have a question on the node2vec algorithm described in this paper. Node2vec is a deep learning algorithm that word2vec to graphs to learn embeddings. The authors claim that it can help find nodes ...
2
votes
1answer
21 views

Marginalise edge weights on graph

I have a directed acyclic graph with a score on each edge. The score of a path is defined to be the sum of the scores on the edges along this path. The probability of a path is the score of such a ...
2
votes
1answer
196 views

Subsampling of Frequent Words in Word2Vec

I am reading through the following paper: https://papers.nips.cc/paper/5021-distributed-representations-of-words-and-phrases-and-their-compositionality.pdf Under section 2.3 on page 4 the authors ...
1
vote
1answer
35 views

Question on word probability for hierarchical softmax used in natural language processing

I am reading the following paper: https://papers.nips.cc/paper/5021-distributed-representations-of-words-and-phrases-and-their-compositionality.pdf On page 4 of the paper they describe the ...
0
votes
1answer
25 views

Naive Bayes' Classification and Using the Entire Vocabulary in the Denominator

I am working through the NLP notes for Naive Bayes' classification here: https://web.stanford.edu/~jurafsky/slp3/6.pdf Below $c$ is the class of the observation and $w_i$ is the $i$th word of a text ...
2
votes
0answers
15 views

How does the Earley parser accept the input string as syntactically correct?

https://en.wikipedia.org/wiki/Earley_parser An input string of length $n$ is syntactically correct if at least one $(S \rightarrow X_1 ... X_m • 0)$ is in $E_n$. Why is this the case ? The part ...
2
votes
0answers
37 views

Proving the probability of zero occurrences in training using Good-Turing maximum likelihood estimate

Background Good-Turing (GT) smoothing is used in language models to estimate the counts of words in the test set that have not been seen in the training set. In GT smoothing, $N_c$ is the count of ...
0
votes
0answers
25 views

Any references/current work on anomaly detection in conversations?

Suppose I have a lot of data on conversations between humans and chatbots (human text, chatbot text, times, media used for chat, etc), and I want to be able to detect anomalies in these conversations. ...
2
votes
0answers
50 views

Automatic learning/discovery of logics

Are there efforts to automatically discover new logics? Logics are simple structures - they have formal language, deduction rules, semantics and certain properties that are proved or discarded for ...
3
votes
0answers
73 views

Semantic parsing with Grammatical Framework - is this possible?

So far I have learned about categorial grammars, type logical grammars and formal semantics of natural language, the relevant tools are Cornell Semantic Parsing Framework https://github.com/clic-lab/...
1
vote
1answer
30 views

How to figure out whether two texts refer to the same object or event

Let's assume there is something happen in the world - Football world cup final. And team-1 beat team-2 with the score 3:2. So there is whole bunch of articles on every website about it, each contains ...
0
votes
1answer
99 views

Converting CFG to CNF [closed]

need some help with the following question: I've watched a few youtube tutorials but I'm struggling to convert this specific CFG
3
votes
0answers
67 views

How to translate lambda calculus into (first-order, modal) logic, is it possible at all?

It is possible (using formal semantics) to translate natural language sentences into lambda expressions. So, is it possible to translate those lambda expressions into some logic, e.g. into first-order ...
2
votes
0answers
41 views

Semantic/DRT methods for conversational agents / chatbots / dialogue systems - reference request?

The wiki pages about chabots mention that statistical methods, keyword search and precompiled answers are used for the chatbots. But I feel that there should exist different - semantical approach for ...
5
votes
1answer
333 views

nlp: phonetic edit distance between a word and the closest of a set of words

Let's say someone is using Dragon Dictation, Google Speech, or some other free form dictation software (it will recognize anything they say to the best of its ability). I have some reasonably large ...
1
vote
1answer
45 views

Understanding the geometrical interpretation of word2vec

I'm trying to understand how the $word2vec$ method actually nudges word vectors of similar semantic/syntactic content closer together in the word vector space. I've read here (Quora answer) that it's ...
1
vote
0answers
62 views

HMM Baum-Welch training and pruning

I am working on a HMM tagger that should be initialized with some small data and then supposedly improved with Baum-Welch algorithm on the data. However, the number of states is huge, almost $459^2$, ...
3
votes
1answer
24 views

HMM tagger - Baum Welch training

I am trying to implement a trigram HMM tagger for a language that has over 1000 tags. In my training data I have 459 tags. Now if we consider that states of the HMM are all possible bigrams of tags, ...
1
vote
1answer
22 views

How to use a logistic regression classifier to estimate the confidence of a rule?

The following is an excerpt from the Estimating LAT Confidence section of this paper: ... Since some LAT detection rules are more reliable than others, we would like to have a confidence value in ...
1
vote
1answer
28 views

Using Graphs to construct semi-supervised learning solution

I am currently trying to design a graph-based approach for content tagging. In principle I am trying to two two things: Use an algorithm such as Latent Dirichlet Allocation (LDA) or some variant of ...
0
votes
0answers
325 views

How much training data for speech recognition?

How much training data is needed to build a speech-to-text engine based on machine learning? (To within an order of magnitude or so.) Big companies like Google, Facebook have a massive amount of data....
4
votes
1answer
69 views

How to express modalities in lambda calculus - are some extensions required?

Lambda calculus can be used for encoding semantics of natural language, e.g. http://yoavartzi.com/tutorial/ contains full details about semantic parsing of natural language: converting natural ...
3
votes
1answer
228 views

Semantic readings of the Lambek sequent calculus

I am reading Categorial Grammar: Logical Syntax, Semantics, and Processing by Glyn Morrill and I am stuck with the Fig. 3.9: Can someone explain this set of formulas and |.| function specifically? ...
1
vote
0answers
39 views

Training algorithm for Manual error correction in text

I want to work on a model where I make some manual corrections in my clinical notes data and want the neural network to learn those corrections.The algorithm will be supplied with both corrected and ...
1
vote
0answers
70 views

Which Machine Learning algorithm should I use for gender tagging of words in NLP? [closed]

I have a dataset of about 26,000 words along with their gender tags [m, f or any]. Which ML algorithm should I use for Gender Tagging/Classification purpose. How should I go about it? What should be ...
1
vote
0answers
36 views

Outline detection from patterns in a list of textual articles

Are there NLP algorithms dealing with detecting the repeating patterns in a a list of texts from which a topic keywords and other associative keywords can be derived? I will show it as an example: ...
1
vote
1answer
181 views

TF IDF Spark values are small after cosine similarity

I am creating a software that computes the cosine similarity between 2 strings. I am using Spark as I have to use Java. I created a Bag of words model and computed the cosine similarity successfully. ...
4
votes
2answers
449 views

Examples for CFG that cannot be expressed by regular language

There are nice examples for context free grammars which cannot be expressed with regular language, for example the palindrome and a similar contrived example here, but they are very intuitively ...
0
votes
0answers
41 views

Speech to Text API with Unknown Language

I would like to develop an application that outputs text from the audio of 2 possible languages. For ex: Audio might be English or Spanish, it should return the spoken words. Most speech to text ...
1
vote
1answer
212 views

Is there an Algorithm that can code itself?

Suppose there is algorithm that has a strong NLP module. Could it use the NLP module to read text books such as http://www.amazon.in/Artificial-Intelligence-Modern-Approach-Prentice/dp/0136042597 in ...
0
votes
0answers
73 views

Combining Ontology and Relational Databases in Question Answering system

I'm introducing to the Natural Language Processing field and it's application. I'm planning to build a question answering system for a project, but some approaches are making me a bit confuse about ...
0
votes
0answers
50 views

Gensim LDA all topics zero

I'm trying to run Gensim's LdaModel on a corpus. I load the corpus myself from two XML files. I made a class to iterate over the text pieces in the XML files. It ...
2
votes
0answers
21 views

Where to cut a category tree

Since I don't have CS background I will most probably ask this question the wrong way. I need to choose a node from a tree, where I include all beneath this node leafs in a validation. I have a data ...
3
votes
0answers
364 views

What are the inputs to an LSTM for Slot Filling Task

I am confused on the inputs of a Long-Short Term Memory (LSTM) for the slot filling task in Spoken Language Understanding. Before I worked on this, I implemented a language model with a Recurrent ...
0
votes
0answers
41 views

What is a Neural attention model and how can it be used for text summarization?

We are doing a research on multi-document (one document max 100 words) text summarization. We are looking into abstractive text summarization methods. I need following things to be clarified. Please ...
5
votes
2answers
500 views

How to represent text for a program to add punctuation to a block of text?

I want to try and make a program where I remove the punctuation from a block of text and it then inputs it in the correct places. I have started with a simpler element of just using text and spaces, ...
2
votes
1answer
518 views

Semantic natural language processing - from texts to logical expressions? Universal knowledge base?

My question is - is there a semantic natural language processing that tries to understand the meaning of the texts and that tries to derive the consequences of the understood meaning? Is there a ...