Podcast #128: We chat with Kent C Dodds about why he loves React and discuss what life was like in the dark days before Git. Listen now.

Questions tagged [natural-language-processing]

Natural language processing (NLP)

Filter by
Sorted by
Tagged with
1
vote
1answer
29 views

Using Graphs to construct semi-supervised learning solution

I am currently trying to design a graph-based approach for content tagging. In principle I am trying to two two things: Use an algorithm such as Latent Dirichlet Allocation (LDA) or some variant of ...
27
votes
9answers
3k views

Are programming languages becoming more like natural languages?

Can we study programming languages in the context of linguistics? Do programming languages evolve naturally in similar ways to natural languages? Although full rationality, and mathematical ...
1
vote
0answers
400 views

How much training data for speech recognition?

How much training data is needed to build a speech-to-text engine based on machine learning? (To within an order of magnitude or so.) Big companies like Google, Facebook have a massive amount of data....
4
votes
1answer
91 views

How to express modalities in lambda calculus - are some extensions required?

Lambda calculus can be used for encoding semantics of natural language, e.g. http://yoavartzi.com/tutorial/ contains full details about semantic parsing of natural language: converting natural ...
3
votes
1answer
248 views

Semantic readings of the Lambek sequent calculus

I am reading Categorial Grammar: Logical Syntax, Semantics, and Processing by Glyn Morrill and I am stuck with the Fig. 3.9: Can someone explain this set of formulas and |.| function specifically? ...
1
vote
0answers
48 views

Training algorithm for Manual error correction in text

I want to work on a model where I make some manual corrections in my clinical notes data and want the neural network to learn those corrections.The algorithm will be supplied with both corrected and ...
7
votes
3answers
2k views

Complexity of natural language processing problems [closed]

Which natural language processing problems are NP-Complete or NP-Hard? I've searched the natural-lang-processing and complexity-theory tags (and related complexity tags), but have not turned up any ...
1
vote
0answers
78 views

Which Machine Learning algorithm should I use for gender tagging of words in NLP? [closed]

I have a dataset of about 26,000 words along with their gender tags [m, f or any]. Which ML algorithm should I use for Gender Tagging/Classification purpose. How should I go about it? What should be ...
1
vote
0answers
36 views

Outline detection from patterns in a list of textual articles

Are there NLP algorithms dealing with detecting the repeating patterns in a a list of texts from which a topic keywords and other associative keywords can be derived? I will show it as an example: ...
1
vote
1answer
216 views

Is there an Algorithm that can code itself?

Suppose there is algorithm that has a strong NLP module. Could it use the NLP module to read text books such as http://www.amazon.in/Artificial-Intelligence-Modern-Approach-Prentice/dp/0136042597 in ...
1
vote
1answer
234 views

TF IDF Spark values are small after cosine similarity

I am creating a software that computes the cosine similarity between 2 strings. I am using Spark as I have to use Java. I created a Bag of words model and computed the cosine similarity successfully. ...
3
votes
2answers
674 views

Examples for CFG that cannot be expressed by regular language

There are nice examples for context free grammars which cannot be expressed with regular language, for example the palindrome and a similar contrived example here, but they are very intuitively ...
0
votes
0answers
54 views

Speech to Text API with Unknown Language

I would like to develop an application that outputs text from the audio of 2 possible languages. For ex: Audio might be English or Spanish, it should return the spoken words. Most speech to text ...
0
votes
0answers
92 views

Combining Ontology and Relational Databases in Question Answering system

I'm introducing to the Natural Language Processing field and it's application. I'm planning to build a question answering system for a project, but some approaches are making me a bit confuse about ...
0
votes
0answers
63 views

Gensim LDA all topics zero

I'm trying to run Gensim's LdaModel on a corpus. I load the corpus myself from two XML files. I made a class to iterate over the text pieces in the XML files. It ...
2
votes
0answers
22 views

Where to cut a category tree

Since I don't have CS background I will most probably ask this question the wrong way. I need to choose a node from a tree, where I include all beneath this node leafs in a validation. I have a data ...
2
votes
0answers
502 views

What are the inputs to an LSTM for Slot Filling Task

I am confused on the inputs of a Long-Short Term Memory (LSTM) for the slot filling task in Spoken Language Understanding. Before I worked on this, I implemented a language model with a Recurrent ...
31
votes
5answers
3k views

Finding interesting anagrams

Say that $a_1a_2\ldots a_n$ and $b_1b_2\ldots b_n$ are two strings of the same length. An anagramming of two strings is a bijective mapping $p:[1\ldots n]\to[1\ldots n]$ such that $a_i = b_{p(i)}$ ...
0
votes
0answers
45 views

What is a Neural attention model and how can it be used for text summarization?

We are doing a research on multi-document (one document max 100 words) text summarization. We are looking into abstractive text summarization methods. I need following things to be clarified. Please ...
5
votes
2answers
685 views

How to represent text for a program to add punctuation to a block of text?

I want to try and make a program where I remove the punctuation from a block of text and it then inputs it in the correct places. I have started with a simpler element of just using text and spaces, ...
2
votes
2answers
3k views

Detect if a sentence is in passive voice

Given an English sentence, I am looking for a programmatic way to tell whether the sentence is written in passive voice. Currently, I just check if there is a was ...
0
votes
0answers
1k views

Difference between LR parsing and Shift-Reduce parsing?

I'm learning natural language processing and I can't understand the difference between Shift-Reduce parser and LR parser. As I've understood from Wikipedia, shift-reduce is just a name of a class of ...
1
vote
0answers
36 views

How to store topic-to-topic relationships efficiently and accurately with two huge tables?

I am scanning Wikipedia's DB and retrieving topics (page names, mostly) and their relationships. I want to use it later to create a visual map. A topic table stores the id and the label of each topic....
1
vote
1answer
45 views

Why use maximum likelihood for word prediction?

According to https://www.tensorflow.org/tutorials/word2vec/, the standard approach for predicting the next word in a word sequence is maximum likelihood. The predicted next word is the word that ...
2
votes
0answers
48 views

Phrase generation approaches

In generating reports, sometimes there is a need to produce quite involved phrases in one of natural languages given numerical or boolean parameters. To get a feel of it, it is enough to take a look ...
1
vote
0answers
26 views

Is there a publicly available informer tagger or dataset?

I am working on a question answering system. I've learned that informer spans are valuable features for question classification. However from what I've read I wasn't able to find any publicly ...
4
votes
2answers
113 views

Advice needed. NLP and ML. Where to start? [closed]

Hi fellow computer scientists, I just began my journey to the world of ML and NLP so please bear with me. I'm hoping to find some guidance here. I would be very grateful if anyone could point me in ...
0
votes
0answers
71 views

API to retrieve answers for general questions

I was looking for a service (with API) where I can ask it a general question (aka, when was Einstein born?) and retrieve an answer from the Web. Is there any available service to do that? Have tried ...
0
votes
1answer
67 views

Is it possible to get information from websites as a human? [closed]

If I want to read any websites and get some information as names (or dates) on some part of the site as a human. What solution I could use in order to get these datas using deep learning or machine ...
1
vote
0answers
43 views

Which fields of Computer Science are involved in knowledge-based and text-based dialog systems? [closed]

One of my future goals is to know in depth about building a text-based dialog system to answer questions about a specific topic (say, Tolkien's legendarium), assume that I have a large body of article-...
1
vote
1answer
55 views

Applying Machine learning in biological data

I am trying to solve the following question: Given a text file containing a bunch of biological information, find out the one gene which is {up/down}regulated. Now, for this I have many such (60K) ...
1
vote
0answers
13 views

Evaluation of annotation

I'm doing a research on NLG systems. I need to annotate my corpus (~6 million words) automatically. My algorithm works well and I want to calculate Cohen's Kappa. What I cannot understand is the ...
8
votes
2answers
132 views

What are some efficient ways to find the differences between two large corpuses of text that have similar, but differently ordered content?

I have two large files containing paragraphs of English text: The first text is about 200 pages long and has about 10 paragraphs per page (each paragraph is 5 sentences long). The second text ...
1
vote
0answers
181 views

How the conditional random fields algorithm work? [closed]

Im searching about conditional random fields for a long time and cant understand the concepts of this algorithm. I understood markov chains, and understood that conditional random field is a more ...
4
votes
1answer
80 views

Are there repositories of automatically generated (spam) webpages?

I'm interested to see if I can use machine learning/network analysis methods to automatically detect automatically generated (spam) webpages. I'm particularly interested in webpages that look ...
4
votes
1answer
417 views

How to compute the TF-IDF scores for a handful of documents, but without my own corpus?

I have a small number of documents (I could probably get 4 or 5 documents) and want to assign terms in these documents a score based on its importance to the document. I want to find which terms are ...
3
votes
1answer
221 views

Specific case of a ranking model

So the problem I'm solving is this: I have a list of conversations of 3 messages each (for eg. "hi", " how are you", "remind me to fix this bug" is one conversation, and my problem will have many of ...
0
votes
0answers
38 views

How can I change the text style?

There are algorithms to change the forms of words, to define the subject of the text, as well as the definition of his style (literary, business, scientific, etc.). Do you have any ready ideas or ...
2
votes
1answer
60 views

How do IR researchers evaluate the ranks of documents?

I am developing a new IR system in a specialized context. I understand that a traditional IR system (like a search engine) should rank documents in terms of their relevance for a query. The most ...
3
votes
1answer
4k views

How to calculate an accurate estimated reading time of text?

I suppose the calculation should not be done by only two factors (average reading speed/words per minute, and word count). But at least by a third parameter, that in my opinion should measure the ...
2
votes
1answer
67 views

What is the Best and easiest way to create a Classifer for Sentiment Analysis [closed]

Sentiment analysis using Machine Learning is a hot topic. In the present situation when a person doesn't have a problem in having the training data set then which way should we create the classifier ...
2
votes
0answers
63 views

How to determine agreement between two sentences?

A common Natural Language Processing (NLP) task is to determine semantic similarity between two sentences. Has the question of agreement/disagreement between two sentences been covered in NLP or other ...
2
votes
1answer
420 views

Language grammar correction with supervised learning

I want to work on automatic grammar correction using machine learning (possibly using recurrent or deep neural networks). The algorithm will be supplied with both corrected and initial documents for ...
0
votes
0answers
94 views

Examples for speech recognition systems and spoken dialogue systems

I am collecting material for a MOOC about speech technology. My aim is that students also have examples to try rather than just watching the lecture and some complimentary youtube videos. So the idea ...
10
votes
1answer
583 views

Implementation of Naive Bayes

I am implementing a Naive Bayes algorithm for text categorization with Laplacian smoothing. The problem I am having is that the probability approaches zero because I am multiplying many small ...
3
votes
0answers
187 views

What is the state-of-the-art in machine translation on Chinese/CJK? [closed]

Sorry for my unfamiliarity with the field. Hopefully this question is on-topic here. Currently the machine translation from western European languages to English is arguably quite robust, with Google ...
3
votes
1answer
2k views

Proving that English is not a regular language

I am wanting to try and prove that the English language is not regular. The alphabet is the set of all words in the English dictionary. Looking at sentences, I was able to use this pattern of ...
11
votes
4answers
17k views

Relation and difference between information retrieval and information extraction?

From Wikipedia Information retrieval is the activity of obtaining information resources relevant to an information need from a collection of information resources. Searches can be based on ...
1
vote
1answer
83 views

Question about bigram model

I am trying to build a bigram letter model. I obtain a sequence of words in a form of ['hello','I','am','Johnny']. Firstly, I lower all the words to obtain : ['hello','i','am','johnny']. I am ...
1
vote
1answer
80 views

What's this type of semantic analysis called in NLP?

I'm speculating that this is very much known in NLP, but as I've not studied NLP, I don't know what concepts are related to this: Basically I have entities that are connected by some strings. I want ...