Using the techniques of artificial intelligence and machine learning to extract patterns from large data sets and transforming those data into a useful, organized form for future processing.

learn more… | top users | synonyms

0
votes
0answers
12 views

k-center algorithm in one-dimensional space

I'm aware of the general k-center approximation algorithm, but my professor (this is a question from a CS class) says that in a one-dimensional space, the problem can be solved (optimal solution ...
0
votes
0answers
8 views

Advantages of OPTICS algorithm over DBSCAN?

In data mining clustering process, for density based clustering we can use DBSCAN and OPTICS algorithms. DBSCAN working with user specified global parameter(neighbor radios and MinEps). But OPTICS is ...
0
votes
0answers
13 views

Is this an accepted/valid clustering evaluation metric?

We have a clustering algorithm where the number of clusters isn't known to the algorithm - it iteratively creates clusters out of similar-looking data points. The evaluation metric we're currently ...
4
votes
1answer
54 views

Extracting maximum information from a set of exam answers and their scores

Imagine we have a multiple-choice exam with N questions. Suppose we have a set of K answer sheets to the exam and their total scores (1 for a correct answer on a question, 0 for incorrect). How much ...
1
vote
1answer
46 views

Application of cosine similarity to detect plagiarism

Can anyone tell me how using cosine similarity to see the correlation between two documents actually shows you if someone is plagiarising the other? I understand how cosine similarity works but don't ...
0
votes
1answer
50 views

Which type of randomized algorithm is best suited for web crawling?

I have decided to implement a web crawler for my CS major project. The project is focused towards adaptive search. I want the pages to be as user specific as possible and time efficiency is not much a ...
-1
votes
0answers
5 views

How do I get a citation dataset with full text articles in PDF format? [migrated]

i am doing research in text mining. For that i need a dataset which contain full text pdfs of research paper articles, and all articles should be related with each other in terms of citing. There are ...
0
votes
0answers
19 views

Is k-means parallelizable (other than the data parallelism of the distance function computation)

The k-means algorithm is known to be NP-hard. While the distance function computation in the algorithm loop is data parallel, the algorithm is iterative and may become exponential in the number of ...
0
votes
1answer
49 views

Automatically generate meaningful queries for a data table

My field of research is not Database or AI. But I have some problems to solve, and would like to know which branch this kind of problems belong to, and what are the results. The main question is: ...
0
votes
0answers
49 views

Activity prediction in a kitchen

Here is the scenario: There are three chefs(A- main chef, B and C- assistant) working together to prepare a diner set. The sequence of the event is as following. Start: The three chefs enter the ...
-1
votes
1answer
52 views

difference between multilayer perceptron and linear regression

What is the difference between multilayer perceptron and linear regression classifier. I am trying to learn a model with numerical attributes, and predict a numerical value. Thanks
1
vote
0answers
17 views

Baseline approaches for likes prediction

I have a small user-item matrix (25k x 1.8k) describing how users liked or disliked some of the items. Users don't have any attributes but items have several features. I would like to be able to ...
-1
votes
1answer
16 views

Training Error & Convergence to True Error

I Take some online class for Machine Learning. one of teacher say this sentence. if we have m data points, the training error converges to the true error as m → ∞. i thought, this sentence not ...
0
votes
1answer
34 views

VC Dimension Calculation for Intervals

As i See in ML Course a VC dimension calculation is very theoretical. What is the VC-dimension of intervals in R? The target function is specifieed by an interval, and labels any example positive ...
-1
votes
1answer
40 views

Policy function π in Reinforcement learning unclear

I have one question about policy function in Reinforcement learning. in fact this function indicates which action should be done in each state? Or this function indicate for get the ...
-1
votes
1answer
41 views

Selecting a M.S. in CS focus [closed]

I need to pick a focus in my Masters program. The running candidates are Networking Bioinformatics Data mining Computer Image Recognition In order to avoid this becoming a discussion, I'd be ...
5
votes
3answers
448 views

Word Frequency with Ordering in O(n) Complexity

During an interview for a Java developer position, I was asked the following: Write a function that takes two params: a String representing a text document and an integer providing the ...
0
votes
2answers
27 views

Determine Epsilon for identification [closed]

I have a project in which I need to compare different distances in a database with a distance in input in order to identify a person. For that I use this expression: DistanceDB - DistanceInput < ...
1
vote
2answers
49 views

Need an algorithm to find the input factors that are most affecting the output

I apologize if this question is already answered and appreciate any pointers to existing answers. I'm not familiar with statistical or data mining terms so my search was limited to basic words used in ...
1
vote
1answer
77 views

Best algorithm for correlation between time series?

I have some biological data (ECG), which are quite chaotic in nature, and and some other data; that are not chaotic but related in some way, like fatigue. I want to find out how the time series, ...
3
votes
0answers
38 views

What are some of the methods that NLP practitioners use to automatically learn linguistic features from text? [closed]

I am learning about NLP, with an eye to starting some practical NLP projects. I see that many of the algorithms for relation extraction and named entity recognition require you to identify linguistic ...
2
votes
1answer
24 views

Subspace clustering with random transformation

One approach for clustering a high dimensional dataset is to use linear transformation, and the most common approaches are PCA and random projection (where random projection arises from the ...
3
votes
1answer
78 views

Why linear transformation can improve classification accuracy when the dimensionality of data is high?

Let $X$ be an $m\times n$ ($m$: number of records, and $n$: number of attributes) dataset. When the number of attributes $n$ is large and the dataset $X$ is noisy, classification gets more ...
3
votes
0answers
34 views

Document clustering for summarization

I am curious as to what steps one would reasonably need to take to perform an extraction-based text summarizer. I've taken a look at some papers I've found on Google such as this one, which explains ...
2
votes
2answers
57 views

What kind of model is used by 20 Questions?

Which kind of machine learning concept / model is used in 20 Questions? Is this kind of thing best solved by a neural network? Where I can read something about it?
3
votes
2answers
58 views

Physical Meaning Behind Matrix Factorization

As we all know, Matrix Factorization is an effective method to do rating prediction jobs in recommender systems. Thanks to the work of Yahuda Koren. My question is why MF can do this job? What's the ...
0
votes
2answers
52 views

web content “mining” using supervised learning tequniques

We're accessing an API of a web system for obtaining product information. We require some additional information, which is not available through the API. This information is publically available for ...
3
votes
1answer
44 views

Decision Tree with Unbalanced Data

I have a data set with two classes: one class has at most 2000 members while the size of the second class is unlimited, though it is typically in the hundreds of thousands. I have read that it is ...
1
vote
1answer
24 views

Model for diurnal nature of data [closed]

I have a timeseries dataset of a quantity measured over the period of a week. I want to verify if the data is varying in a diurnal fashion with the help of some mathematical measure. Does any such ...
3
votes
1answer
101 views

Most common subset of size $k$

I'm trying to write an algorithm that detects the most common subset of at least size $k$, from a collection of sets. If there are ties for the most common subset, I want the one of them whose size ...
1
vote
1answer
75 views

Is there an efficient way to cluster a graph according to Jaccard similarity?

Is there an efficient way to cluster nodes in a graph using Jaccard similarity such that each cluster has at least k nodes? Jaccard similarity between nodes i and j: Let S be the set of neighbours ...
1
vote
1answer
207 views

How does BIRCH (Balanced Iterative Reduction Using hierarchical) CF Tree insertion algorithm work?

I am studying Cluster Analysis and I came across BIRCH (Balanced Iterative Reduction Using hierarchical) http://en.wikipedia.org/wiki/BIRCH_(data_clustering) CF Tree insertion algorithm. I am confused ...
3
votes
1answer
54 views

Analysis and classification based on data points

I'm not sure if this is the correct stack exchange or correct tags, but my question is as follows: I am working on a sort-of ratings system for players in a particular game. After allowing the ...
0
votes
3answers
72 views

linklist and memory issues [closed]

I have a doubly link-list like this typedef struct Record { int i; Record* next; Record* prev; }Record; I have over 5 trillions of records that I need ...
3
votes
1answer
262 views

String inputs in Machine Learning

Several popular machine learning algorithms such as Logistic regression or Neural networks require its inputs to be numeric. What I'm interested in is how you make these algorithms work on ...
2
votes
1answer
68 views

Classification of 2d arrays of outlines

I am not experienced in machine learning but I have been looking at the Python toolklit scikit-learn. I have thousands of mammograms and I have written an algorithm ...
2
votes
1answer
94 views

Document definition in information retrieval

Is there common formal definition of Document in information retrieval field? In many researches authors don't define the term Document(maybe because it is evident for them). Wikipedia says "text ...
1
vote
2answers
148 views

Getting started with Data Mining [closed]

This is the first time that I'm looking in depth into the topic, although I've always been curious. Could someone let me know about online resources (courses, tutorials, etc) and books that cover the ...
4
votes
1answer
130 views

Anomaly/outlier detection using fuzzy clustering

I understand that fuzzy clustering using FCM produces a membership matrix for the set of data points we feed to it. What characteristics will an anomalous cluster produced during this method have? ...
1
vote
2answers
91 views

Event Prediction through Machine Learning

I have a large data set consisting of ca. 40 categorical data items and a few interval data items (real numbers, less than 5 such items). Most categories should have a lot of values that repeat ...
1
vote
0answers
41 views

How to cluster nodes based on the number of dependencies

I have a problem where, there are a set of nodes and dependencies between them. I want to cluster them based on the maximum number of dependencies. Dependencies can be thought of as number of edges ...
3
votes
2answers
172 views

Can we understand SVM without knowledge of Machine Learning [closed]

I was told by my adviser (future one) to look into libsvm library or any other and try to get familiar with it.. to work on a programming project (on Machine Learning) (will start in a month). my ...
1
vote
0answers
32 views

Is it possible to discern the quality of web data from networks derived from it?

One topic I've recently looked at is co-occurence networks formed from Twitter tweets. This is how I felt after looking at the tweets of random people: This leads me to the question: Question: ...
1
vote
2answers
49 views

Looking for Rating Functions

I'm looking for something i would call rating functions. I'm searching for some literature about this concept. I'm not really sure about the terminology, but what I mean should be pretty obvious. A ...
1
vote
1answer
81 views

Maximum variance and useful information of dataset

I am reading through PCA and it says that the maximum variance principal component has most of the information. Can we apply that to any data set? If a data set has n attributes and most of the ...
1
vote
1answer
194 views

Difference between Data point, attribute, feature?

Can anyone tell me the difference between the following in data mining? I am taking a class this semester and the professor is using the terms so frequently, I don't know what these mean anymore. ...
1
vote
1answer
51 views

Pool of photos for term extraction

I need a pool of photos (if possible with description) for my project. I mainly have to perform term extraction for semantic searching. Is there something available out there that is made available ...
1
vote
1answer
74 views

In context of data mining, what does it mean for an association rule measure to be maximal?

In context of data mining, what does it mean for an association rule measure to be maximal? I cannot understand the term maximal in this context. I know of maximal independent sets in algorithms ...
2
votes
3answers
2k views

Relation and difference between information retrieval and information extraction?

From Wikipedia Information retrieval is the activity of obtaining information resources relevant to an information need from a collection of information resources. Searches can be based on ...
2
votes
0answers
100 views

What limits the implementation of proximity operators for text indexing and searching?

I was asking on academia.se, if anyone knows scientific search engines offering a proximity operator like Google Web Search does, while Google Scholar Search does not. That's sad, because this ...