Skip to main content

Questions tagged [data-mining]

Using the techniques of artificial intelligence and machine learning to extract patterns from large data sets and transforming those data into a useful, organized form for future processing.

Filter by
Sorted by
Tagged with
0 votes
0 answers
21 views

Restaurant Galaxy schema

I want to make a Galaxy schema of a Restaurant. There are 2 fact tables sales and purchases. sales are related to customer and purchases to supplier of ingredients. Now my question is how can i make ...
MUHMMAD MURTAZA's user avatar
1 vote
1 answer
101 views

What books are there to learn to implement these graph algorithms?

I saw a post on Reddit (https://www.reddit.com/r/math/comments/ci50d3/visualizing_mathematical_subjects/) that utilizes label propagation, Fruchterman-Reingold algorithm, and edge betweenness ...
ZENG's user avatar
  • 113
0 votes
0 answers
40 views

Persona matching algorithm

I have a project to match two groups of people. Under insurance, if the initial sales agent leaves the insurer, their customers will become so-called “orphan customers”. I've given a big data set ...
Yumi's user avatar
  • 1
1 vote
1 answer
45 views

Machine learning and test split for time series data

I have used different machine learning algorithms to predict solar panels' power output. There are ten independent features for weather data. In all models, I set time as an index and have used the ...
graphicart86's user avatar
0 votes
0 answers
123 views

Efficient way to do self join with minimum support?

In the apriori algorithm there's the self join step, So, say we have 1 3 2 3 3 5 2 5 If I were to do an exhaustive join I'd end up with a tuple including (1, 3, ...
Cookie's user avatar
  • 101
1 vote
3 answers
2k views

How can we express value of cosine similarity of two documents into percentage?

We were doing project work for plagiarism checking. For this purpose, we have taken a term frequency vector of two documents and measured the similarity using a cosine similarity measure. The value of ...
Tushar Saha's user avatar
0 votes
0 answers
104 views

Difference between C-Index and Spearman correlation

Suppose I have a list that reflects the priority of web pages for recrawling: l1 = [3, 2, 1, 4, 2, 5] Now, I have tried to estimate the priorities with two ...
Ashkan Khosravani's user avatar
1 vote
1 answer
25 views

Would samples be considered data redundancy if they are similar to each other fairly naturally?

I am working on building ML/DL solution for a problem where that data is considered, naturally similar and I am worried if that would be considered as data redundancy. My question is, is that so? and ...
Luka's user avatar
  • 11
1 vote
0 answers
48 views

What do we mean by permissible transformations in types of attributes-:nominal,ordinal,interval,ratio? [closed]

I am studying data mining and I stumbled upon types of attributes. They are Nominal Ordinal Interval Ratio Data mining book by Tan,Steinbech,Kumar says Permissible transformations for-: nominal-:...
Team B.I's user avatar
  • 263
0 votes
0 answers
42 views

What are the confusion matrix values?

I'm currently going through past paper questions and was wondering if I could get some help answering this one? 'Consider a classification model which is applied to a set of records, of which 100 ...
curiousCoder's user avatar
1 vote
0 answers
13 views

About the paper Privacy-preserving in association rule mining using an improved discrete binary artificial bee colony

I don't understand two parts in this paper: The min notion on page 4 line 357 (equation 10d): I understand this as to find all the $M_{10}$, $M_{11}$, $M_{01}$ first and then try to minimize the ...
nguyenhienph's user avatar
0 votes
1 answer
142 views

How to detect outliers using DBSCAN?

I am working on a Fraudulent Cash Transaction Detection System using DBSCAN and I want to know what is the proper way to identify outliers? Thank you ##Edite## I had a problem how to represent the ...
Xx_22's user avatar
  • 1
1 vote
1 answer
26 views

In a set of sentences, how could I determine the fewest sentences that contains all characters?

So, for the sake of simplicity, I am going to use English characters for this example. Let's say I have a set of strings of characters in English ranked by difficulty: Easy, Intermediate, Advanced. So ...
Lathan's user avatar
  • 23
0 votes
0 answers
35 views

How to handle distribution of values with same attributes into different classes

I'm a student studying a data mining course and have come across a problem. I need to explain the problem with the help of an example scenario as I do not know how to explain the problem in any other ...
mahesh Rao's user avatar
0 votes
1 answer
160 views

PageRank vector

I computed the PageRank vector for the example given in https://en.wikipedia.org/wiki/PageRank (where the picture shows that node B ends up with a score of 38.4, node C with 34.3, node D with 3.9). I ...
jm jm's user avatar
  • 21
2 votes
1 answer
175 views

How does an inverted index reduce storage requirements?

In p. 7 of the book "Introduction to Information Retrieval" (by Manning et al), the authors explain how, given a collection of text documents, an inverted index is built by tokenizing, then ...
jm jm's user avatar
  • 21
0 votes
0 answers
26 views

ML - What are the benefits of transforming variables with long tail?

What are the obvious benefits of transforming variables with long tail distributions? To extend it more, why the machine learning algorithms will perform better afer such a transformation?
lucy_in_the_sky's user avatar
1 vote
0 answers
48 views

Can anyone think of applications of a 3 way (k-way) dot product in computer science or data mining

I have developed a locality sensitive hashing algorithm for the 3-way or k-way dot product. When I say 3-way dot product I mean the following. Suppose we have $x,y,z \in [-1,1]^{S}$ for $S \in \...
Mihir Mongia's user avatar
0 votes
1 answer
642 views

What is a range of an attribute in data mining?

In my data mining course, we are working with a data set. I have to identity the ranges of each attribute. I am given the set of data for all 150 points and i am given the visualization (charts) of ...
Lola Martinez's user avatar
2 votes
2 answers
222 views

Why is it not always possible to compute the centroid of feature vectors?

Hi in the data mining and machine learning course that I'm taking there is a subject on feature spaces and there is this part about feature vector aggregation and metric spaces that I don't really ...
Mads's user avatar
  • 21
1 vote
1 answer
881 views

Why does the BFR (Bradley, Fayyad and Reina) algorithm assume clusters to be normally distributed around its centroid?

I'm following a course on data mining based on the lectures from Stanford University and the book Mining of massive datasets. On the topic of clustering, the BFR algorithm is explained with this ...
R. dV's user avatar
  • 145
-1 votes
1 answer
58 views

Dimension Reduction - Which feature should remove to reduce the dimension of the matrix

Let's suppose that we have the following 2 tables: If we want to reduce the dimension by one(in every table) which feature we should remove and why ? I am confused about the way that i should work ...
Emily Serone's user avatar
1 vote
0 answers
34 views

While computing signature matrix in min hashing, can I take nth row of the permutation P in which document d has value 1?

I am learning about some techniques to find similarity between documents. One of the methods is Min Hashing. According to Min Hashing we can find a signature matrix given a random permutation, P. ...
A Beginner's user avatar
3 votes
1 answer
42 views

Find plane within margin of error of >50% of points

There are $N < 3\times10^4$ 3D points. At least 50% of them lie approximately in the same plane, i.e. the distance between the plane and each point is at most $p$. Find such a plane. Attempt: ...
Ignacio's user avatar
  • 133
1 vote
2 answers
87 views

What kind of logarithm does $\log$ mean in Wikipedia articles?

Sorry if this seems like a dumb question, but what what type of logarithm is $\log$ in Wikipedia articles? Cheers.
Robert Hanigan's user avatar
2 votes
0 answers
33 views

In topological data analysis, do bar codes that begin and end at the same index mean anything?

The typical workflow in topological data analysis is from point cloud data to filtration to a list of bar codes corresponding to each dimension. A filtration is a sequence of simplicial complexes, ...
Eben Kadile's user avatar
2 votes
0 answers
81 views

What is the definition of a "Clustering Feature" in BIRCH algorithm?

The paper for BIRCH (a clustering algorithm) contains definitions of a Clustering Feature (CF) where the notation is unclear (cf. PDF page 3 / section 4). A cluster contains N d-dimensional entries $ ...
c11o's user avatar
  • 21
0 votes
0 answers
29 views

What are Key benefits of Ontologies in Systematic Literature Review?

I am working on a Systematic Literature Review (SLR) and about to done with data synthesis. After SLR, I want to create an Ontology and include different details of the SLR in Ontology. I have almost ...
Khan's user avatar
  • 1
0 votes
1 answer
180 views

Bias or not when finding patterns using data mining techniques?

I am currently following a course on Data Mining and i am very curious about the deeper underlying method. As far as i have learned so far data mining is about finding unknown patterns that can be ...
CuriousGeorge's user avatar
1 vote
0 answers
118 views

Combining Computer Science and Humanities

I currently hold a bachelors in Computer science and a masters in Art History. I really want to combine the two and I know of Digital Humanities but I'm not completely aware of where Digital Humanists ...
dcs1's user avatar
  • 11
0 votes
1 answer
266 views

How is density of a substance in grams per cubic centimeter a discrete attribute?

I'm going over the different types of attributes in data-mining (nominal, ordinal, discrete, etc) and there is one example in the book that states that the density of a substance in grams per cubic ...
Phoenixdeath's user avatar
0 votes
0 answers
97 views

Naviers Stokes equation and machine learning

I am looking for a reference explaining how to solve Navier-Stokes numerically using Machine learning algorithms . Thank you in advance for your help .
ABRAICH Ayoub's user avatar
1 vote
0 answers
83 views

Understanding Time Series Data for Classification

I have collected data from numerous volunteers driving a simulator in 8 different scenarios (classes). A volunteer drives in a map for 4 minutes in one scenario (one experiment), then he drives in the ...
Rajiv's user avatar
  • 11
4 votes
1 answer
255 views

Finding (and possibly extracting) source code in heterogenous text data set

I'm looking for a way to recognize and possibly extract source code from text files that may contain only source code, source code mixed with plain text or just plain text without any source code. ...
Marv's user avatar
  • 143
1 vote
1 answer
54 views

How to figure out whether two texts refer to the same object or event

Let's assume there is something happen in the world - Football world cup final. And team-1 beat team-2 with the score 3:2. So there is whole bunch of articles on every website about it, each contains ...
Ph0en1x's user avatar
  • 141
1 vote
0 answers
597 views

Algorithms for tabulating/counting/frequency counting?

It is common in data science to receive two equal length vectors (array of dimension 1), say Categories and Weights. We aim to find all unique values of Categories and sum up the corresponding ...
xiaodai's user avatar
  • 131
1 vote
1 answer
343 views

Data Searching from a large data set without reading each element

I have just started learning algorithms and data structures and I came by an interesting problem. I need some help in solving the problem. There is a data set given to me. Within the data set are ...
user77108's user avatar
1 vote
0 answers
25 views

How may I look for 'regions' of text in a larger corpus of different texts

I have an extremely large (100GB+) corpus of many different texts. All of them are in English and 'well' formatted. They are not loaded into any kind of database, think of them as a huge collection of ...
Alex Morales's user avatar
2 votes
1 answer
138 views

Specific Examples with Explanation of Similarities and Differences of how Distance Functions are used Across Different Fields [closed]

I took a tangent from a student project I had done a number of years ago and spent some time studying distance functions. (please note that the above link contains the full question with links as I ...
Jjjjjjjjjj's user avatar
1 vote
0 answers
308 views

What is the best stream data clustering algorithm that can handle non-static, uncertain data? [closed]

I have gone through many algorithms including streaming k-means, CluStream etc and they all have their pros and cons. What is the best performing algorithm in terms of Computational Complexity ...
Cybernix's user avatar
1 vote
0 answers
35 views

which paper to take that would best fit to becoming a data scientist? [closed]

My name is Webber from Auckland, New Zealand. I am currently in my second year studying the bachelor of computer and information science. I have chosen to do a double major in computational ...
webber's user avatar
  • 11
0 votes
1 answer
885 views

List count of occurrences pairs, triplets, etc. from sets

A receipt is an array of products. I have an array of receipts. I need to generate a report in where I can find the products often bought together. For instance, for a single receipt where the ...
Berry's user avatar
  • 101
1 vote
0 answers
39 views

Outline detection from patterns in a list of textual articles

Are there NLP algorithms dealing with detecting the repeating patterns in a a list of texts from which a topic keywords and other associative keywords can be derived? I will show it as an example: ...
xralf's user avatar
  • 111
1 vote
2 answers
202 views

How to use Neural Network classification if data not same size?

I have data like this. [0 1 0 1 0] [0 1 0 1 0 1 1] [0 1 0 1 ] [0 1 0 1 0 1 1 1 1 0] ... I want to classify with Neural Network but my data different size . I can ...
user572575's user avatar
2 votes
0 answers
26 views

Where to cut a category tree

Since I don't have CS background I will most probably ask this question the wrong way. I need to choose a node from a tree, where I include all beneath this node leafs in a validation. I have a data ...
Thagor's user avatar
  • 121
12 votes
5 answers
19k views

Data Science vs Operations Research

The general question, as the title suggests, is: What is the difference between DS and OR/optimization. On a conceptual level I understand that DS tries to extract knowledge from the available data ...
PsySp's user avatar
  • 251
1 vote
0 answers
88 views

Indoor path prediction with machine learning and pattern matching

Ok. You'll have to bear with me because I'm new to this. I have an idea for a research project. It involves trying to predict a path that someone might take in an indoor environment. My idea is to use ...
Nix's user avatar
  • 11
1 vote
1 answer
212 views

Method for finding correlation between data sets

Let's say that I have $N$ data sets where I have data points at some fixed frequency, such as "daily". What would be a good method for finding correlation between any of the data sets, or choosing a ...
Alan Wolfe's user avatar
  • 1,358
0 votes
0 answers
87 views

LSH and bands for document similarity

I have 50 documents and im using LSH+Minhashing Jaccard and i get this results: for bands:25 -> false positives: 200 for bands:10 -> false positives: 80 for bands:2 -> false positives: 28 for bands:1 ...
Lee Yaan's user avatar
  • 101
0 votes
0 answers
110 views

Minhash Jaccard to find similarity between documents

I wrote a program to check the similarity of 40 documents. Firstly i use exact Jaccard method and secondly Minhash Jaccard method. I consider that the 2 methods differ if their values are differ more ...
Lee Yaan's user avatar
  • 101