# Tag Info

4

Your best bet is to go with major open source projects (Apache, Open Office, Mozilla, MySql, perhaps Linux). Since these are done by large online communities all communication occurs online through bug databases and forums or mailing lists (which are usually archived). If they do code reviews they may be archived with a tool like Review Board or will be ...

4

DBpedia provides structured information from Wikipedia. It uses RDF, and is one of the most known Linked Data datasets. Also of interest (but not using information from Wikipedia) may be Freebase and OpenCyc.

4

In general, you can look at hard combinatorial instances. If 50 variables is not a hard limit, try e.g., an instance arising from combinatorial block design having 105 variables described in [1]. The actual instance in DIMACS format is here provided by the authors. For the instance, the authors write in [1] that: "We issue a challenge for any solver to solve ...

3

TSPLIB contains some instances of Hamiltonian Cycle. Haythorpe [1] proposes a set of around a thousand instances which are claimed to be "structurally difficult" (not necessarily large). Haythorpe, Michael. "FHCP Challenge Set: The first set of structurally difficult instances of the Hamiltonian cycle problem." arXiv preprint arXiv:1902....

3

See the international SAT solver competition. It's a competition that tests various solvers by running them on a library of hard SAT instances. You can download the instances they used to evaluate the SAT solvers. I would suggest you look at these competitions and see if any of the instances they've used would meet your needs.

3

A standard approach is to pick a standard image bank of other images that you might run into that aren't apples, and use that as your other class. The way you figure out what kind of images you need is: figure out what kind of images your classifier might be run on, and try to make the training set as similar as possible to the kinds of images that it will ...

3

Question: Where can I access automatically generated (spam) webpages? You could use: a parody generator program. They're usually based on Markov chains (e.g. SubredditSimulator) or context-free grammars (e.g. the well known SCIGen... according to the authors "with Markov chains you might get something syntactically correct, but it is likely to be boring"!)...

3

I think you should have a fourth dataset where the images are in their natural environments because you will need information with white background when you will be testing the datasets.

2

You have a dataset with at least billions of entries, whose size is that of a medium-sized hard disk. This is large. If something takes only “many hours” and not many months, that's as good as you can expect. Sorting the file is a viable strategy to get rid of the duplicates. You'll also need to canonicalize the pairs. If you stick with a flat file, the ...

2

I propose this, more direct to code (in java, c, etc.) read each line (until CR/LF) at begining, read a number, until spaces => number n after that, take next characters of line split them by , for each, trim (delete space before or after) => strings s1, s2, s3 then output n s1 CRLF n s2 CRLF n s3, etc.

2

You are probably interested in IAPS, and want to read Analysis of Physiological Signals for Emotion Recognition Based on Support Vector Machine. But to be honest this classification is doomed in advance, not only pictures are stripped of below threshold of effector activisation but also you do not get baseline as point of reference. If you are really ...

2

If you want to detect anomalies, my suggestion would be to build a classifier to classify individual frames of the video to determine whether that frame contains an anomaly. You'll need to categorize what you count as an "anomaly" and arrange for labelled data in your training set. But then you can extract all 6800 frames from the training videos, label ...

2

You can group attributes together, but (assuming you're building some kind of decision tree) your decision tree will no longer be a binary tree, and you probably won't be any better off than just splitting by one attribute. Say for example you group $x_1$ and $x_2$ together, then there are 4 possible values for this new meta-attribute. You can use the same ...

2

The short answer is: I don't know. Visualizations There's been lots of work on visualizing the MNIST data set, in 2 dimensions, and even in 3 dimensions. For instance, here's an embedding into 2D space using a neighbor embedding visualization technique (a highly nonlinear embedding): Credit: Zhirong Yang, Jaakko Peltonen, and Samuel Kaski. See their ...

1

The other approach that I can think of is the obvious one: process the file one line at a time, and consult a dictionary of "seen" lines. What may not be obvious is how to represent the dictionary. There are plenty of on-disk data structures that would work, and this is a well-researched area given that this is what databases do for a living. Some ...

1

You can apply transfer learning. Leverage one of the existing datasets and deep learning networks that can classify faces e.g. celebrities. Add/remove layers on top of the feature extraction layers and use your small data set. Provided that the base dataset / network is large and rich enough you should be able to apply it to your dataset. See https://...

1

The term "point cloud" is normally used to refer to a set of points (locations) in 3D space. So, a table of temperatures would normally not be called a "point cloud".

1

Question 1: How to solve the finding uniques problem If your data is consistently formatted in this way: [ ['somestring', 'someotherstring'], ['anotherstring', 'anotherotherstring'], ] And so on, you can actually just call dict directly on it and it will reduce all of the first items to keys, and keep the last of the second items as the values, and this ...

1

I would say Algorithms by Robert Sedgewick and Kevin Wayne is a classic book which explains quite well the basics of these concepts. However you could also find a lot of resources on the web for free.

1

The simplest answer is: analog data is continuous, while digital data is quantized. In other words, a signal that at any point in time might have any real value between 0 and 1 would be considered analog. A signal that at any point in time has a value of either 0 or 1 would be considered digital.

1

There is no way around collecting more data if you want to increase your test set and thus how meaningful your benchmark results are. If you, however, are worried that your training set is too small, then there are two main ways to deal with that: Data augmentation which adds invariant transformations to the data (e.g. see Analysis and Optimization of ...

1

I know nothing about machine learning, so take the following with a pinch of salt. (I originally posted it as a comment and was encouraged to repost as an answer, so I guess it can't be terrible.) You get more data by, well, collecting more data... Simply using some algorithm to generate more data similar to the data you already have won't help, because it ...

1

Here is a fairly old link, which only lists a few classes specifically, but it might be a place to start. I'd agree with David Eppstein that a longer list might not be all that useful. There's no quick way to tell for sure whether a given CA rule is Class 4, except in the really simple cases. In many cases, with enough work you may be able to design ...

1

Let's start with comparing two time series. To compare them, you probably want to use cross-correlation. The cross-correlation computes the correlation between signal $f$ and a delayed version of signal $g$, for all possible delays. An efficient way to compute the cross-correlation is indeed by using a FFT, taking the product, and then applying an inverse ...

1

Metadata is an infamously ill-defined term. Roughly speaking, it's trying to get at a distinction along the following lines: data is the "content" you care about; metadata is other information that is secondary or exists primarily for technical reasons (e.g., addressing information, tags, technical data to facilitate storage or communication). A classical ...

1

You can for instance take as ground truth data the movielens dataset, remove some rating links between users and movies. You can rank your algorithm by counting the number of link that you can guess right. Usually machine learning algorithm also guess the rating score.

1

In this sort of situation, two standard answers are: Figure out what the practical applications of your algorithm are. Find a dataset associated with that particular application, and try your algorithm on it and see how well it works. Measure success using some metric that is appropriate for that particular application. Look through the research ...

1

Another option is to use an encoding scheme for coordinates like geohash (http://en.wikipedia.org/wiki/Geohash). From wikipedia: The main usages of Geohashes are as a unique identifier. represent point data e.g. in databases. Geohashes have also been proposed to be used for geotagging. When used in a database, the structure of geohashed data has two ...

1

For quick look-up on multidimensional points you need to create a R-tree index (http://en.wikipedia.org/wiki/R-tree) or any of its variants (http://en.wikipedia.org/wiki/Spatial_index#Spatial_index). From wikipedia: R-trees are tree data structures used for spatial access methods, i.e., for indexing multi-dimensional information such as geographical ...

1

Hash map / Hash table (depending on the language used) Per comments, Some reasoning why: According to Introduction to Algorithms by Thomas H. Cormen, Charles E. Leiserson, Ronald L. Rivest, and Clifford Stein, the big O of hash tables is O(1) to O(N) for insert, size, and time. So best case is O(1) solution. If you are able to control the upper and lower ...

Only top voted, non community-wiki answers of a minimum length are eligible