3
$\begingroup$

DISCLAIMER : I'm pretty new to machine learning field, so forgive me if my questions are somehow naive ... I already searched on Internet about this topic and find nothing interesting so I am asking here ....

I wonder if a ML algorithm can be consider as a way to store data ? There is programs that "contains" thousands of decimal of PI (e.g. https://www.quora.com/What-is-the-smallest-C-program-generating-pi-without-using-the-math-library) : in some way it's a "compressed" version of the first thousands decimal of PI, right ?

In the same way ML (e.g. Deep Neural Net) seems to store information. Can we mesure how many bits is stored in a Deep Neural Net ? Can we use ML algorithms data structure to compress information and retrieve it later (even if it's not 100% accurate) ? ...

Is there (not too technical ) papers that discuss these possibilities ?

Thanks :)

$\endgroup$
2
$\begingroup$

ML algorithms are definitely capable of storing information. Generally, any ML algorithm can be trained to remember a set of data. This is commonly referred to as overfitting and something that most ML researchers try very hard to avoid.

Also feature learning and compression are common themes in ML. In this case, the goals are actually to find efficient ways to represent the data. This allows to store information efficiently. Because you asked for some illustrative papers: image compression with autoencoders, image compression with PCA, ...

Methods like generative models are probably also interesting as their purpose is to discover the distribution of some data in order to generate new data. This technique can be used to generate new images that are not just random noise for example.

These are just some examples with some random papers that are top hits when searching for the terms I mentioned. Probably every machine learning algorithm can be said to store some sort of information, but for some models the stored information is more useful/meaningful than for other models.

Generating pi with machine learning has probably not been done (at least it has not been published). However, nothing keeps you from trying it out. You could try to overfit a model to learn digits of pi or you could just try what happens when you feed an LSTM with a very large sequence of pi-digits (you would have to find a reasonable objective, but might be interesting), or do some sequence to sequence learning with encoders and decoders to predict digits for pi. Probably, the methods would not perform well from a ML perspective, but some interesting things might happen.

PS: as you might have noticed, I have mentioned quite some neural networks. There are other methods out there as well: do not get too biased.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.