ML algorithms are definitely capable of storing information. Generally, any ML algorithm can be trained to remember a set of data. This is commonly referred to as overfitting and something that most ML researchers try very hard to avoid.
Also feature learning
and compression are common themes in ML. In this case, the goals are actually to find efficient ways to represent the data. This allows to store information efficiently. Because you asked for some illustrative papers: image compression with autoencoders, image compression with PCA, ...
Methods like generative models are probably also interesting as their purpose is to discover the distribution of some data in order to generate new data. This technique can be used to generate new images that are not just random noise for example.
These are just some examples with some random papers that are top hits when searching for the terms I mentioned. Probably every machine learning algorithm can be said to store some sort of information, but for some models the stored information is more useful/meaningful than for other models.
Generating pi with machine learning has probably not been done (at least it has not been published). However, nothing keeps you from trying it out. You could try to overfit a model to learn digits of pi or you could just try what happens when you feed an LSTM with a very large sequence of pi-digits (you would have to find a reasonable objective, but might be interesting), or do some sequence to sequence learning with encoders and decoders to predict digits for pi. Probably, the methods would not perform well from a ML perspective, but some interesting things might happen.
PS: as you might have noticed, I have mentioned quite some neural networks. There are other methods out there as well: do not get too biased.
