I have a simple problem. I can't seem to even find the right search terms to get me pointed in the direction I need to be heading.
I'm writing a bunch of integers to disk. Lot's of them.
Starting with the integer 2, I add it to another integer, write the result to disk, and start the process again using the result as the seed.
This pattern is helping me generate data I need for other research, but I need to apply this pattern until I've reached integers with a length of 10,000 digits or more.
So, here is my simple question: given a set of data of a known data type, how do I calculate the storage space required to store that set?
At the moment, I'm doing this incredibly simple task in Python, recording longs to disk as binary in a single file. But, on my very small machine, I was only able to store the integers my process produced with values between 0 and 42 billion.
Short of my goal of recording a data set with values between 0 and 1e10000.
Given that I have no exposure to formulas for calculating hardware requirements for storing data like this, I have no idea if I'm using the most efficient language and data type for storing as many integers as possible in as little space as possible.