The awkward status of Mersenne Twister

Question

Random number generators generally fit into 3 categories IMO:

1. ad-hoc designs serving as stub to be replaced by something else if this ever happens,
2. considerate designs aiming at achieving statistical quality and usable in their respective application domain,
3. cryptographic RNGs of various kinds and purposes needs to fend off attack by mathematical cryptanalysis and brutal-force attack running on supercomputers.

Generally, a block-box instance of type-3 satisfies all requirement of type-2 and type-1 barring implementation efficiency details; and to be honest, there are type-3 algorithms that's already quite efficient.

The only reason I can think of not to use a type-3 RNG, would be export control - some country have restrictions on export of millitary-grade cryptography systems.

And considerate designs are plentiful, so I can't see why anyone have to settle with Mersenne Twister, which have a huge state whose only benefit would be having a absurdly long period. The other reason I can think of using it, is that people have already used it, and stopping using it, will mean losing scientific reproducibility.

So is that all the reason MT cannot be easily replaced? Have I overlooked something?

Answer 1

Substantive reasons: Some PRNGs intended for statistical simulations are faster than cryptographic PRNGs. Yes, cryptographic PRNGs are pretty fast, but some statistical ones are even faster, and are good enough for some purposes.

Non-technical reasons: There is a lot of inertia and many people just use whatever they have heard of, regardless of whether technically it is the best choice. Mersenne Twister has good name recognition. And, Mersenne Twister is the "IBM" of statistical PRNGs: you probably won't get fired for using Mersenne Twister, if you need a statistical PRNG. (See the concepts of "path-dependence" and "satisficing".)

---

In practice a long period is a poor criterion to use for selecting a PRNG. Any decent PRNG has a period that is more than long enough. I see focusing on the period as an instance of "looking under the lamppost for your keys"; it is easy to measure the period, and harder to measure pseudorandomness, and it's tempting to focus on the metric you can measure rather than the one that matters. Think of the long period as more of a marketing bullet point than anything that matters in practice. (Yes, yes, I'm aware that a short period makes a PRNG flawed. I hope you'll give me some credit for having some background in this area, and ask a separate question if you want to understand in greater depth the relative value of hte period as a criterion for evaluating candidate PRNGs.)

Answer 2

On top of what D.W. said, scientific reproducibility is absolutely a real concern. I personally know a researcher who spent a couple of days ensuring that their PRNG was bit-for-bit accurate to an existing one, when writing a new Monte Carlo simulation in a different language.

It's easier to make valid comparisons between simulation techniques if you have changed fewer underlying assumptions. If you used MT (or LFib, another long-period PRNG which is often used in that space for multithreading reasons), then you are not open to the critique that you chose a PRNG to get the result you wanted.