True Random Numbers - Computerphile 

with also an italian dictionary

🤌

Ay, I'm glad-a you liked eet. You wanna some coffee? 😄

Straight from Numberphile I see

been a long time since i've seen smurfs in the wild

It's often used to seed the OS-level PRNG

A shortwave antenna - literally one tiny quartz diode - is a more reliable TRNG than this gross attempt.

@simonsomething2620 I was also curious, so I looked it up. (Had a much better response before it got lost in transit) Basically, thermal noise on the processor is measured, converted to a numerical state, and then that value is used as the seed for a pseudo-random number generator.

Yes. Like the different sources of entropy, as he stated in the video.

Some are temperature and humidity for example

Yes! This! Is it munging environmental measurements, or something like sampling between asynchronous clocks?

@@damian_smith Or cosmic radiation? I know this is sometimes used, because it's incredibly random.

@@edwardian23 hmm. Posted but maybe RU-vid didn't like the link. I only know approximately but you can toggle a bit at, say, 100MHz and then sample it at, say, 101MHz. Sometimes you'll sample it while the voltages are changing and get a random answer.

I was going to ask this question to the general public this is even better! When his code runs on Compiler Explorer is he calling a real rdseed?

Interesting he didn't seem to know where the term "Godbolt" comes from

@@SamMason0 who can blame him :) It's a funny thing, many people don't realise... But the site's called "Compiler Explorer"; I can't call it the Other Name...it's too weird :D

@@SamMason0 where does it come from?

@@hwstar9416 it's Matt's surname... It was just his personal site, but the compiler explorer got kind of popular and took over. He gave an interesting talk about it recently at C++ On Sea

I rarely make comments and very rarely negative about a video but honestly I felt like this video was a complete waste of my time. I watched waiting for the part where he would explain the source of the true randomness that the hardware could leverage but it never came. Thanks for looking it up and providing the answer, I thought it would be something related to the CMB or some electrical voltage wave but thermal noise in the silicon is even more interesting.

Linux also has a similar system in place, where it collects entropy from all hardware connected to the system into /dev/random and /dev/urandom. I wonder what's slower; using rdseed or using the getrandom() syscall. Both should provide true randomness

That's because it's computerphile... Not sixty symbols...

I would assume that means that the circuit uses a reverse-biased P/N junction to generate noise (which a reverse biased junction does naturally) and then it clocks that noise through a schmitt trigger to turn it into a stream of 0's and 1's to create the output.

@@aqua-op Lol, yeah I was (and still am, haven't slept) a bit drunk, so I was kinda easy with loose words. But I'm glad because others wouldn't have said it. It wasn't that harsh. He obviously understands what's happening, just didn't explain the important part. It was still interesting when he said such things could have a bias toward one side or one set of numbers. Anyway thanks for your comment. This, (randomness, or lack thereof) is one of my favourite things. 🙂

Also that's why in many games you can't abuse save-loads to get desired rng

I can add the example of Shattered Pixel dungeon and many offshoots of it, for the same reason.

Wouldn't the player's input cause a variable count of numbers to be pulled from the sequence, and the desired loot get a different number if they are not decided at game start but rely on a seed? Say, for example, you go where a monster spawns, which has some properties, and then you go for your loot? The Sims 2 had a very bad random number, and you could see the same set of people being chosen. Adding together any variable data like time and object identifiers could improve upon it.

@@j7ndominica051 Easily solved by having the seed used for world generation and a different rng for everything else

@@j7ndominica051 Loot and monsters are not determined from seed only, but all the blocks and structures (villages, pyramids etc.) are fully determined, so it's essentially the same world, it looks the same

Sadly, it is implementation defined and not required by the C++ standard to do so.

@@sledgex9 Well, as long as you know whether your IDE does it or not.

if you need a different seed every time just use clock( ) or time( ). True purpose of true random numbers is cryptography. Imagine you use a rand( ) with some seed, no matter how secure the seed is, the sequence itself is deterministic. Thus you can predict it and brake some encryptions. But even if the sequence is truly random, then, as it was stated, it might not be evenly distributed. That means, that the numbers must belong to a determinable subset of numbers, and it will be easier to brutforce the key; that turns an almost impossible task to doable task within a couple of months.

@@Raspredval1337 How exactly does the evenness of the number distribution or the lack of it thereof means the the following numbers belong to deterministic subset?

@@Raspredval1337 The problem with using time for your seed is that if the attacker knows when your program started, it gives a rather small set of possible seeds to exhaustively try. Trash for cryptography pruposes

Threads: I'm about to end this man's whole career

Both of these are circumstance dependent though - /dev/random on most Linux distros uses true random numbers and blocks reads when "empty", but iirc can sometimes be plugged into a pseudorandom number generator that continuously reseeds from hardware entropy (eg in environments where hardware entropy is scarce such as VMs which don't always have access to a dedicated hardware entropy generator), /dev/urandom is non blocking and traditional behaviour is to use a pseudorandom number generator once true entropy is exhausted until more true entropy is generated. Notably, in many BSD implementations /dev/random is just a link to /dev/urandom, so all random numbers generated on BSDs are potentially pseudorandom numbers generated from a true random seed instead of forced true random.

Something I tend to do with my programs is to start out with a "seed" derived from the TOD clock, then generate a small integer, get that many random numbers, then save the last one to a file to be used in future as the "seed" instead of using the TOD clock. The procedure of generating a small integer and then getting that many random numbers and saving the last one as the new seed seems to provide sufficient randomness for most games.

I had a windows-based casino game back in the 90s. In some games, each round started with the same sequence of random numbers, making it easy to cheat.

To be as generic as possible and skipping over how implementations change: The Hardware RNG is a buffer of entropy samples collected from entropy source(s) that are typically fed through thermal or electrical noise, conditioned for quality and governed with an entropic heath check of sorts. These are analog components on the die. Often this process is so slow that rather than being used directly, the random numbers are actually used to seed Procedural RNGs, after being conditioned with a Detrrministic Random Bit Generator to remove identifiable characteristics of the hardware noise.

alpha scatter from its Plutonium 238 power source.

@@mytech6779 don't mislead the children

you clearly missed the point of the question you basically didnt say anything at all .. given a biased coin toss how do you unbias it without knowing the bias priori .. I dont know the answer but I know that because I dont know the answer that you arent even close to ever being informative here because if you were thats where you would have went .. you dont even know enough to know what the fundamental issue is ("collect entropy" is not meaningful)@@orbatos

You might need to have a look on stochastic computing that exploit cosmic noise to perform approximate computing. You will be fascinated by the fact that a simple AND gate could do arithmetic multiplication by exploiting probability.

The important difference between PRNG and TRNG is not the distribution, you can always fix that for either type. What you want is to have no ability to predict the NEXT number in the sequence with certainly-better-than-chance probability.

You're encouraged to extract the entropy from these instructions via some cryptographic primitives, e.g. sha hash the output. This would remove any bias/"skew" and get you back to a uniform distribution.

@@ed_halley That is just another way of saying that the important difference is precisely the evenness of the distribution. That is because you can not determine the probability of the next number unless you know the distribution.

That's the job of libraries like DIHARDER and BigCrush.

@@davidgillies620 Why do both those libraries sound so dramatic

Woah, a wild tr1p! *edit:* speaking to that end part, it's kind of like on the Z80, seeding a PRNG with the R register. It's usually limited to 128 possible starting configurations, but still helpful

No.

To expand on the "no": a pseudo random number generator usually has guaranteed behaviors on the generated numbers **per seed**, given you take a "large" amount of samples. That is, no matter the seed, eventually the generated numbers will approximate a given distribution. Usually bias in PRNGs come from limited sampling or poor use.

No. Pseudorandom number generators are like mathematical blenders designed to always produce something uniformly distributed. Even if you throw all 0s in there it will blend it up to something uniformly distributed. However, it is important to keep in mind that if your initial seed is constantly nonuniform, such as, it is mostly 0s with just a few bits changed, then this can make it easier for an attacker to guess the seed. Any nonuniform but truly random bit stream can be converted into a uniformly random bit stream using a technique called a von Neumann extractor.

@@amihartz ahh that makes sense, thanks

It may seem that there is bias. I believe that the numbers go from 0 to 2^31. That is 0 to 2.1billion. So.. note that any leading zeroes are removed and about half the numbers between 0 and 2.1B do actually start with a 1. Now note that any numbers less than 1B do not start with a zero. And one out of 9 starts with a 1. That is probably about half the numbers starting with a 1. The same should be true of the sequence generated earlier.

It isn't a distribution that spans some power of 10.

The function he calls gives 32 random bits, it's then converted to an int that spans between ±2.1 billion and displayed as base 10, that's why there's bias

Not sure. Around 10:45, he says that "we don't know what source of entropy the computer is using".. so maybe the answer is "I don't know how it generates random numbers". He says that a typical pseudorandom generator picks numbers with an equal probability distribution, so that every number is equally possible. But this "true" random generator has a skewed probability(?) (which sounds to me like it would make it 'less' random)? Maybe I should just research it instead of typing a reply without knowing anything heh.

As far as I know intel uses thermal noise, but we are unable to check what they are actually doing in hardware, it's just too complicated and miniaturized

Electronic Design: “Understanding Intel's Ivy Bridge Random Number Generator” explains the design. A meta stable circuit is constantly flipping its value. Then it is sampled, I.e. the randomness is copied from meta stable to an unpredictable clocked signal. From that point on, Intel does a bunch of deterministic stuff to assure quality. But at the core of it: have a signal switch value at infinite speed (meta stable), then clock it. The clocked signal is unpredictable.

@@randomgeocacher thanks!

All "true randomness" means is statistical independence. A bit stream is truly random if it satisfies the equation P(0)P(1)=P(1)P(0), meaning, it is equally as probably as a 0 will occur after a 1 in the bit stream as a 1 would occur after a 0, because each bit is statistically independent of each other bit. Often, true randomness is falsely conflated with uniformity, which is the idea that 0s occur with equal probability as 1s. However, a bit stream 0101010101 is uniform yet clearly not random. Any truly random but nonuniform distribution can also be easily converted into a uniform distribution by making use of statistical independence by assigning pairs of 01 to 0 and 10 to 1 and throwing out pairs of 00 and 11. Since 01 and 10 have equal probability of occurring, you would be guaranteed to get a uniformly distributed set of random numbers. This only works if they are truly random, because the statistical independence and will fail if the bits are statistically dependent upon one another as statistical dependence follows different mathematical laws. The trick to building a truly random number generator is thus to just generate bits which are as independent of each other as possible, where no analyst who sits down and studies a bit stream could find any statistical relations between any of the bits. Often this is done by sampling many different sources, like computers often generate random numbers by sampling fan noise, CPU temperature, keyboard and mouse movements, so on and so forth. Often they will just take the least significant bits from these sources as well. If they use the entire CPU temperature, then it might be predictable that it would be a larger number in the day and a lower number at night due to when people would typically use the CPU. If you just sample the tiniest fluctuations in CPU temperature, though, then there is not an obvious correlation between what programs are running and those tiny fluctuations in temperature. You can then combine these many different entropy sources into a single bit stream which should produce statistically independent bits, and either directly throw it into a pseudorandom number generator as the seed, or you can keep sampling until you have enough to turn it into a uniformly random probability distribution and use that as the seed. Cryptographically secure pseudorandom number generators these are so well understood you can write one from scratch in about 5 minutes (like ChaCha20) making it pointless to actually use the true random numbers directly. It makes a lot more sense to use them to seed a pseudorandom number generator.

I thought the same thing lol

That would be very bad cryptography if it could be broken by knowing the general source of entropy. He avoided clouding the demonstartion because the details change with each model of processor and are pretty tedius in any case. Mostly it involves collecting the signal-noise on analog inputs such as thermal sensors.

He didn't meant to say the source is unknown. He meant the statistical properties of the source is unknown... (As for any true random number generator)

I agree, the whole point of the video is not mentioned, except when the guy asks and he says it's unknown. Poor.

Just to be clear on my question. An un biased seed has to be a true random ‘seed’ seems like a chicken or the egg type question.

I kind of disagree of the unimportance of a true random number. Using “Rand()” in Excel (that most analysts do) is not random. It is paramount in statistical surveys where a population is truely randomly surveyed. It’s just a “random as we can get it” survey.

Technically any set of photos can be used to create random numbers. Using a webcam pointed at anything will do it. Even in the dark. The classic of being pointed at a wall of lava lamps really doesn't _need_ the lava lamps. Why boils down to how these cameras work. You never get exactly the same values out for each pixel.

​@@DampeS8NYou don't need the lava lamps, true, but you don't *need* the camera either, since CPUs have built in generators. The reason for the lava lamps is because CloudFlare needs *a lot* of random numbers and they're not just relying on sensor noise but actually relying on the random appearance of the lava lamps themselves to get it.

Around 2005 I ran into that issue that /dev/random would stall due to not enough entropy. It took me ages to find out why unit tests with TLS stalled until I moved my mouse. Then I bought an USB "entropykey" that could provide entropy to /dev/random.

@@kirkanos771 For one Linux's /dev/random implementation no longer blocks like that. That's why I mentioned the implementation change. Second, the device I had fed the kernel's entropy pool, and thus /dev/random, such that it never blocked. That was the entire point of my comment. Trying to make a gotcha comment to me when you don't understand what I'm saying is really weird.

...why brick the device? Why not just shut it down or something?

@@amihartz To fail secure. It's a source of entropy. If that entropy ever becomes biased, the secure thing to do is make the device unrecoverable and unusable so that no one will ever use a biased entropy source. Because it wouldn't be entropy at that point.

@@klfjoat It depends on what you mean by "biased." Uniformity has nothing to do with security, an incredibly nonuniform source of entropy is still cryptographically secure as long as successive bits in the bit stream are statistically independent of one another. If by "bias" you mean they become statistically dependent, then yes, you should not use such a source, but if it is nonuniform yet satisfies statistical independence then it does not hamper the security. Also, destroying hardware still seems like a drastic decision and massive waste. There are ways to disable hardware without destroying it...

Cryptography, mostly

it's a Nespresso coffee machine. hth

Random or broken? 😄

@@blucat4 probably both

@@blucat4 can't trace back the sequence if I don't know my own seed! *WHACK* Works great.

@@XenoTravis Is that the new With_Hand_Attack_Computer_Keyboard method? 😄

Well technically no since your inputs arent truly random. Just less predictable

Depends what you mean by "true" random. At a very low level, only quantum phenomena are truly random. However, for the level of randomness we usually need, even at the highest levels, user input (maybe not keystrokes cause they can be too repetitive) or a combination of several inputs can be considered true random at a high level of abstraction

Of course manual input is truly random. That is, unless we’ve all become part of an AI 😊

@@TheKastellan so you saying that can predict the exact movement of my laser mouse ?

Like in the example at the end, where he feeds the true random seed into the pseudo random number generator, taking mouse movements etc. will generate a pseudo random number. This does not mean that it is in someway unsafe or "not random enough", it just means that technically, if you know the original state, you know the results. There are alot of "cryptographically safe random number generators" that produce pseudo random numbers, but are "impossible" to predict or recreate.

According to quantum mechanics there is true randomness, not just that we don't understand what is happening. Heisenberg uncertainty principle for example.

@@foible2085 You mean according to your _interpretation_ of quantum mechanics. Quantum mechanics is a deterministic theory just like any others. If you flip a truly random and fair coin an infinite number of times and formed a probability distribution of the results, the results are guaranteed to deterministically converge to a distribution of 50%/50%. While each individual flip may be random, the probability distribution is deterministic. Quantum mechanics gives deterministic probability distributions of what ensembles of systems will converge to according to the Born rule. Whether or not a single system (just one sample, or the behavior of just a single particle when the experiment is only ran once) is truly random is out of the scope of quantum mechanics itself (as a body of mathematics) and a question for philosophy and interpretation. The uncertainty principle could be due to intrinsic uncertainty in the universe but there are also interpretations that posit it is caused by interacting with the particle disturbing its state (see Spekkens' toy model for example).

@@foible2085 Depends on how you interpret quantum mechanics. There are ways to interpret quantum mechanics deterministically, but they are unpopular as they rely on presuming that there is some variable which for some reason is hidden from us by the natural world and so we cannot know of it. (If it wasn't hidden from us, then we could go measure it, and then you would have a new theory rather than just an interpretation.) If we can't observe something, it is simpler to just not presume it exists in the first place.

Came here to comment exactly this. The pseudo random generator will return the same result for the same seed every time, which basically makes it a mapping function where you say "if input is 1, output 9; If input is 3, output 5;etc". Feeding it with a skewed true random number will simply result in another skewed random number. It makes no sense at all to use it as a seed.

The equally distributed is a property of the PRNG regardless of the set used

​@@MatthisDayerI agree, but the PRNG is applied over a skewed set of numbers. The resulting set should also be skewed.

@@thomaswolfe9490 they would be skewed to a set of still equally distributed sequences

I would argue the thing that makes some TRNGs insecure isn't that the source is thermal noise, but the fact that circuit implementations to sample the noise introduce biases.

The thermal noise effects these hardware number generators rely on are created by quantum effects in the silicon though, so if these are deterministic then the entire universe is deterministic and true random number generators don't exist at all.

Yes and no. The benefits of PRNGs (particularly cryptographic PRNGS) is that we have mathematical proofs about their output statistics. The best you can get from a TRNG is a physics based circuit model and the output from a statistical test suite confirming said model.

There are two components of picking a random number: the random picking and the distribution from which it is picked. E.g. 1, 1, 1, 1, 1 is a random sequence of numbers from a uniform distribution starting and ending at 1. Hardware RNG can guarantee that the process is random, but it's really hard to prove that the distribution is uniform (or of any other particular shape), since the distribution is related to the current physical environment of the processor. PRNGs are mathematical, so their distribution can be determined, but since they're just a deterministic algorithm - we have to pick their initial inputs truly-randomly, otherwise they're predictable.

@@d3line The uniformity of the random number distribution genuinely does not matter in the slightest. If the distribution is truly random, meaning, each successive bit is statistically independent from the next, then it is trivial to convert it into a uniform random number distribution because it follows the statistical law P(A)P(B)=P(B)P(A), so all pairs of AB (01) and BA (10) would be guaranteed to be uniformly distributed even if the overall distribution is not. You can then just map those pairs to 0 and 1 respectively and throw out the pairs AA (00) and BB (11) and you have a uniform distribution. This is called a _von Neumann extractor._

Back in the day, you could cycle the sequence, and the errors that that causes can be really hard to notice.

@@DrDeuteron Not sure I follow, can you elaborate a little?

@@Kwauhn. a pseudo random number generator repeats its sequence, at which point it is not random, so calculations based on random numbers respond bizarrely

@@DrDeuteron I see. You're saying that using a hardware random seed makes it harder to predict the behavior of a pseudo-random function, right? Like, if you used a language standard RNG function that uses standard seed initialization, it would be easier to guess the function and predict its outcome?

@@Kwauhn. no, I’m saying if you a do large complex simulation or montecarlo, and you call the rng more times than it has unique outputs, your calculation can get really wrong in subtle ways.

Didn't pay attention to his name nor the room he was in. For me (I'm from Brazil) he sounded like from some place in eastern Europe.

Let's hear you speaking in Italian, or Spanish, or French, or any other language. Your language accent would get in the way...

BORAT! His accent remember me Borat speaking. That's it!

Because there are more than 2 billion numbers with 9 or 10 digits. But only 100 million with 8 or fewer digits.

there are 10 one digit numbers, 90 two digits numbers, 900 three digits numbers, the more the digits, the more numbers

Yes; the range [1, 9] has aproximately 10 times less ammount of numbers than [10, 99], which has aproximately 10 times less ammount of numbers than [100, 999] and so on. Since the numbers whith more digits are more represented, they are going to appear more frequently than the ones with less digits.

He programmed it that way.

At a low level the RNG actually just spits out a string of ones and zeros, so the digit count is determined by the software implementation and how many random binary digits it asks the hardware for

Code compiled for x86 won't run on ARM or POWER. That's a compile-time check. What language are you writing code in?

@@pierreabbat6157 You're right, of course. Compiling for x86 means that you don't have to worry about it, and iirc early on in the video it's stated that this instruction has been supported for a long time.

​@@WalderFreyBasically any Intel processor since 2012 and any AMD processor since 2015 should support it (technically AMD is planning to roll out ARM CPUs as well but I don't think any of those are on the market yet and it wouldn't surprise me if those included an equivalent function since they're initially going to target the server market where CPU RNG is far more important as desktop users need less entropy and generate their own by interacting with the system).

CPUID is used to look for X86 feature flags like RDRAND support.

@@pierreabbat6157 My PC runs as 12th gen i7 and it has the instruction, but when I moved it to my server which is running a 2nd gen i3 the program was crashing, and it took me a bit of digging to find that in my cryptography library I had put an option to use that instruction and had it turned on, and that CPU did not support the instruction despite being x86.

My understanding is that Linux still mixes the hardware generator output in with other sources of entropy, including the random timing variations in disk activity, and random fluctuations in inputs like mice and various hardware interrupts. The last time I looked into it in detail though was during the controversy not long after it was first implemented during the whole NSA revelation, when it turned out it was being mixed in in a suboptimal way that meant it was theoretically possible for a specially designed hardware RNG to unrandomise the pool if created by an exceptionally sophisticated attacker.

​@bosstowndynamics5488 You are right. Linux mixes multiple different entropy sources.

True, but at the moment our best understanding of physics at the scale used by these systems is using quantum physics, which describes a true random universe, so the debate position that it's deterministic is pretty weak. Plus, even if it is, in order to predict the outcome you would need the "seed" for the entire universe, and fortunately most cybercriminals still have difficulty determining that

It doesn't change anything.

@@misterhat5823 That's what I'd guess, as well.

​@@antiHUMANDesignsit doesn't change anything because your pool of inputs is already truly random. Regardless of how biased a human is, if the initial pool is truly random then it doesn't matter which number the human picks.

@@mareau2193 If the distribution is perfectly uniform both in the serquence and in the picking, I get that the result shoudl be uniformly random, but what if there's a small deviation from uniform distribution in both algorithms? That is, in both the sequence and the picking algorithm. Does the result get twice the deviation, or less of a deviation from perfectly uniforms, or is the deviation as large as the algorithm with the largest deviation, or perhaps the one with the smallest deviation? That's more technically what I'm asking.

No. If you amplify the thermal noise in a resistor, or the shot noise in a diode, you get random events that are impossible to duplicate (in other words, you can't get "identical conditions") I suppose if you consider amplification of thermal or shot noise to be "an input", then OK. You can't get truly random numbers out of a deterministic process.