Intel Secure Key (RDRAND) possibly strange behaviour
I've been using the Intel-provided RNG feature for some time, to provide myself with some randomness by means of a C++/CLI program I wrote myself.
However, after some time, something struck me as particularly suspicious. Among other uses, I ask for a random number between 1 and 4 and wrote the result down on paper each time. Here are the results :
2, 3, 3, 2, 1, 3, 4, 2, 3, 2, 3, 1, 3, 2, 3, 1, 2, 4, 2, 2, 1, 2, 1, 3, 1, 3, 3, 3, 3.
Number of 1s : 6
Number of 2s : 9
Number of 3s : 12
Number of 4s : 2
Total : 29
I'm actually wondering if there's a problem either with Intel's RNG, my algorithm, methodology or something else maybe ? Or do you consider the bias not to be significant enough yet ?
I'm using Windows 10 Pro, my CPU is an Intel Core i7-4710MQ.
Compiled with VS2017.
Methodology :
- Start a Powershell command prompt
- Load my assembly with
Add-Type -Path <mydll>
- Invoke
[rdrw.Random]::Next(4)
- Add one to the result
A detail that may be of importance : I don't ask for that number very often, so there's some time between draws and it usually comes when the RNG hasn't been used for some time (one hour at least).
And yes it's a lazy algorithm, I didn't want to bother myself with exceptions.
Algorithm follows :
#include <immintrin.h>
namespace rdrw {
#pragma managed(push,off)
unsigned long long getRdRand() {
unsigned long long val = 0;
while (!_rdrand64_step(&val));
return val;
}
#pragma managed(pop)
public ref class Random abstract sealed
{
public:
// Returns a random 64 bit unsigned integer
static unsigned long long Next() {
return getRdRand();
}
// Return a random unsigned integer between 0 and max-1 (inclusive)
static unsigned long long Next(unsigned long long max) {
unsigned long long nb = max - 1;
unsigned long long mask = 1;
unsigned long long draw = 0;
if (max <= 1)
return 0;
// Create a bitmask that's at least as big as the biggest acceptable value
while ((nb&mask) != nb)
{
mask <<= 1;
mask |= 1;
}
do
{
// Throw unnecessary bits
draw = Next() & mask;
} while (draw>nb);
return draw;
}
// return a random unsigned integer between min and max-1 inclusive
static unsigned long long Next(unsigned long long min, unsigned long long max) {
if (max == min)
return min;
if (max < min)
return 0;
unsigned long long diff = max - min;
return Next(diff) + min;
}
};
}
Thanks for your insights !
random cryptography visual-studio-2017 c++-cli rdrand
|
show 3 more comments
I've been using the Intel-provided RNG feature for some time, to provide myself with some randomness by means of a C++/CLI program I wrote myself.
However, after some time, something struck me as particularly suspicious. Among other uses, I ask for a random number between 1 and 4 and wrote the result down on paper each time. Here are the results :
2, 3, 3, 2, 1, 3, 4, 2, 3, 2, 3, 1, 3, 2, 3, 1, 2, 4, 2, 2, 1, 2, 1, 3, 1, 3, 3, 3, 3.
Number of 1s : 6
Number of 2s : 9
Number of 3s : 12
Number of 4s : 2
Total : 29
I'm actually wondering if there's a problem either with Intel's RNG, my algorithm, methodology or something else maybe ? Or do you consider the bias not to be significant enough yet ?
I'm using Windows 10 Pro, my CPU is an Intel Core i7-4710MQ.
Compiled with VS2017.
Methodology :
- Start a Powershell command prompt
- Load my assembly with
Add-Type -Path <mydll>
- Invoke
[rdrw.Random]::Next(4)
- Add one to the result
A detail that may be of importance : I don't ask for that number very often, so there's some time between draws and it usually comes when the RNG hasn't been used for some time (one hour at least).
And yes it's a lazy algorithm, I didn't want to bother myself with exceptions.
Algorithm follows :
#include <immintrin.h>
namespace rdrw {
#pragma managed(push,off)
unsigned long long getRdRand() {
unsigned long long val = 0;
while (!_rdrand64_step(&val));
return val;
}
#pragma managed(pop)
public ref class Random abstract sealed
{
public:
// Returns a random 64 bit unsigned integer
static unsigned long long Next() {
return getRdRand();
}
// Return a random unsigned integer between 0 and max-1 (inclusive)
static unsigned long long Next(unsigned long long max) {
unsigned long long nb = max - 1;
unsigned long long mask = 1;
unsigned long long draw = 0;
if (max <= 1)
return 0;
// Create a bitmask that's at least as big as the biggest acceptable value
while ((nb&mask) != nb)
{
mask <<= 1;
mask |= 1;
}
do
{
// Throw unnecessary bits
draw = Next() & mask;
} while (draw>nb);
return draw;
}
// return a random unsigned integer between min and max-1 inclusive
static unsigned long long Next(unsigned long long min, unsigned long long max) {
if (max == min)
return min;
if (max < min)
return 0;
unsigned long long diff = max - min;
return Next(diff) + min;
}
};
}
Thanks for your insights !
random cryptography visual-studio-2017 c++-cli rdrand
3
Pretty far from significant, indeed. Try at least a hundred thousand draws and if the result is still biased to a specific value then there are clear issues. Full random tests from NIST require about 7 GB (max, to run all the tests without repeating random values for the tests). Needless to say, you want to use some kind of table in your application instead of a piece of paper.
– Maarten Bodewes
Nov 11 at 22:02
@MaartenBodewes : I hear you, you must be right and I'll continue assuming there's no bias for now !
– NovHak
Nov 12 at 0:51
However, that got me the idea of something vicious. <paranoid>Let's imagine the RNG is perfectly honest except for the first generation after boot (or after a given amount of RNG idle time) that would be predictable, for example the result of a function taking system time as input. That would pass all those 7GB statistical tests, but still, that first draw would be predictable. Of course that would not be much, but who knows if it could not be exploited in some way ?</paranoid>
– NovHak
Nov 12 at 1:01
There is no way to find out if the RNG is honest. You could be looking at a stream cipher with a known key, for all you know. You can only test if the RNG is behaving itself within the test set (and then conclude that it probably will for the other values as well). This is also why/dev/urandom
only uses the CPU as seed on Linux.
– Maarten Bodewes
Nov 12 at 2:30
Another note: if you extract out the random bit generator then you may be able to test the code without the randomness. Then the code is correct assuming that the random bit source is indeed random.
– Maarten Bodewes
Nov 12 at 2:42
|
show 3 more comments
I've been using the Intel-provided RNG feature for some time, to provide myself with some randomness by means of a C++/CLI program I wrote myself.
However, after some time, something struck me as particularly suspicious. Among other uses, I ask for a random number between 1 and 4 and wrote the result down on paper each time. Here are the results :
2, 3, 3, 2, 1, 3, 4, 2, 3, 2, 3, 1, 3, 2, 3, 1, 2, 4, 2, 2, 1, 2, 1, 3, 1, 3, 3, 3, 3.
Number of 1s : 6
Number of 2s : 9
Number of 3s : 12
Number of 4s : 2
Total : 29
I'm actually wondering if there's a problem either with Intel's RNG, my algorithm, methodology or something else maybe ? Or do you consider the bias not to be significant enough yet ?
I'm using Windows 10 Pro, my CPU is an Intel Core i7-4710MQ.
Compiled with VS2017.
Methodology :
- Start a Powershell command prompt
- Load my assembly with
Add-Type -Path <mydll>
- Invoke
[rdrw.Random]::Next(4)
- Add one to the result
A detail that may be of importance : I don't ask for that number very often, so there's some time between draws and it usually comes when the RNG hasn't been used for some time (one hour at least).
And yes it's a lazy algorithm, I didn't want to bother myself with exceptions.
Algorithm follows :
#include <immintrin.h>
namespace rdrw {
#pragma managed(push,off)
unsigned long long getRdRand() {
unsigned long long val = 0;
while (!_rdrand64_step(&val));
return val;
}
#pragma managed(pop)
public ref class Random abstract sealed
{
public:
// Returns a random 64 bit unsigned integer
static unsigned long long Next() {
return getRdRand();
}
// Return a random unsigned integer between 0 and max-1 (inclusive)
static unsigned long long Next(unsigned long long max) {
unsigned long long nb = max - 1;
unsigned long long mask = 1;
unsigned long long draw = 0;
if (max <= 1)
return 0;
// Create a bitmask that's at least as big as the biggest acceptable value
while ((nb&mask) != nb)
{
mask <<= 1;
mask |= 1;
}
do
{
// Throw unnecessary bits
draw = Next() & mask;
} while (draw>nb);
return draw;
}
// return a random unsigned integer between min and max-1 inclusive
static unsigned long long Next(unsigned long long min, unsigned long long max) {
if (max == min)
return min;
if (max < min)
return 0;
unsigned long long diff = max - min;
return Next(diff) + min;
}
};
}
Thanks for your insights !
random cryptography visual-studio-2017 c++-cli rdrand
I've been using the Intel-provided RNG feature for some time, to provide myself with some randomness by means of a C++/CLI program I wrote myself.
However, after some time, something struck me as particularly suspicious. Among other uses, I ask for a random number between 1 and 4 and wrote the result down on paper each time. Here are the results :
2, 3, 3, 2, 1, 3, 4, 2, 3, 2, 3, 1, 3, 2, 3, 1, 2, 4, 2, 2, 1, 2, 1, 3, 1, 3, 3, 3, 3.
Number of 1s : 6
Number of 2s : 9
Number of 3s : 12
Number of 4s : 2
Total : 29
I'm actually wondering if there's a problem either with Intel's RNG, my algorithm, methodology or something else maybe ? Or do you consider the bias not to be significant enough yet ?
I'm using Windows 10 Pro, my CPU is an Intel Core i7-4710MQ.
Compiled with VS2017.
Methodology :
- Start a Powershell command prompt
- Load my assembly with
Add-Type -Path <mydll>
- Invoke
[rdrw.Random]::Next(4)
- Add one to the result
A detail that may be of importance : I don't ask for that number very often, so there's some time between draws and it usually comes when the RNG hasn't been used for some time (one hour at least).
And yes it's a lazy algorithm, I didn't want to bother myself with exceptions.
Algorithm follows :
#include <immintrin.h>
namespace rdrw {
#pragma managed(push,off)
unsigned long long getRdRand() {
unsigned long long val = 0;
while (!_rdrand64_step(&val));
return val;
}
#pragma managed(pop)
public ref class Random abstract sealed
{
public:
// Returns a random 64 bit unsigned integer
static unsigned long long Next() {
return getRdRand();
}
// Return a random unsigned integer between 0 and max-1 (inclusive)
static unsigned long long Next(unsigned long long max) {
unsigned long long nb = max - 1;
unsigned long long mask = 1;
unsigned long long draw = 0;
if (max <= 1)
return 0;
// Create a bitmask that's at least as big as the biggest acceptable value
while ((nb&mask) != nb)
{
mask <<= 1;
mask |= 1;
}
do
{
// Throw unnecessary bits
draw = Next() & mask;
} while (draw>nb);
return draw;
}
// return a random unsigned integer between min and max-1 inclusive
static unsigned long long Next(unsigned long long min, unsigned long long max) {
if (max == min)
return min;
if (max < min)
return 0;
unsigned long long diff = max - min;
return Next(diff) + min;
}
};
}
Thanks for your insights !
random cryptography visual-studio-2017 c++-cli rdrand
random cryptography visual-studio-2017 c++-cli rdrand
asked Nov 11 at 20:49
NovHak
11
11
3
Pretty far from significant, indeed. Try at least a hundred thousand draws and if the result is still biased to a specific value then there are clear issues. Full random tests from NIST require about 7 GB (max, to run all the tests without repeating random values for the tests). Needless to say, you want to use some kind of table in your application instead of a piece of paper.
– Maarten Bodewes
Nov 11 at 22:02
@MaartenBodewes : I hear you, you must be right and I'll continue assuming there's no bias for now !
– NovHak
Nov 12 at 0:51
However, that got me the idea of something vicious. <paranoid>Let's imagine the RNG is perfectly honest except for the first generation after boot (or after a given amount of RNG idle time) that would be predictable, for example the result of a function taking system time as input. That would pass all those 7GB statistical tests, but still, that first draw would be predictable. Of course that would not be much, but who knows if it could not be exploited in some way ?</paranoid>
– NovHak
Nov 12 at 1:01
There is no way to find out if the RNG is honest. You could be looking at a stream cipher with a known key, for all you know. You can only test if the RNG is behaving itself within the test set (and then conclude that it probably will for the other values as well). This is also why/dev/urandom
only uses the CPU as seed on Linux.
– Maarten Bodewes
Nov 12 at 2:30
Another note: if you extract out the random bit generator then you may be able to test the code without the randomness. Then the code is correct assuming that the random bit source is indeed random.
– Maarten Bodewes
Nov 12 at 2:42
|
show 3 more comments
3
Pretty far from significant, indeed. Try at least a hundred thousand draws and if the result is still biased to a specific value then there are clear issues. Full random tests from NIST require about 7 GB (max, to run all the tests without repeating random values for the tests). Needless to say, you want to use some kind of table in your application instead of a piece of paper.
– Maarten Bodewes
Nov 11 at 22:02
@MaartenBodewes : I hear you, you must be right and I'll continue assuming there's no bias for now !
– NovHak
Nov 12 at 0:51
However, that got me the idea of something vicious. <paranoid>Let's imagine the RNG is perfectly honest except for the first generation after boot (or after a given amount of RNG idle time) that would be predictable, for example the result of a function taking system time as input. That would pass all those 7GB statistical tests, but still, that first draw would be predictable. Of course that would not be much, but who knows if it could not be exploited in some way ?</paranoid>
– NovHak
Nov 12 at 1:01
There is no way to find out if the RNG is honest. You could be looking at a stream cipher with a known key, for all you know. You can only test if the RNG is behaving itself within the test set (and then conclude that it probably will for the other values as well). This is also why/dev/urandom
only uses the CPU as seed on Linux.
– Maarten Bodewes
Nov 12 at 2:30
Another note: if you extract out the random bit generator then you may be able to test the code without the randomness. Then the code is correct assuming that the random bit source is indeed random.
– Maarten Bodewes
Nov 12 at 2:42
3
3
Pretty far from significant, indeed. Try at least a hundred thousand draws and if the result is still biased to a specific value then there are clear issues. Full random tests from NIST require about 7 GB (max, to run all the tests without repeating random values for the tests). Needless to say, you want to use some kind of table in your application instead of a piece of paper.
– Maarten Bodewes
Nov 11 at 22:02
Pretty far from significant, indeed. Try at least a hundred thousand draws and if the result is still biased to a specific value then there are clear issues. Full random tests from NIST require about 7 GB (max, to run all the tests without repeating random values for the tests). Needless to say, you want to use some kind of table in your application instead of a piece of paper.
– Maarten Bodewes
Nov 11 at 22:02
@MaartenBodewes : I hear you, you must be right and I'll continue assuming there's no bias for now !
– NovHak
Nov 12 at 0:51
@MaartenBodewes : I hear you, you must be right and I'll continue assuming there's no bias for now !
– NovHak
Nov 12 at 0:51
However, that got me the idea of something vicious. <paranoid>Let's imagine the RNG is perfectly honest except for the first generation after boot (or after a given amount of RNG idle time) that would be predictable, for example the result of a function taking system time as input. That would pass all those 7GB statistical tests, but still, that first draw would be predictable. Of course that would not be much, but who knows if it could not be exploited in some way ?</paranoid>
– NovHak
Nov 12 at 1:01
However, that got me the idea of something vicious. <paranoid>Let's imagine the RNG is perfectly honest except for the first generation after boot (or after a given amount of RNG idle time) that would be predictable, for example the result of a function taking system time as input. That would pass all those 7GB statistical tests, but still, that first draw would be predictable. Of course that would not be much, but who knows if it could not be exploited in some way ?</paranoid>
– NovHak
Nov 12 at 1:01
There is no way to find out if the RNG is honest. You could be looking at a stream cipher with a known key, for all you know. You can only test if the RNG is behaving itself within the test set (and then conclude that it probably will for the other values as well). This is also why
/dev/urandom
only uses the CPU as seed on Linux.– Maarten Bodewes
Nov 12 at 2:30
There is no way to find out if the RNG is honest. You could be looking at a stream cipher with a known key, for all you know. You can only test if the RNG is behaving itself within the test set (and then conclude that it probably will for the other values as well). This is also why
/dev/urandom
only uses the CPU as seed on Linux.– Maarten Bodewes
Nov 12 at 2:30
Another note: if you extract out the random bit generator then you may be able to test the code without the randomness. Then the code is correct assuming that the random bit source is indeed random.
– Maarten Bodewes
Nov 12 at 2:42
Another note: if you extract out the random bit generator then you may be able to test the code without the randomness. Then the code is correct assuming that the random bit source is indeed random.
– Maarten Bodewes
Nov 12 at 2:42
|
show 3 more comments
active
oldest
votes
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53253092%2fintel-secure-key-rdrand-possibly-strange-behaviour%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
active
oldest
votes
active
oldest
votes
active
oldest
votes
active
oldest
votes
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Some of your past answers have not been well-received, and you're in danger of being blocked from answering.
Please pay close attention to the following guidance:
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53253092%2fintel-secure-key-rdrand-possibly-strange-behaviour%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
3
Pretty far from significant, indeed. Try at least a hundred thousand draws and if the result is still biased to a specific value then there are clear issues. Full random tests from NIST require about 7 GB (max, to run all the tests without repeating random values for the tests). Needless to say, you want to use some kind of table in your application instead of a piece of paper.
– Maarten Bodewes
Nov 11 at 22:02
@MaartenBodewes : I hear you, you must be right and I'll continue assuming there's no bias for now !
– NovHak
Nov 12 at 0:51
However, that got me the idea of something vicious. <paranoid>Let's imagine the RNG is perfectly honest except for the first generation after boot (or after a given amount of RNG idle time) that would be predictable, for example the result of a function taking system time as input. That would pass all those 7GB statistical tests, but still, that first draw would be predictable. Of course that would not be much, but who knows if it could not be exploited in some way ?</paranoid>
– NovHak
Nov 12 at 1:01
There is no way to find out if the RNG is honest. You could be looking at a stream cipher with a known key, for all you know. You can only test if the RNG is behaving itself within the test set (and then conclude that it probably will for the other values as well). This is also why
/dev/urandom
only uses the CPU as seed on Linux.– Maarten Bodewes
Nov 12 at 2:30
Another note: if you extract out the random bit generator then you may be able to test the code without the randomness. Then the code is correct assuming that the random bit source is indeed random.
– Maarten Bodewes
Nov 12 at 2:42