Welcome Guest [Log In] [Register]
Welcome to Crypto. We hope you enjoy your visit.


You're currently viewing our forum as a guest. This means you are limited to certain areas of the board and there are some features you can't use. If you join our community, you'll be able to access member-only sections, and use many member-only features such as customizing your profile, sending personal messages, and voting in polls. Registration is simple, fast, and completely free.


Join our community!


If you're already a member please log in to your account to access all of our features:

Username:   Password:
Add Reply
2D Randomness Visualisation with Python, matplotlib and numpy
Topic Started: Feb 20 2017, 11:12 AM (102 Views)
Karl-Uwe Frank
NSA worthy
[ *  *  *  *  *  * ]
A very basic Python script for 2D visualising randomness of given files
is available here

http://www.freecx.co.uk/crypto/cryptanalysis/Random-Visualisation/

Additionally inside the 2D-Example-Binary-Files folder you may find
resulting images of tests from several different PRNG/cipher algorithms
as well as the binary keystream files processed by the script.

It is a very simple test of which I assume that it should not show any
pattern in the keystream output of a cryptographically secure random
number generator.

Generation of the binary test file is just either piping the keystream
of the cipher into the desired test file or encrypting a prepared file
which consist of only 0x00 and pass the file's location - pure binary
keystream or encrypted file - as parameter to the script.

Nothing fancy so far but might still be helpful somehow.

Cheers,
Karl-Uwe



Below the test results on two binary files generated using two very
unreliable PRNG algorithms as well as the corresponding 2D image.


Binary Keystream File:
http://www.freecx.co.uk/crypto/cryptanalysis/Random-Visualisation/2D-Example-Binary-Files/zero_4MB.bin.crystalline

Resulting 2D-Image File
http://www.freecx.co.uk/crypto/cryptanalysis/Random-Visualisation/2D-Example-Binary-Files/zero_4MB.bin.crystalline.png

Code:
 
========[space]Running[space]Rabbit[space]Test[space]=========

=========[space]Summary[space]results[space]of[space]Rabbit[space]=========

[space]Version:[space][space][space][space][space][space][space][space][space][space]TestU01[space]1.2.3
[space]File:[space][space][space][space][space][space][space][space][space][space][space][space][space]zero_4MB.bin.crystalline
[space]Number[space]of[space]bits:[space][space][space]4194304
[space]Number[space]of[space]statistics:[space][space]38
[space]Total[space]CPU[space]time:[space][space][space]00:00:04.91
[space]The[space]following[space]tests[space]gave[space]p-values[space]outside[space][0.001,[space]0.9990]:
[space](eps[space][space]means[space]a[space]value[space]<[space]1.0e-300):
[space](eps1[space]means[space]a[space]value[space]<[space]1.0e-15):

[space][space][space][space][space][space][space]Test[space][space][space][space][space][space][space][space][space][space][space][space][space][space][space][space][space][space][space][space][space][space][space][space][space][space]p-value
[space]----------------------------------------------
[space][space]1[space][space]MultinomialBitsOver[space][space][space][space][space][space][space][space][space][space][space][space][space][space]eps[space][space]
[space][space]2[space][space]ClosePairsBitMatch,[space]t[space]=[space]2[space][space][space][space][space][space]4.6e-69
[space][space]3[space][space]ClosePairsBitMatch,[space]t[space]=[space]4[space][space][space][space][space]2.5e-147
[space][space]4[space][space]AppearanceSpacings[space][space][space][space][space][space][space][space][space][space][space][space][space][space]6.2e-4
[space][space]6[space][space]LempelZiv[space][space][space][space][space][space][space][space][space][space][space][space][space][space][space][space][space][space][space][space][space][space]1[space]-[space]eps1
[space][space]7[space][space]Fourier1[space][space][space][space][space][space][space][space][space][space][space][space][space][space][space][space][space][space][space][space][space][space][space]1[space]-[space]eps1
[space][space]8[space][space]Fourier3[space][space][space][space][space][space][space][space][space][space][space][space][space][space][space][space][space][space][space][space][space][space]5.6e-148
[space][space]9[space][space]LongestHeadRun[space][space][space][space][space][space][space][space][space][space][space][space][space][space][space][space][space][space]6.6e-5
[space]10[space][space]PeriodsInStrings[space][space][space][space][space][space][space][space][space][space][space][space][space][space][space][space][space]eps[space][space]
[space]11[space][space]HammingWeight[space][space][space][space][space][space][space][space][space][space][space][space][space][space][space][space][space][space][space][space]eps[space][space]
[space]12[space][space]HammingCorr,[space]L[space]=[space]32[space][space][space][space][space][space][space][space][space][space][space][space][space]1.1e-4
[space]14[space][space]HammingCorr,[space]L[space]=[space]128[space][space][space][space][space][space][space][space][space][space][space]1[space]-[space][space]4.4e-5
[space]15[space][space]HammingIndep,[space]L[space]=[space]16[space][space][space][space][space][space][space][space][space][space][space][space][space]eps[space][space]
[space]16[space][space]HammingIndep,[space]L[space]=[space]32[space][space][space][space][space][space][space][space][space][space][space][space][space]eps[space][space]
[space]17[space][space]HammingIndep,[space]L[space]=[space]64[space][space][space][space][space][space][space][space][space][space][space][space][space]eps[space][space]
[space]18[space][space]AutoCor[space][space][space][space][space][space][space][space][space][space][space][space][space][space][space][space][space][space][space][space][space][space][space][space]1[space]-[space][space]3.2e-6
[space]19[space][space]AutoCor[space][space][space][space][space][space][space][space][space][space][space][space][space][space][space][space][space][space][space][space][space][space][space][space]1[space]-[space]5.8e-13
[space]20[space][space]Run[space]of[space]bits[space][space][space][space][space][space][space][space][space][space][space][space][space][space][space][space][space][space][space][space][space][space]eps[space][space]
[space]20[space][space]Run[space]of[space]bits[space][space][space][space][space][space][space][space][space][space][space][space][space][space][space][space][space][space][space][space][space]9.1e-4
[space]21[space][space]MatrixRank,[space]32[space]x[space]32[space][space][space][space][space][space][space][space][space][space][space][space][space]7.6e-4
[space]24[space][space]RandomWalk1[space]H[space][space][space][space][space][space][space][space][space][space][space][space][space][space][space][space][space][space][space][space]eps[space][space]
[space]24[space][space]RandomWalk1[space]M[space][space][space][space][space][space][space][space][space][space][space][space][space][space][space][space][space][space][space][space]eps[space][space]
[space]24[space][space]RandomWalk1[space]J[space][space][space][space][space][space][space][space][space][space][space][space][space][space][space][space][space][space][space][space]eps[space][space]
[space]24[space][space]RandomWalk1[space]R[space][space][space][space][space][space][space][space][space][space][space][space][space][space][space][space][space][space][space][space]eps[space][space]
[space]24[space][space]RandomWalk1[space]C[space][space][space][space][space][space][space][space][space][space][space][space][space][space][space][space][space][space][space][space]eps[space][space]
[space]25[space][space]RandomWalk1[space]H[space](L[space]=[space]1024)[space][space][space][space][space][space][space][space][space]eps[space][space]
[space]25[space][space]RandomWalk1[space]M[space](L[space]=[space]1024)[space][space][space][space][space][space][space][space][space]eps[space][space]
[space]25[space][space]RandomWalk1[space]J[space](L[space]=[space]1024)[space][space][space][space][space][space][space][space][space]eps[space][space]
[space]25[space][space]RandomWalk1[space]R[space](L[space]=[space]1024)[space][space][space][space][space][space][space][space][space]eps[space][space]
[space]25[space][space]RandomWalk1[space]C[space](L[space]=[space]1024)[space][space][space][space][space][space][space][space][space]eps[space][space]
[space]26[space][space]RandomWalk1[space]H[space](L[space]=[space]10016)[space][space][space][space][space][space][space][space]eps[space][space]
[space]26[space][space]RandomWalk1[space]M[space](L[space]=[space]10016)[space][space][space][space][space][space][space][space]eps[space][space]
[space]26[space][space]RandomWalk1[space]J[space](L[space]=[space]10016)[space][space][space][space][space][space]6.9e-10
[space]----------------------------------------------
[space]All[space]other[space]tests[space]were[space]passed

*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*

========[space]Running[space]Alphabit[space]Test[space]========

=========[space]Summary[space]results[space]of[space]Alphabit[space]=========

[space]Version:[space][space][space][space][space][space][space][space][space][space]TestU01[space]1.2.3
[space]File:[space][space][space][space][space][space][space][space][space][space][space][space][space]zero_4MB.bin.crystalline
[space]Number[space]of[space]bits:[space][space][space]4194304
[space]Number[space]of[space]statistics:[space][space]17
[space]Total[space]CPU[space]time:[space][space][space]00:00:00.32
[space]The[space]following[space]tests[space]gave[space]p-values[space]outside[space][0.001,[space]0.9990]:
[space](eps[space][space]means[space]a[space]value[space]<[space]1.0e-300):
[space](eps1[space]means[space]a[space]value[space]<[space]1.0e-15):

[space][space][space][space][space][space][space]Test[space][space][space][space][space][space][space][space][space][space][space][space][space][space][space][space][space][space][space][space][space][space][space][space][space][space]p-value
[space]----------------------------------------------
[space][space]1[space][space]MultinomialBitsOver,[space]L[space]=[space]2[space][space][space][space][space][space][space]eps[space][space]
[space][space]2[space][space]MultinomialBitsOver,[space]L[space]=[space]4[space][space][space][space][space][space][space]eps[space][space]
[space][space]3[space][space]MultinomialBitsOver,[space]L[space]=[space]8[space][space][space][space][space][space][space]eps[space][space]
[space][space]4[space][space]MultinomialBitsOver,[space]L[space]=[space]16[space][space][space][space][space][space]eps[space][space]
[space][space]5[space][space]HammingIndep,[space]L[space]=[space]16[space][space][space][space][space][space][space][space][space][space][space][space][space]eps[space][space]
[space][space]6[space][space]HammingIndep,[space]L[space]=[space]32[space][space][space][space][space][space][space][space][space][space][space][space][space]eps[space][space]
[space][space]7[space][space]HammingCorr,[space]L[space]=[space]32[space][space][space][space][space][space][space][space][space][space][space][space][space]1.1e-4
[space][space]8[space][space]RandomWalk1[space]H[space](L[space]=[space]64)[space][space][space][space][space][space][space][space][space][space][space]eps[space][space]
[space][space]8[space][space]RandomWalk1[space]M[space](L[space]=[space]64)[space][space][space][space][space][space][space][space][space][space][space]eps[space][space]
[space][space]8[space][space]RandomWalk1[space]J[space](L[space]=[space]64)[space][space][space][space][space][space][space][space][space][space][space]eps[space][space]
[space][space]8[space][space]RandomWalk1[space]R[space](L[space]=[space]64)[space][space][space][space][space][space][space][space][space][space][space]eps[space][space]
[space][space]8[space][space]RandomWalk1[space]C[space](L[space]=[space]64)[space][space][space][space][space][space][space][space][space][space][space]eps[space][space]
[space][space]9[space][space]RandomWalk1[space]H[space](L[space]=[space]320)[space][space][space][space][space][space][space][space][space][space]eps[space][space]
[space][space]9[space][space]RandomWalk1[space]M[space](L[space]=[space]320)[space][space][space][space][space][space][space][space][space][space]eps[space][space]
[space][space]9[space][space]RandomWalk1[space]J[space](L[space]=[space]320)[space][space][space][space][space][space][space][space]1.1e-16
[space][space]9[space][space]RandomWalk1[space]R[space](L[space]=[space]320)[space][space][space][space][space][space][space][space][space]4.3e-4
[space]----------------------------------------------
[space]All[space]other[space]tests[space]were[space]passed[space]



Binary Keystream File:
http://www.freecx.co.uk/crypto/cryptanalysis/Random-Visualisation/2D-Example-Binary-Files/badRNG_4MB.bin

Resulting 2D-Image File
http://www.freecx.co.uk/crypto/cryptanalysis/Random-Visualisation/2D-Example-Binary-Files/badRNG_4MB.bin.png

Code:
 
========[space]Running[space]Rabbit[space]Test[space]=========

=========[space]Summary[space]results[space]of[space]Rabbit[space]=========

[space]Version:[space][space][space][space][space][space][space][space][space][space]TestU01[space]1.2.3
[space]File:[space][space][space][space][space][space][space][space][space][space][space][space][space]badRNG_4MB.bin
[space]Number[space]of[space]bits:[space][space][space]4194304
[space]Number[space]of[space]statistics:[space][space]38
[space]Total[space]CPU[space]time:[space][space][space]00:00:05.59
[space]The[space]following[space]tests[space]gave[space]p-values[space]outside[space][0.001,[space]0.9990]:
[space](eps[space][space]means[space]a[space]value[space]<[space]1.0e-300):
[space](eps1[space]means[space]a[space]value[space]<[space]1.0e-15):

[space][space][space][space][space][space][space]Test[space][space][space][space][space][space][space][space][space][space][space][space][space][space][space][space][space][space][space][space][space][space][space][space][space][space]p-value
[space]----------------------------------------------
[space][space]1[space][space]MultinomialBitsOver[space][space][space][space][space][space][space][space][space][space][space]4.5e-100
[space][space]2[space][space]ClosePairsBitMatch,[space]t[space]=[space]2[space][space][space][space][space][space][space]4.9e-4
[space][space]3[space][space]ClosePairsBitMatch,[space]t[space]=[space]4[space][space][space][space][space][space]1.1e-13
[space][space]4[space][space]AppearanceSpacings[space][space][space][space][space][space][space][space][space][space][space][space][space][space][space]eps[space][space]
[space][space]7[space][space]Fourier1[space][space][space][space][space][space][space][space][space][space][space][space][space][space][space][space][space][space][space][space][space][space][space][space][space]eps[space][space]
[space][space]8[space][space]Fourier3[space][space][space][space][space][space][space][space][space][space][space][space][space][space][space][space][space][space][space][space][space][space][space][space][space]eps[space][space]
[space]12[space][space]HammingCorr,[space]L[space]=[space]32[space][space][space][space][space][space][space][space][space][space][space][space][space][space]eps[space][space]
[space]13[space][space]HammingCorr,[space]L[space]=[space]64[space][space][space][space][space][space][space][space][space][space][space][space][space][space]eps[space][space]
[space]14[space][space]HammingCorr,[space]L[space]=[space]128[space][space][space][space][space][space][space][space][space][space][space][space][space]eps[space][space]
[space]16[space][space]HammingIndep,[space]L[space]=[space]32[space][space][space][space][space][space][space][space][space][space][space][space][space]eps[space][space]
[space]17[space][space]HammingIndep,[space]L[space]=[space]64[space][space][space][space][space][space][space][space][space][space][space][space][space]eps[space][space]
[space]24[space][space]RandomWalk1[space]H[space][space][space][space][space][space][space][space][space][space][space][space][space][space][space][space][space][space][space][space]eps[space][space]
[space]24[space][space]RandomWalk1[space]M[space][space][space][space][space][space][space][space][space][space][space][space][space][space][space][space][space][space][space][space]eps[space][space]
[space]24[space][space]RandomWalk1[space]J[space][space][space][space][space][space][space][space][space][space][space][space][space][space][space][space][space][space][space][space]eps[space][space]
[space]24[space][space]RandomWalk1[space]R[space][space][space][space][space][space][space][space][space][space][space][space][space][space][space][space][space][space][space][space]eps[space][space]
[space]24[space][space]RandomWalk1[space]C[space][space][space][space][space][space][space][space][space][space][space][space][space][space][space][space][space][space][space][space]eps[space][space]
[space]25[space][space]RandomWalk1[space]H[space](L[space]=[space]1024)[space][space][space][space][space][space][space][space][space]eps[space][space]
[space]25[space][space]RandomWalk1[space]M[space](L[space]=[space]1024)[space][space][space][space][space][space][space][space][space]eps[space][space]
[space]25[space][space]RandomWalk1[space]J[space](L[space]=[space]1024)[space][space][space][space][space][space][space][space][space]eps[space][space]
[space]25[space][space]RandomWalk1[space]R[space](L[space]=[space]1024)[space][space][space][space][space][space][space][space][space]eps[space][space]
[space]25[space][space]RandomWalk1[space]C[space](L[space]=[space]1024)[space][space][space][space][space][space][space][space][space]eps[space][space]
[space]26[space][space]RandomWalk1[space]H[space](L[space]=[space]10016)[space][space][space][space][space][space][space]4.5e-6
[space]26[space][space]RandomWalk1[space]M[space](L[space]=[space]10016)[space][space][space][space][space][space][space]1.9e-4
[space]----------------------------------------------
[space]All[space]other[space]tests[space]were[space]passed

*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*

========[space]Running[space]Alphabit[space]Test[space]========

=========[space]Summary[space]results[space]of[space]Alphabit[space]=========

[space]Version:[space][space][space][space][space][space][space][space][space][space]TestU01[space]1.2.3
[space]File:[space][space][space][space][space][space][space][space][space][space][space][space][space]badRNG_4MB.bin
[space]Number[space]of[space]bits:[space][space][space]4194304
[space]Number[space]of[space]statistics:[space][space]17
[space]Total[space]CPU[space]time:[space][space][space]00:00:00.33
[space]The[space]following[space]tests[space]gave[space]p-values[space]outside[space][0.001,[space]0.9990]:
[space](eps[space][space]means[space]a[space]value[space]<[space]1.0e-300):
[space](eps1[space]means[space]a[space]value[space]<[space]1.0e-15):

[space][space][space][space][space][space][space]Test[space][space][space][space][space][space][space][space][space][space][space][space][space][space][space][space][space][space][space][space][space][space][space][space][space][space]p-value
[space]----------------------------------------------
[space][space]4[space][space]MultinomialBitsOver,[space]L[space]=[space]16[space][space][space][space]1[space]-[space]2.8e-11
[space][space]6[space][space]HammingIndep,[space]L[space]=[space]32[space][space][space][space][space][space][space][space][space][space][space][space][space]eps[space][space]
[space][space]7[space][space]HammingCorr,[space]L[space]=[space]32[space][space][space][space][space][space][space][space][space][space][space][space][space][space]eps[space][space]
[space][space]8[space][space]RandomWalk1[space]H[space](L[space]=[space]64)[space][space][space][space][space][space][space][space][space][space][space]eps[space][space]
[space][space]8[space][space]RandomWalk1[space]M[space](L[space]=[space]64)[space][space][space][space][space][space][space][space][space][space][space]eps[space][space]
[space][space]8[space][space]RandomWalk1[space]J[space](L[space]=[space]64)[space][space][space][space][space][space][space][space][space][space][space]eps[space][space]
[space][space]8[space][space]RandomWalk1[space]R[space](L[space]=[space]64)[space][space][space][space][space][space][space][space][space][space][space]eps[space][space]
[space][space]8[space][space]RandomWalk1[space]C[space](L[space]=[space]64)[space][space][space][space][space][space][space][space][space][space][space]eps[space][space]
[space][space]9[space][space]RandomWalk1[space]H[space](L[space]=[space]320)[space][space][space][space][space][space][space][space][space][space]eps[space][space]
[space][space]9[space][space]RandomWalk1[space]M[space](L[space]=[space]320)[space][space][space][space][space][space][space][space][space][space]eps[space][space]
[space][space]9[space][space]RandomWalk1[space]J[space](L[space]=[space]320)[space][space][space][space][space][space][space][space][space][space]eps[space][space]
[space][space]9[space][space]RandomWalk1[space]R[space](L[space]=[space]320)[space][space][space][space][space][space][space][space][space][space]eps[space][space]
[space][space]9[space][space]RandomWalk1[space]C[space](L[space]=[space]320)[space][space][space][space][space][space][space][space][space][space]eps[space][space]
[space]----------------------------------------------
[space]All[space]other[space]tests[space]were[space]passed



Edit: the inevitable typo :-)
Edited by Karl-Uwe Frank, Feb 20 2017, 11:23 AM.
cHNiMUBACG0HAAAAAAAAAAAAAABIZVbDdKVM0w1kM9vxQHw+bkLxsY/Z0czY0uv8/Ks6WULxJVua
zjvpoYvtEwDVhP7RGTCBVlzZ+VBWPHg5rqmKWvtzsuVmMSDxAIS6Db6YhtzT+RStzoG9ForBcG8k
G97Q3Jml/aBun8Kyf+XOBHpl5gNW4YqhiM0=
Offline Profile Quote Post Goto Top
 
1 user reading this topic (1 Guest and 0 Anonymous)
« Previous Topic · Utilities · Next Topic »
Add Reply