| Welcome to Crypto. We hope you enjoy your visit. You're currently viewing our forum as a guest. This means you are limited to certain areas of the board and there are some features you can't use. If you join our community, you'll be able to access member-only sections, and use many member-only features such as customizing your profile, sending personal messages, and voting in polls. Registration is simple, fast, and completely free. Join our community! If you're already a member please log in to your account to access all of our features: |
- Pages:
- 1
- 2
| pride.crypto.metrics; python tools for statistical analysis of crypto | |
|---|---|
| Tweet Topic Started: Apr 21 2016, 12:21 AM (460 Views) | |
| E. Rose | Apr 21 2016, 12:21 AM Post #1 |
|
Member
![]() ![]() ![]() ![]()
|
I develop a python module called pride.crypto.metrics. It contains various tests for various types of cryptographic primitives, such as hash functions and block ciphers. I have found the tools invaluable when researching my own algorithms. I am putting it here in the hope that someone else may find it useful as well. Tests include:
The code is available on github here: https://github.com/erose1337/pride/blob/master/crypto/metrics.py and is being attached as a zip to this post. You should not have to install the complete pride package just to use pride.crypto.metrics; The script should function as a standalone, save for the demo test of AES/random data. Testing of random data does require ent; ent should be located in the same directory the script is run from. Let me know if there's any problems, if you need help using it, or if you find it helpful. It could be documented a little better, but it's pretty straightforward to use. Edited by E. Rose, Apr 21 2016, 12:23 AM.
|
![]() |
|
| Replies: | |
|---|---|
| Karl-Uwe Frank | Apr 28 2016, 12:37 AM Post #16 |
|
NSA worthy
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]()
|
Great! Thanks again, it straightens the process firmly. Your test routine seems to be a fine tool for measuring hash output quality. Just one suggestion which I found in diehard and the TestU01 suites so useful is, that you maybe include some common hash algorithm to choose from, enabling someone who is testing his own algorithm to verify his results against those of some common hash of checksum functions. |
|
cHNiMUBACG0HAAAAAAAAAAAAAABIZVbDdKVM0w1kM9vxQHw+bkLxsY/Z0czY0uv8/Ks6WULxJVua zjvpoYvtEwDVhP7RGTCBVlzZ+VBWPHg5rqmKWvtzsuVmMSDxAIS6Db6YhtzT+RStzoG9ForBcG8k G97Q3Jml/aBun8Kyf+XOBHpl5gNW4YqhiM0= | |
![]() |
|
| E. Rose | Apr 28 2016, 01:15 AM Post #17 |
|
Member
![]() ![]() ![]() ![]()
|
Well thank you! When run as a module, metrics.py will run metrics on sha256 and on the random oracle hash function. This provides examples results that a "real" hash function would produce, along with the metrics of a truly random function. Oh, and I cleaned up the support for block ciphers/stream ciphers; The updated version on github supports test_block_cipher and test_stream_cipher functions too. Those methods accept an encrypt method, along with a key and iv/nonce; plaintext data/ciphertext data is supplied/created by the tests. Edited by E. Rose, Apr 28 2016, 01:16 AM.
|
![]() |
|
| Karl-Uwe Frank | Apr 28 2016, 11:01 AM Post #18 |
|
NSA worthy
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]()
|
Just added two 32 bit hashes. Do you consider these results below correct?
|
|
cHNiMUBACG0HAAAAAAAAAAAAAABIZVbDdKVM0w1kM9vxQHw+bkLxsY/Z0czY0uv8/Ks6WULxJVua zjvpoYvtEwDVhP7RGTCBVlzZ+VBWPHg5rqmKWvtzsuVmMSDxAIS6Db6YhtzT+RStzoG9ForBcG8k G97Q3Jml/aBun8Kyf+XOBHpl5gNW4YqhiM0= | |
![]() |
|
| E. Rose | Apr 28 2016, 03:46 PM Post #19 |
|
Member
![]() ![]() ![]() ![]()
|
The ent test looks a little weird; It says all values are equal in the serial correlation section. The period test looks like it is suffering from the same problem somehow. Otherwise it looks appropriate. Let me take a look at what might be happening with the ent/period test problem. |
![]() |
|
| Karl-Uwe Frank | Apr 28 2016, 06:11 PM Post #20 |
|
NSA worthy
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]()
|
If you look at the Test_Data_{}.bin files you will realise that they are filled with all 0x0. By the way, it might be useful in my opinion either to delete these binary test files after the test run or rename same in a way that they correlate with the hash which generated them. Edited by Karl-Uwe Frank, Apr 28 2016, 06:11 PM.
|
|
cHNiMUBACG0HAAAAAAAAAAAAAABIZVbDdKVM0w1kM9vxQHw+bkLxsY/Z0czY0uv8/Ks6WULxJVua zjvpoYvtEwDVhP7RGTCBVlzZ+VBWPHg5rqmKWvtzsuVmMSDxAIS6Db6YhtzT+RStzoG9ForBcG8k G97Q3Jml/aBun8Kyf+XOBHpl5gNW4YqhiM0= | |
![]() |
|
| E. Rose | May 1 2016, 12:05 AM Post #21 |
|
Member
![]() ![]() ![]() ![]()
|
I pushed an updated version to github. I incorporated your feature request and the files that hold random data for testing are now deleted at the end, and hopefully fixed that bug. |
![]() |
|
| Karl-Uwe Frank | May 1 2016, 09:59 PM Post #22 |
|
NSA worthy
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]()
|
Still Test_Data.bin contains all zero byte. |
|
cHNiMUBACG0HAAAAAAAAAAAAAABIZVbDdKVM0w1kM9vxQHw+bkLxsY/Z0czY0uv8/Ks6WULxJVua zjvpoYvtEwDVhP7RGTCBVlzZ+VBWPHg5rqmKWvtzsuVmMSDxAIS6Db6YhtzT+RStzoG9ForBcG8k G97Q3Jml/aBun8Kyf+XOBHpl5gNW4YqhiM0= | |
![]() |
|
| E. Rose | May 1 2016, 11:17 PM Post #23 |
|
Member
![]() ![]() ![]() ![]()
|
The problem is being caused because of how test data is generated. Data is generated via a hash chain. The hash chain begins with the hash of null bytes, and the output becomes the next input. Both the hashes that you are testing with do not work on null strings. Applying either of them to a string of null bytes always returns a string of null bytes. For example:
When supplied "\x00", "\x00\x00", and "\x00" * 100, sha256 produces three distinct outputs. MurmurOAAT produces "\x00\x00\x00\x00" for all three inputs. This is what was causing your test data files to be filled with all zeros. I can change the test to start with the hash of "\x01", which will fix the random data not being generated correctly. I'm also going to add an additional, separate test to detect this feature badhash/MurmurOAAT both happen to have, because it wouldn't have showed up if the initial hash weren't "\x00", and I feel it's something worth noting. The updated version is on now github; I tested it with MurmurOAAT and it produces the following results:
Thanks for your patience! Edited by E. Rose, May 1 2016, 11:17 PM.
|
![]() |
|
| Karl-Uwe Frank | May 2 2016, 11:07 AM Post #24 |
|
NSA worthy
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]()
|
Well, now it throws the error File "./pride.crypto/metrics.py", line 8, in <module> from utilities import slide ImportError: No module named utilities |
|
cHNiMUBACG0HAAAAAAAAAAAAAABIZVbDdKVM0w1kM9vxQHw+bkLxsY/Z0czY0uv8/Ks6WULxJVua zjvpoYvtEwDVhP7RGTCBVlzZ+VBWPHg5rqmKWvtzsuVmMSDxAIS6Db6YhtzT+RStzoG9ForBcG8k G97Q3Jml/aBun8Kyf+XOBHpl5gNW4YqhiM0= | |
![]() |
|
| E. Rose | May 2 2016, 04:04 PM Post #25 |
|
Member
![]() ![]() ![]() ![]()
|
My apologies. I was refactoring my code and moved that. The updated version has it fixed. Sorry it's taken so long to sort this out! |
![]() |
|
| Karl-Uwe Frank | May 2 2016, 10:48 PM Post #26 |
|
NSA worthy
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]()
|
No need to say sorry. You should take your time and you fixed it.These test result look reliable to me. |
|
cHNiMUBACG0HAAAAAAAAAAAAAABIZVbDdKVM0w1kM9vxQHw+bkLxsY/Z0czY0uv8/Ks6WULxJVua zjvpoYvtEwDVhP7RGTCBVlzZ+VBWPHg5rqmKWvtzsuVmMSDxAIS6Db6YhtzT+RStzoG9ForBcG8k G97Q3Jml/aBun8Kyf+XOBHpl5gNW4YqhiM0= | |
![]() |
|
| 1 user reading this topic (1 Guest and 0 Anonymous) | |
| « Previous Topic · Utilities · Next Topic » |
- Pages:
- 1
- 2





![]](http://z2.ifrm.com/static/1/pip_r.png)
metrics.py (9.48 KB)




12:32 AM Jul 11