| Welcome to Crypto. We hope you enjoy your visit. You're currently viewing our forum as a guest. This means you are limited to certain areas of the board and there are some features you can't use. If you join our community, you'll be able to access member-only sections, and use many member-only features such as customizing your profile, sending personal messages, and voting in polls. Registration is simple, fast, and completely free. Join our community! If you're already a member please log in to your account to access all of our features: |
| Brute Forcing The Adfgvx Cipher | |
|---|---|
| Topic Started: Apr 19 2008, 08:06 AM (2,745 Views) | |
| Revelation | Apr 19 2008, 08:06 AM Post #1 |
|
Administrator
![]() ![]() ![]() ![]() ![]() ![]() ![]()
|
The ADFGVX cipher is a pretty good cipher: it's a pain to crack manually and brute force probably takes a very long time. But there must be a way to make a smarter brute force. I'm thinking about quick guesses. If the F appears a lot in the ciphertext, it means the row and/or the column with F is a row with letters with high frequencies. So the chances of having a number in the F row and column is not that high. Futhermore, if you've guessed the keylength and column order you can do a frequency check. Have you got any more ideas to improve the brute forcer? |
|
RRRREJMEEEEEPVKLWENFNVJKEEEEEAOLKAFKLXCFZAASDJXZTTTTTTTLSIOWJXMOKLAFJNNKFNXN RAGRBAQEMHIGDJVDSEOXVIYCELFHWLELJFIENXLRATALSJFSLCYTKLASJDKMHGOVOKAJDNMNUITN RRRRLJVEEEEECLYVYHNVPFTAEEEEEMWLMEIRNGLARWJAKJDFLWNTIERJMIPQWOTZEOCXKNUBNXCN RJIRPOWEANFUSNCZVDVZNMSFEKLOEPZLDKDJWSAAAAAAAOERHJCTNCKFRIMVKSOFOMKMANREWNBN RZUDRGXEEEEENFQIDVLQNCKNEEEEEDGLLLLLLAWIOSNCDARLODMTOEJXMILDFJROTKJSDNLVCZNN | |
![]() |
|
| jdege | Apr 19 2008, 12:53 PM Post #2 |
|
Elite member
![]() ![]() ![]() ![]() ![]() ![]() ![]()
|
I'd think the keyspace of ADFGVX is far too large for brute-forcing to have any chance. What I'm wondering if there are any statistical tests that would determine whether you had successfully reversed the transposition. It's a two-stage cipher, polybius-square substitution followed by columnar transposition. Is there a test we can perform on a trial solution of the transposition, that would determine we had it correct, without having had to break the substitution first? I'd be very surprised if there were not, but I don't know what it would be. Remember - this was cracked, in pre-computer days. But only occasionally, in times of high traffic. My guess is that they attacked the transposition by simultaneous anagramming of messages of identical size. |
| When cryptography is outlawed, bayl bhgynjf jvyy unir cevinpl. | |
![]() |
|
| Revelation | Apr 22 2008, 02:32 PM Post #3 |
|
Administrator
![]() ![]() ![]() ![]() ![]() ![]() ![]()
|
Well, we could check if the letter frequencies match the ones of standard English when we've done a transposition. That might help. We could also run a quick pattern check on the bigrams to see if a given crib is found. |
|
RRRREJMEEEEEPVKLWENFNVJKEEEEEAOLKAFKLXCFZAASDJXZTTTTTTTLSIOWJXMOKLAFJNNKFNXN RAGRBAQEMHIGDJVDSEOXVIYCELFHWLELJFIENXLRATALSJFSLCYTKLASJDKMHGOVOKAJDNMNUITN RRRRLJVEEEEECLYVYHNVPFTAEEEEEMWLMEIRNGLARWJAKJDFLWNTIERJMIPQWOTZEOCXKNUBNXCN RJIRPOWEANFUSNCZVDVZNMSFEKLOEPZLDKDJWSAAAAAAAOERHJCTNCKFRIMVKSOFOMKMANREWNBN RZUDRGXEEEEENFQIDVLQNCKNEEEEEDGLLLLLLAWIOSNCDARLODMTOEJXMILDFJROTKJSDNLVCZNN | |
![]() |
|
| jdege | Apr 22 2008, 02:53 PM Post #4 |
|
Elite member
![]() ![]() ![]() ![]() ![]() ![]() ![]()
|
From Wikipedia:
Aegean Park has a book, "General Solution of the ADFGVX Cipher System," by J. Rives Childs, that would be interesting. Friedman's Military Cryptanalysis IV also discusses it. But it's so much more fun to try to figure out a break on your own, before you read about how others have done it. |
| When cryptography is outlawed, bayl bhgynjf jvyy unir cevinpl. | |
![]() |
|
| Donald | Apr 28 2008, 02:37 PM Post #5 |
|
Elite member
![]() ![]() ![]() ![]() ![]() ![]() ![]()
|
I've always found the ADFGVX cipher to be one of the most intimidating pen and paper ciphers around. Which is why I used a variation of it in my impractical cipher. Fractionated ciphers just seem impossible to me. Especially if you go to the next step and recombine the transposed fractionated message back into single letters. I recognize that they are NOT impossible, but I can't figure out how to even get a theoretical toe hold on them. It's well beyond me. In the above referenced thread rot13 mentioned trying to attack it by tracing column and row coincidences. BUT, I'm certain I couldn't make that work. Not yet anyway. And I note, NO ONE really made attempt on my challenge, even the known plain text one. ADFGVX is quite a cipher. |
![]() |
|
| Revelation | Apr 28 2008, 03:49 PM Post #6 |
|
Administrator
![]() ![]() ![]() ![]() ![]() ![]() ![]()
|
Well, I still think that my pattern sweep can tell whether you've got a good transposition or not. A transposition with keylength < 10 is cracked in about 10 seconds. That means that the pattern sweep takes about 15 seconds. That's do-able. When you've got a pattern match, solve the mono sub using frequencies. In theory it sounds like the ADFGVX can be brute forced in an acceptable amount of time. I want to make a C application that does all of the above. Since most of us are programmers, we can find ways to optimise it.
Well, we all hate fractionating systems, for obvious reasons I'll have a look at it again when we have brute forced the ADFGVX cipher. |
|
RRRREJMEEEEEPVKLWENFNVJKEEEEEAOLKAFKLXCFZAASDJXZTTTTTTTLSIOWJXMOKLAFJNNKFNXN RAGRBAQEMHIGDJVDSEOXVIYCELFHWLELJFIENXLRATALSJFSLCYTKLASJDKMHGOVOKAJDNMNUITN RRRRLJVEEEEECLYVYHNVPFTAEEEEEMWLMEIRNGLARWJAKJDFLWNTIERJMIPQWOTZEOCXKNUBNXCN RJIRPOWEANFUSNCZVDVZNMSFEKLOEPZLDKDJWSAAAAAAAOERHJCTNCKFRIMVKSOFOMKMANREWNBN RZUDRGXEEEEENFQIDVLQNCKNEEEEEDGLLLLLLAWIOSNCDARLODMTOEJXMILDFJROTKJSDNLVCZNN | |
![]() |
|
| jdege | Apr 28 2008, 04:27 PM Post #7 |
|
Elite member
![]() ![]() ![]() ![]() ![]() ![]() ![]()
|
IFF you have a good test for whether the result of a given trial transposition looks like the output of the substitution phase. The first step is a fractionated substitution, creating two letters for one. But it's a simple substitution, and a digraph frequency count would show a normal language distribution. For that matter, converting it back to single characters using any arbitrary Polybius square would yield a result that was only a simple substitution away from the plaintext. So there's no question that there are statistical measures that distinguish the output of the substitution phase. The question is whether the output of the trial transpositions are statistically distinct. It's clear to me that most of them will be. The majority of incorrect transpositions will yield something like a random shuffle, that would result in a digraph frequency distribution very different from plain text. But I'd not feel in comfortable in stating that all of the incorrect transpositions will be so. It seems perfectly possible to me that some number of incorrect transpositions will look correct. Whether this is true, or how many of them there are, I'd not even pretend to say, but it seems to me that the number of transpositions that could be correct is the major driver as to whether this is a viable approach. Of course, given multiple messages encoded with the same key, things are very different. While it might be possible that a number of incorrect transpositions yield normal-looking frequency distributions, the odds that the same incorrect transposition would yield normal distributions are very low. Still, you've been playing around with this. How many false positives have you been seeing? |
| When cryptography is outlawed, bayl bhgynjf jvyy unir cevinpl. | |
![]() |
|
| Donald | Apr 28 2008, 06:58 PM Post #8 |
|
Elite member
![]() ![]() ![]() ![]() ![]() ![]() ![]()
|
AH! Ok, it took me a bit, but I think I see your point. If the sequence was the standard step 1: fractionate, step 2: transpose Then we just try various transpositions until our digram frequency looks like a mono substitution frequency. And that undoes step 2. From there, any polybius square will convert the remaining crypt text into a mono alphabetic substitution, which is easy to solve. Brilliant! And NICE!!!! BUT, If the sequence is: step 1: fractionate, step 2: transpose step 3: unfractionate THIS still baffles me. How do you undo step 3? All of your results should look more or less like random data. If you don't use the correct square, you get a digraph substitution of the original step 2 data, and the original step 2 data is scrambled to start with. I don't see any way to detect that you have gotten closer to (or hit) the correct step 2 text, nor do I see any way to use an incorrect digraph substitution to your advantage. Because the connection between the step 2 data and the original plain text is involved in the relationships between the letters positions in the fractionating step that happened in step 1. It's the relative positions. Right? an incorrect digraph substitution of step 2's output doesn't give us the same cipher, but with a different square for step 1. It gives us the wrong relationships between the letters, and therefore a completely invalid and useless result. Of course, as long as the relationships between the letters used in the squares in step 1 and step 3 are the same, they do not need to be the exact same squares. But that should still eliminate all but a tiny fraction of the keyspace as useless during a brute force attack. With a playfair you can use shotgun hill climbing. Just decrypt the cipher text with a random square and then make changes to that square keep the relationships that make the decrypted text look more and more like ordinary plain text. I only understand this approach on the most theoretical level, but I can see how it works sort of. BUT, with the 3 step ADFGVX I don't see how shotgun hill climbing would work because the intermediate text does not show digram or mono text frequencies. You have to undo the transposition to get something with normal digram frequencies. Which makes a brute force attack very impractical. |
![]() |
|
| Revelation | Apr 28 2008, 07:09 PM Post #9 |
|
Administrator
![]() ![]() ![]() ![]() ![]() ![]() ![]()
|
That's what I suggested in my second and third post and that's what my application would do: first sweeping the pattern of a crib through the text and then checking the frequency of the bigrams (btw, I haven't started writing it yet) Your other sequence is a pain, especially if you use two different squares in step 1 and 3. I'm not sure how to solve that. |
|
RRRREJMEEEEEPVKLWENFNVJKEEEEEAOLKAFKLXCFZAASDJXZTTTTTTTLSIOWJXMOKLAFJNNKFNXN RAGRBAQEMHIGDJVDSEOXVIYCELFHWLELJFIENXLRATALSJFSLCYTKLASJDKMHGOVOKAJDNMNUITN RRRRLJVEEEEECLYVYHNVPFTAEEEEEMWLMEIRNGLARWJAKJDFLWNTIERJMIPQWOTZEOCXKNUBNXCN RJIRPOWEANFUSNCZVDVZNMSFEKLOEPZLDKDJWSAAAAAAAOERHJCTNCKFRIMVKSOFOMKMANREWNBN RZUDRGXEEEEENFQIDVLQNCKNEEEEEDGLLLLLLAWIOSNCDARLODMTOEJXMILDFJROTKJSDNLVCZNN | |
![]() |
|
| jdege | Apr 28 2008, 09:41 PM Post #10 |
|
Elite member
![]() ![]() ![]() ![]() ![]() ![]() ![]()
|
Can I get by with "ADFGVX doesn't unfractionate, do it doesn't matter"? Didn't think so. Now if we're doing Bifid, with an even key-length, it doesn't matter, either, because then the two halves of each letter map to the two halfs of another letter:
Notice how the first letter of the ciphertext is constructed from the left halves of the first and second letters of the plaintext, and how the third letter of the ciphertext is constructed from the right halves of the first and second letters of the plaintext. In other words, with a key-length of four, the first and second letters of the plaintext entirely determine the first and third letters of the ciphertext, and the second and fourth letters of the plaintext entirely determine the second and fourth letters of the ciphertext. It can be solved as an ordinary digram substitution. With an odd key-length, this doesn't work. How to solve odd-length Bifids? I've a couple of explanations around here, but I've not studied them. (And the explanations aren't as straightforward as the even-length Bifid, so my fast skims haven't yielded anything useful.) As for the generic problem, with an unknown transposition? Not a clue. |
| When cryptography is outlawed, bayl bhgynjf jvyy unir cevinpl. | |
![]() |
|
| jdege | Apr 29 2008, 03:42 PM Post #11 |
|
Elite member
![]() ![]() ![]() ![]() ![]() ![]() ![]()
|
And you were a step or two ahead of us. I was just figuring it out.
When I think about it, I'm always driven back to the same question - how can we recognize when we've properly reversed one step? Fractionate+transpose+unfractionate. Is there something we can recognize in a fractionate+transpose, that would allow us to determine that we have properly reversed the unfractionate? If so, does a not-quite-there attempt at unfractionating look more like a fully-there attempt at unfractionating? If so, then the heuristic methods should work - shotgun hill climbing, simulated annealing, genetic algorithms. But it'd take an in-depth look at the properties of fractionate+transpose to determine whether that was the case. One thing's for certain - no exhaustive search of a Polybius square keyspace is going to work. Exhaustive search of single columnar transposition is doable, assuming moderate key lengths. 8!, 9!, or 10! can be handled. So using exhaustive search to remove the transposition is feasible, particularly since we think we may have a good test for recognizing the correct result. But 25! is a whole different world. |
| When cryptography is outlawed, bayl bhgynjf jvyy unir cevinpl. | |
![]() |
|
| Revelation | May 14 2008, 11:07 AM Post #12 |
|
Administrator
![]() ![]() ![]() ![]() ![]() ![]() ![]()
|
I've started building the app. It only does the pattern check now. The results are not great, but not bad too. The pattern check yields about 200 false positives for a simple and short pattern. If the keylength gets larger (8+), the number of false positives increases rapidly. I'm hoping the frequency check will do us some good. For the .c file and the .exe, look in the zip. It should also compile on Unix, haven't tested it there yet. Here's the code:
If you've got some time, you may want to experiment with the app, like trying largers texts (maybe Loki's challenge). Haven't had the time for that. |
|
RRRREJMEEEEEPVKLWENFNVJKEEEEEAOLKAFKLXCFZAASDJXZTTTTTTTLSIOWJXMOKLAFJNNKFNXN RAGRBAQEMHIGDJVDSEOXVIYCELFHWLELJFIENXLRATALSJFSLCYTKLASJDKMHGOVOKAJDNMNUITN RRRRLJVEEEEECLYVYHNVPFTAEEEEEMWLMEIRNGLARWJAKJDFLWNTIERJMIPQWOTZEOCXKNUBNXCN RJIRPOWEANFUSNCZVDVZNMSFEKLOEPZLDKDJWSAAAAAAAOERHJCTNCKFRIMVKSOFOMKMANREWNBN RZUDRGXEEEEENFQIDVLQNCKNEEEEEDGLLLLLLAWIOSNCDARLODMTOEJXMILDFJROTKJSDNLVCZNN | |
![]() |
|
| jdege | May 14 2008, 01:07 PM Post #13 |
|
Elite member
![]() ![]() ![]() ![]() ![]() ![]() ![]()
|
It compiles clean on Linux. |
| When cryptography is outlawed, bayl bhgynjf jvyy unir cevinpl. | |
![]() |
|
| Revelation | May 14 2008, 08:27 PM Post #14 |
|
Administrator
![]() ![]() ![]() ![]() ![]() ![]() ![]()
|
If you see room for optimisation, please let me know. It should run as fast as possible. |
|
RRRREJMEEEEEPVKLWENFNVJKEEEEEAOLKAFKLXCFZAASDJXZTTTTTTTLSIOWJXMOKLAFJNNKFNXN RAGRBAQEMHIGDJVDSEOXVIYCELFHWLELJFIENXLRATALSJFSLCYTKLASJDKMHGOVOKAJDNMNUITN RRRRLJVEEEEECLYVYHNVPFTAEEEEEMWLMEIRNGLARWJAKJDFLWNTIERJMIPQWOTZEOCXKNUBNXCN RJIRPOWEANFUSNCZVDVZNMSFEKLOEPZLDKDJWSAAAAAAAOERHJCTNCKFRIMVKSOFOMKMANREWNBN RZUDRGXEEEEENFQIDVLQNCKNEEEEEDGLLLLLLAWIOSNCDARLODMTOEJXMILDFJROTKJSDNLVCZNN | |
![]() |
|
| jdege | May 15 2008, 01:23 AM Post #15 |
|
Elite member
![]() ![]() ![]() ![]() ![]() ![]() ![]()
|
I don't think you're going to see much room for optimization in the code you have so far. It'll be your test routines that will be the pace where I'd expect most of your time to be spent. And there, I'd expect two places where optimization might pay off. 1. How you access the statistics. 2. What sort of math you do on the statistics. And in truth, I don't see much room for improvement in 1. You plan on matching 2-grams, or maybe 4-grams? Both are small enough to keep in a non-sparse array, there isn't anything that's faster than array lookup. As for 2? Use integers, instead of floats. Add logarithms instead of multiplying, etc. Lot's of possibilities. But my real advice? Use a profiling tool. No point in optimizing where your program isn't spending any time. And remember - premature optimization is the root of all evil. |
| When cryptography is outlawed, bayl bhgynjf jvyy unir cevinpl. | |
![]() |
|
| 1 user reading this topic (1 Guest and 0 Anonymous) | |
| Go to Next Page | |
| « Previous Topic · General · Next Topic » |





![]](http://209.85.122.85/static/1/pip_r.png)



I'll have a look at it again when we have brute forced the ADFGVX cipher.
4:40 PM Nov 23