| Welcome to Crypto. We hope you enjoy your visit. You're currently viewing our forum as a guest. This means you are limited to certain areas of the board and there are some features you can't use. If you join our community, you'll be able to access member-only sections, and use many member-only features such as customizing your profile, sending personal messages, and voting in polls. Registration is simple, fast, and completely free. Join our community! If you're already a member please log in to your account to access all of our features: |
| Vague ramblings on Unit Test frameworks, Mocking frameworks, and Inversion of Control containers | |
|---|---|
| Tweet Topic Started: Feb 26 2014, 01:00 PM (1,176 Views) | |
| jdege | Mar 4 2014, 03:27 AM Post #16 |
|
NSA worthy
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]()
|
So, I've been rambling on about dependencies, in an abstract way. Time to get a bit more concrete. What does a dependency look like? Simple. It's any reference to an object within a function. And in this context, an object is any variable, any function, or any datatype. If I access a global variable, that is a dependency. Ditto if I call a global function. Even something as simple as C's printf() - if I use it in a function, I must have C's standard library linked into my application, and I am writing output to the standard library's stdout filehandle. (Which brings up another issue - when I add a dependency to an object, I'm adding a dependency to everything that object depends on. Using printf() for output seems simple, but it brings in a lot of baggage. It's convenience comes at a cost.) The thing about global variables is that they create a dependency on a single specific object, which is inflexible - and makes it harder to use specific values in tests. If you remember back to the video - using the system clock within a function created serious difficulties in testing to make sure that the code that was supposed to run only on Tuesdays actually worked right. And this inflexibility is inherent in having only one of something, it doesn't go away just because you've packaged that "global variable" in one of the clever ways we try to hide it. Class static member variables still have the problems of global variables, because there is only one of them. The popular OO singleton design pattern still has these problems, because there is still only one of them, even if it is wrapped up pretty. If my function takes arguments, it creates a dependency on those arguments. If my area() function takes a polygon as an argument, it has a dependency on a polygon. Not any particular polygon, but it must have some polygon. And in a strongly-typed language, it must understand the polygon type. If I'm writing in an OO language, and I have arguments passed to my constructor, my class has dependencies on those arguments. If my object has settable properties, it has dependencies on them. And if my class has private member variables that are initialized within the class, or my function has local variables that are initialized within the function, I have dependencies. In other words, I always have dependencies, no matter what I do. The issue is, do they cause me problems? |
| When cryptography is outlawed, bayl bhgynjf jvyy unir cevinpl. | |
![]() |
|
| mok-kong shen | Mar 4 2014, 11:46 AM Post #17 |
|
NSA worthy
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]()
|
IMHO an answer to the rather hard to deal with question would be: "It depends". To take a much broader view: Modern civilization has created everywhere high dependencies. For example, one that lives in downtown of a city certainly depends in availabilty of his foods for cooking on the supermarkets. If there were a walk-out of the employees of the supermarkets, would that cause one problems? On the other hand, since such a probability is intuitively rather small, one would rationally take in this case the risks instead of taking security measures like stockpiling constantly a large amount of foods at home for such emergencies. One has always to consider the trade-offs in all practical situations and above all there can be no "absolute" rule of behaviour in any concrete situation that applies to everybody everywhere in the world "alike" and valid for the same person "at all times", I am afraid. Returning to the context of this thread, I like to mention that software engineering in my personal view has and will continue to have some elements common with art, i.e. the human factors play some role, despite efforts and recent advancements in formal verifications etc. For, after all, it's engineering and hence not to be compared with the strict discipline pure math. Edited by mok-kong shen, Mar 4 2014, 02:18 PM.
|
![]() |
|
| jdege | Mar 4 2014, 02:31 PM Post #18 |
|
NSA worthy
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]()
|
Yep. It depends entirely upon the context. For example, in most environments, you'd use floating point numbers freely, without any cost and without any concern for the dependency that usage creates. After all, floating point numbers are a language primitive - they'll always be there, and they'll always work, and you simply don't need to worry about it. Until you start working in embedded microprocessors that lack floating point hardware, and implement floating point operations in software libraries. In that case, to use floating point imposes a real cost, it imposes a complexity - and it imposes a dependency you need to worry about. |
| When cryptography is outlawed, bayl bhgynjf jvyy unir cevinpl. | |
![]() |
|
| mok-kong shen | Mar 4 2014, 04:40 PM Post #19 |
|
NSA worthy
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]()
|
There are also problems that are independent of programming techniques as such but could be critical nonetheless. In numerical computations involving floating point numbers, for example, there sometimes may be a couple of different algorithms that can serve the same purpose but are different in stability such that under circumstances one algorithm employed may deliver very inaccurate results in comparison with another. |
![]() |
|
| novice | Mar 5 2014, 03:35 PM Post #20 |
|
Super member
![]() ![]() ![]() ![]() ![]() ![]()
|
OK -- I've got the point that dependencies matter. And from your earlier posting
So where do we go from here? |
![]() |
|
| mok-kong shen | Mar 6 2014, 02:33 PM Post #21 |
|
NSA worthy
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]()
|
I sippose that your question implies "what should one do?" IMHO, since evidently dependencies can't to eliminated, what can rationally be done is to attempt to have the dependencies be as transparent as possible in the program codes, i.e. the programs are to be designed in such a way that the dependencies are all "localized" and hence be much more easily to be consciously seen by the programmers during development so as to be able to avoid (as far as possible) potential problems (due to dependencies) that could occur on the one hand and that errors that eventually do occur could be more easily detected and quickly repaired on the other hand. Here a few relevant terms that come to mind are IMHO: structured programming, top-down programm development, OOP, functional programming, modelling languages, standard program (procedure) libraries, design patterns, program development environments, program verification systems, program documentaton systems. It's evident that all depends on the size and the importance of the programming project and none of the terms I mentioned is relevant, in case one is writing a code that outputs "Hello World!". Edited by mok-kong shen, Mar 6 2014, 04:07 PM.
|
![]() |
|
| jdege | Mar 6 2014, 03:37 PM Post #22 |
|
NSA worthy
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]()
|
That is exactly what we've learned, from our experience in unit testing. Unit tests only work when the unit under test can be isolated from dependencies, so the unit testing practitioners developed practices to make it easier to isolate dependencies, and that turned out to have major benefits in areas far beyond testing. |
| When cryptography is outlawed, bayl bhgynjf jvyy unir cevinpl. | |
![]() |
|
| jdege | Mar 10 2014, 10:15 PM Post #23 |
|
NSA worthy
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]()
|
So, finally, what is this "unit testing" thing all about? Unit testing, in its larger sense, has been around forever. The basic idea is simple - each component of which a software system is composed should be tested individually, in isolation, before trying to integrate it with others. And as such, it was an idea more talked about that actually practiced. Normally, the "unit under test" would be a complex subsystem developed by a team, and tested in isolation only because it needed to integrate with another complex subsystem developed by another team. The idea of testing down to the lowest level was rarely considered. because it was clearly far more effort than it was worth. Until, that is, Kent Beck and the beginning of the xUnit frameworks. The idea of xUnit is very simple - write the tests in the same langauage as the code, at the same time as the code, within a framework that can automate the running of the tests. This promised to significantly lower the cost per test. So, what is a test, and what do these frameworks provide? At its core, a unit test is a function, like any other. It does what you program it to do. It's marked, in some way, so that the framework knows that it is a unit test, and can include it when it runs all the tests. How it is marked depends on the language. In C++, you derive your test classes from a base class provided by the framework. The C# unit test frameworks rely on annotating test classes with custom attributes provided by the framework. Different languages provide different capabilities. The essential thing is that when you write a test case, you somehow tell the framework that it is a test case. The second essential capability that a framework provides is a way for you to tell the framework that the test has passed or failed. This is usually done through some sort of assertion capability. If you're writing a test for a square-root function, for example, you might calculate square root of four, you'd them assert that the result was equal to two: double r = Math.sqrt(4.0); Assert.areEqual(r, 2.0, "Calculating srqt(4.0)"); If the assertion fails, the test framework will report that the test failed. The third capability that a test framework provides is the ability to run the tests, and to report the results. Again, how this works depends on the framework, and that depends on the language. In languages like C# and Java, that compile to bytecode that can be examined and run by external code, the test frameworks include test runners than can run the tests from any executable file. In languages like C++, that compile to binary executables, the test frameworks provide a framework into which the tests - and the units under test - can be linked to provide a stand-alone test executable. The details aren't important - the essential function is that there is a way to run all the tests, easily, and to identify which have failed. |
| When cryptography is outlawed, bayl bhgynjf jvyy unir cevinpl. | |
![]() |
|
| jdege | Mar 14 2014, 02:05 AM Post #24 |
|
NSA worthy
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]()
|
Is anyone still reading this? |
| When cryptography is outlawed, bayl bhgynjf jvyy unir cevinpl. | |
![]() |
|
| novice | Mar 14 2014, 09:01 AM Post #25 |
|
Super member
![]() ![]() ![]() ![]() ![]() ![]()
|
Yes I have been reading all your episodes, though not the responses from Mok Kong Shen on whom I have pressed the ignore button. My reaction to your teach-in is to apply a test as part of the developing code as I write each section or function of a program. I have said before that my programs are to do with cipher solving, so they are a lot simpler than the stuff you professional programmers are creating. At the end of the day I adjudge the program as OK if a known piece of ciphertext with the known key gives the known plaintext. That of course does not test the efficiency of the solving algorithm, which in the worst case is so poor that it will not solve. But such an outcome, when it happens (as it sometimes does), becomes visible when attacking the first unknown cryptogram. The conclusion is then not 'error' but rather 'bad mechanism' and is a signal for more thought and a better algorithm. I have followed this integrated approach in one instance since reading your articles and am happy with it. The theory is right (includes your three capabilities) and it works in practice. Result happiness. I gathered from your last episode that it was the final one. If you have more to propose, I shall certainly be reading it. |
![]() |
|
| Grip2000 | Mar 14 2014, 12:32 PM Post #26 |
|
no member
![]() ![]() ![]() ![]() ![]() ![]() ![]()
|
Hi Jdege, Yes, of course. I find your posts on this topic very exciting ! BR GRip |
![]() |
|
| james | Mar 14 2014, 04:38 PM Post #27 |
|
Elite member
![]() ![]() ![]() ![]() ![]() ![]() ![]()
|
Yes! |
![]() |
|
| mok-kong shen | Mar 14 2014, 07:35 PM Post #28 |
|
NSA worthy
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]()
|
Since you asked, I like to stress that it seems to me to be doubtful that there is any silver bullet in software testing methodology. Anyway, http://en.wikipedia.org/wiki/Software_testing doesn't point in that direction, if I don't err. I suppose that it depends to quite a high degree on the software project itself, which can be in a very wide spectrum, i.e. different projects may correspond to different optimal testing methods. |
![]() |
|
| nullsole | Mar 18 2014, 11:24 PM Post #29 |
|
Member
![]() ![]() ![]() ![]()
|
Yes! As someone who recently took up a programming language AND linear algebra from the suggestions here. Have finished 3 Python courses, and 1 and 1/2 (am in it now) courses in Linear Algebra per advise from this board. AND have done and studied and tried to add to all the tutorial presented here (in Python) I have been following with GREAT interest. I know I'm not at the level of the others here, but for me --- I take what I understand, look up what I don't and move from there. In other words, I am following and learning and taking it in. Thank you! For this and your tutorials ... |
![]() |
|
| jdege | Mar 19 2014, 03:48 AM Post #30 |
|
NSA worthy
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]()
|
Most of us who were programming, back when the xUnit testing frameworks became available were quite excited at the idea, and determined that wee'd write such tests for our own code. Very few of us actually did so. We quickly discovered that writing effective unit tests was far more difficult that it would appear. And the core of the problem was the dependencies that I've been ranting about. Trying to write a test for each bit of functionality in isolation is only possible if you can actually run each bit of functionality in isolation. If your code is a tangled mess without clean separation between modules, it's impossible to work with the pieces individually. In essence, trying to write unit tests exposes flaws in a project's architecture in a way that nothing we'd try to do before did. To write unit tests, you have to actually follow the standard principles of separation of concerns, etc., that we'd all given lipservice to for many years, but only actually followed most of the time. There basic problem is simple - if module A depends on module B, module B has to be present in order for the first on module A to run. If module B is something like an instantiated database, with a certain set of expected records, then the cost of creating an environment in which the test can run can be very high. The solution, in a generic sense, has long been well-known - it's called Dependency Inversion. The idea is simple - rather than have module A depend on module B, have module A and module B both depend on an abstraction of the behavior that module A expects module B to provide. This abstraction might be an interface, it might be an abstract class, it might be a pure virtual class, it might simply be a standardized set of method names (in a language that does duck typing). What facilities different languages provide differs. That's what Harcourt was doing, in the video I linked in the original post, when he created the interface IClock - abstracting out the behavior that his AutoCue class depends on. Conceptually, very simple. The problem is that, no matter how simple this is conceptually, it's a signficant amount of work. Which is why so many developers who thought that unit testing was a really neat idea in concept, never actually used it all that much in practice. Every class need to derive from an interface? And we need to write new classes that derive from these interfaces, in every test? And then we need to find a way to pass instances of the correct class to every object as it's being run - the production class in production and the test class in test? It's an enormous overhead. But there is a solution. (To be continued). |
| When cryptography is outlawed, bayl bhgynjf jvyy unir cevinpl. | |
![]() |
|
| 1 user reading this topic (1 Guest and 0 Anonymous) | |
| Go to Next Page | |
| « Previous Topic · General · Next Topic » |





![]](http://z2.ifrm.com/static/1/pip_r.png)



7:28 PM Jul 11