Unit testing, mocking, and various other testing techniques.
Why? Any project of sufficient complexity is very hard to test. If all you're doing is code -> build -> run to debug your code, you can very easily break something that's not in your immediate attention.
The problem is that good unit testing is hard, and time consuming. It can be so time consuming, that unless you can really plan in advance how you test, you could spend more time writing test code than real application code. (This is what happens when writing professional, industrial-strength code.)
So, when a hobby project becomes sufficiently interesting enough; such that the code will be so complicated that your code -> build -> run loop won't hit most of your code, you should think about how to have automated tests. They don't have to be "pure, by the book" unit tests, but they should be an approach that can hit most of your program in an automated manner.
You don't need to do "pure" mocking either. If you're writing something that calls a webservice, you could write a mock webserver and redirect your program to it. If you're writing something that works with pipes, you could have a set of known files with known results, and always compare them.
The goal is that you should cover most of your program with code -> build -> tests; and only do code -> build -> run for experimentation.
Let me second this. And in particular, I strongly encourage every developer to try starting a new project in a test-driven fashion (by which I mean that you advance the code by writing a bit of test and making it pass, and then doing that over and over.)
There's a qualitative difference between working in a well-tested code base that's very hard to describe convincingly. A lot of my early development experience was in code bases that had little or no testing. Experiencing a well-tested code base totally changed things for me. Instead of work being a death-of-a-thousand-cuts experience, it became pleasant, steady progress.
> Experiencing a well-tested code base totally changed things for me. Instead of work being a death-of-a-thousand-cuts experience, it became pleasant, steady progress.
I had the luxury of taking a well known data process and rewriting it with integration tests (input in, matching output with a golden file). It changed my professional life. Whereas before our deployment process included a 3 day wait and manual data checking on stage, after I was able to do deploys multiple times a day with confidence.
Unit tests can give you false positives (test failed but code is correct) and false negatives (test passed but code failed).
And TDD seems to create so many tests that you get huge false positive rates. I recently jumped on a project and I made a couple of fairly small code changes (a couple of hours) which caused 100 tests to fail. I then spent the next two days going through and correcting all 100 tests none of which found an issue in my code.
If you're saying that it's possible to do testing badly, I agree, just like it's possible to write production code badly. Sometimes teams new to unit testing do it ritualistically, without really understanding the purpose. That can lead to all sorts of bad outcomes. E.g., lots of tests that look impressive and even generate good coverage numbers, but don't really test what matters. Or tests that are highly duplicative, such that changing one thing in the code requires changing a lot of things in tests.
I have definitely dealt with code bases like that, and that sucks. But I have also dealt with code bases where the tests were great, and that's an amazing experience.
To do TDD well, I think it's important to release early and often and to reflect on one's experience (e.g., with weekly team retrospectives). That way if people are doing something unhelpful, like writing very duplicative tests, pretty soon they'll become an impediment to progress. The team will learn to write the useful tests, while skipping the ones that might fit some hypothetical pattern. It also helps people learn to design for testability; often, painful tests are a sign of bad design of the production code.
- Focus on "automated testing", don't get obsessed with philosophising about "the true nature of a 'unit'", or other such dogma.
- Be empirical: base your rules on what works; don't base your work on "the rules".
- The goal of testing is to expose problems in our program: "test failure" is a success, because we've found a problem (even if that problem is with the test!). Anything else is secondary (e.g. isolating the location of failures, documenting our API, etc.). Avoiding this goal defeats the point (e.g. choosing to ignore edge cases).
- Focus on functionality rather than implementation details, e.g. 'changing a user's email address' rather than 'the setEmail method of the User class'. This improves reliability and makes failures more useful/meaningful (i.e. "this feature broke" vs "this calling convention has changed").
- Mocking is a crutch: it works-around problems that can usually be avoided entirely during design; it can still be very useful when a design can't be changed (e.g. adding tests to a legacy system).
- Testing a real thing is objectively better than testing a fake thing; we should only mock if testing the real thing is unacceptable.
- If two components always exist together, pretending that they're independent is a waste of time and complexity.
- Having some poor tests is better than having no tests. Tests can be added, removed and improved over time, just like anything else.
- "Property checking" is a quick way to find edge-cases and scenarios we wouldn't have thought of.
- Fast feedback loops are important. Reducing I/O and favouring pure calculation usually speeds up testing more than reducing the number or size of tests (e.g. "unit" vs "end-to-end"). Incidentially, this is also how we avoid having to mock.
The type of engineers who would screw up 100 unit tests independently are exactly the kind of engineers who should be forced to write tests for their code. Can you imagine the integration tests had they not been doing any testing at all?
I don't think so. They probably could have been written better, they weren't written poorly, but it's really hard to write 200 unit tests for a feature that don't break when the feature is updated.
This is the gospel truth. It does take discipline though, because writing tests sucks. I like to have a policy of never committing a non-trivial function without a test. That way, I can never put it off and wind up with a huge chunk of untested code.
Mocking is a huge design smell. The more mocks or integration tests your projects requires to get full coverage the less modular your program is. A program that uses many mocks is a sign of very very poor design. You will find the code more complex to reason about and much harder to reuse code without necessitating a lot of glue code to make things work together. Without proper knowledge you won't even know the program is poorly designed.
I will grant you that 90% of programmers out there don't know how to design programs in a truly modular way, so most engineering projects will require extensive mocking. In fact most engineers can go through their entire career without knowing that they are making their programs more complex and less modular then it needs to be. Following certain design principles I have seen incredibly complex projects require nearly zero mocking (very very rare though).
Mocking indicates a module is dependent on something. Dependency is different from composition.
What's going on here? Both examples involve the creation of module C from A and B.
left: 'A' exists as wrapper code around B and is useless on its own. To unit test A you must mock B.
right: every module is reuseable on its own. Nothing needs to be mocked during unit testing. No dependencies.
The only exception to the right example where you MUST mock is a function that does IO. IO functions cannot be unit tested period, they can only be tested with integration tests.
There's a name for the left approach. It's called Object oriented programming using inheritance or composition(the oop version of composition; not functional composition) as a design pattern. (both are bad)
There's also a name for the right approach. It's called functional programming using function composition.
I don't advocate that you strictly follow either style. Just know that when you go left you lose modularity and when you go right you gain it. All functional programming does is force your entire program to be modular down to the smallest primitive unit. Extensive mocking in your program means you went too far to the left.
tangent: Another irony around this world is that a lot of functional programmers (javascript and react developers especially) don't even know about the primary benefit of functional programming. They harp about things like "immutability" or how its more convenient to write a map reduce rather than a for loop without truly ever knowing the real benefits of the style. They're just following the latest buzzword.
Forgive me, if I'm being dense, but doesn't either of these cases depend on how the composed objects are being used?
In your functional example A is an input to B (or vice versa?), how do you propose testing one of the modules without first instantiating the other one?
I'll give you two examples. One functional and the other OOP. Both programs aim to simulate driving given an input of 10 energy units to find the final output energy.
#oop
engine = Engine(10)
car = Car(engine)
car.drive() #result 8
class Car:
def __init__(self, engine):
self.engine = engine
def ignite(self):
self.engine.energy =- 1
def run(self):
self.engine.energy =- 1
def drive(self):
self.ignite()
self.run()
return self.engine.energy
class Engine:
def __init__(self, energy):
self.energy = energy
# ignite not testable without engine
# run not testable without engine
# drive not testable without engine and a car
# ignite, run, and drive are not modular cannot be used without engine.
# engine testable with any integer.
# Car useless without engine
# engine useless without car
#functional \
def composeAnyFunctions(a,b):# returns function C from A and B. See illustration above.
return lambda x: a(b(x))
def ignite(total_energy):
return total_energy - 1
def run(total_energy):
return total_energy -1
drive = composeAnyFunctions(run, ignite)
drive(10) #result 8
# compose testable with any pair of functions
# run testable with any integer
# ignite testable with any integer
# drive testable with any integer
# all functions importable and reuseable with zero dependencies.
# input_energy -> ignite -> run -> output_energy
"I think the lack of reusability comes in object-oriented languages, not functional languages. Because the problem with object-oriented languages is they’ve got all this implicit environment that they carry around with them. You wanted a banana but what you got was a gorilla holding the banana and the entire jungle." - Joe Armstrong
you don't necessarily need the car or engine to simulate the energy output of driving.
I've been using static methods in java that follows the pure function way, it proved very easy to maintain even to those who inherited my code later on.
That's mainly just namespacing. The only point to use an Object in object oriented programming is to unionize objects and state. To combine them together into a single primitive. This combination breaks compose-ability.
Static functions avoid state. You put them in an object in java because java has to have everything in an object. In any other language these would just be top level functions namespaced into a package or something. You are basically using java in a more functional way. Which is fine.
Thank you so much for a concrete example. I need to think about this some more. Clearly the code make sense, but in a wider context, can you have a banana without a jungle? I'm dabbling with some functional programming but I definitely have more experience with oop, so what you're saying is difficult for me to grasp, but the benefits are hard to ignore.
There are downsides to FP as well. I am not advocating one over the other. But there is a concrete theoretical reason why FP is more modular, reuseable and organized than OOP code.
Smalltalk is possibly the only OOP language that lets objects be compose-able and modular. Check out Pharo if you're interested. If you learn smalltalk well enough, you could apply its principles to traditional OOP languages and gain the modularity benefits.
But other than that, is there any other concrete downside?
For using Pure Functions, I don't see any downside to this. Aside from it being impossible to use for outside the program side-effects like IO to device.
I mostly agree with what you're saying, but I will add that is is also possible to write well-designed, modular, easy-to-test (minimal mocks) OOP code. It does provide more guns to shoot yourself in the foot with, I will admit.
Yes you are correct. Check out smalltalk, it fits the paradigm you describe. It was actually rated the most productive programming language in the world according to namcook. Ironically, it's Definitely one of the least popular languages as well.
No. You are putting words in my mouth and accusing me of being extreme. I am NOT promoting one paradigm over the other. TLDR? I hope you read my stuff. I find it rude if someone just comments with a one liner and summarizes everything I said into a catchphrase that is a perversion of the truth. I feel like a presidential candidate.
Anyway, this is what I am saying:
If you use functional programming your code will be more modular and reusable because the paradigm forces you to be that way.
If you use Object Oriented Programming your program will automatically be less reusable and less modular but more object oriented.
This is all I am saying. Your mistaken statement that I am promoting one style over the other is based off of this assumption: Modular programs are better than less modular programs. This is not True.
Something like a physics engine is a better fit for OOP then it is for functional. Although your program will be less modular as a result, OOP is still a better fit because physical objects are easily modelled with OOP objects.
Trees, graphs and algorithms involving things of that nature are a better fit for objected oriented programming then functional because many of these algorithms involve mutating nodes. Again, if you follow this style your program will become less modular overall as a result.
The ideal program is one that spans the spectrum of both OOP and functional. When it calls for it use functional or OOP depending on context. Overall for complex web applications that most startups make, in my opinion, the program should be more functional then it is OOP. A web request is basically a function that takes in a request as an input and outputs a response. The form factor of a function better fit for this, and you get high modularity as a side benefit. There is no point in simulating the request/response paradigm in a stateful Object while losing modularity in the process.
For a game. OOP is better in my opinion. Gaming entities involve constant mutation of things with state so OOP is a better fit. UI is a better fit for OOP as widgets are better represented by objects (FRP aka react&redux, imo works well but is an awkward abstraction)
There is one exception to this rule. In general Objects in object oriented programming are not compose-able. However, Smalltalk is an object oriented language where objects ARE compose-able. Smalltalk is the language that coined the term "object oriented" and although it is no longer popular as it was before it is still a very robust language and learning from it has huge benefits.
Why? Any project of sufficient complexity is very hard to test. If all you're doing is code -> build -> run to debug your code, you can very easily break something that's not in your immediate attention.
The problem is that good unit testing is hard, and time consuming. It can be so time consuming, that unless you can really plan in advance how you test, you could spend more time writing test code than real application code. (This is what happens when writing professional, industrial-strength code.)
So, when a hobby project becomes sufficiently interesting enough; such that the code will be so complicated that your code -> build -> run loop won't hit most of your code, you should think about how to have automated tests. They don't have to be "pure, by the book" unit tests, but they should be an approach that can hit most of your program in an automated manner.
You don't need to do "pure" mocking either. If you're writing something that calls a webservice, you could write a mock webserver and redirect your program to it. If you're writing something that works with pipes, you could have a set of known files with known results, and always compare them.
The goal is that you should cover most of your program with code -> build -> tests; and only do code -> build -> run for experimentation.