Seven Habits To Create Reliable Software - Test Your Code

This is the second part of a collection of seven blog posts about how to write reliable software. When a new developer joins the company we go over this list, and so I’ve decided to organize my thoughts on the subject a bit and share them with a wider audience.

"The first draft of anything is shit."

E. Hemingway.

Remember the time when you had to write essays in high school? You didn’t pass the first version of your text to the teacher. You usually had one or two drafts before the final one. It’s the same with the new feature you’ve just completed.

Learn to work as if there is no QA team to clean the crap after you. Actually most startups and ‘unicorn’ software companies do not have dedicated QA roles. The same shift starts to happen with Microsoft and their ‘combined engineering’ initiative. Remember that in the past they were famous for 1/1 Dev/QA ratio. The current trend, that I think will continue to increase is to shift left. This means more testing is done by the developer. As the dev salaries grow (sometimes the budget for this comes by decreasing the number of QAs), so are the expectations of what they are brining to the table. So it’s no surprise that the latest company to slash the dedicated QA role is Yahoo!. Whatever the reasons for this move are, Yahoo! reports that the teams are now more invested in quality. One could even argue that the quality goes up.

So even if you have a dedicated QA in your company now, it does not mean that you’ll have such a luxury in the future. Be a professional. The buck stops with you. Don’t throw crap over the wall.

The bare minimum you should do is to manually check the happy paths of your new feature. It’s way better if you also write unit tests to cover the feature. Think the 80/20 rule. To you and to the management, this may seem like unnecessary task that’s only slowing you down, but believe me it pays off in the future.

In my experience, developers who perform those two steps (happy path, high level sanity check and low level unit tests) catch more than 80% of the problems, before engaging anyone else in the delivery pipeline. Focus your efforts here as the bugs are easy to find and cheaper to fix in the beginning.

It’s even better if you can write high level API or UI tests. This may be harder for you as developer because you have to shift focus and see the forest (your deployed application) rather than the trees (the methods in your code). Also the tests are more flaky than unit but that’s whole other topic for new blog post series.

Remember that the price of a bug increases, the later in the cycle it’s found. It’s the most expensive when it’s found by your customers. This is how the flow looks like:

Notice how many people are involved and how costly it is (salaries paid, time to manage, resources to coordinate), no to mention the possibility of reputation loss. So you need to make sure that you do everything in your power to not let bugs that can be caught automatically not slip in production. Let the machines do what they are really good at - repetitive checks.

I’m still amazed that in 2016 we need to talk about benefits of unit tests. Even if you throw away the test later, you’ve still covered some scenarios you’d normally can’t test on high level. Like what happens when an exception occurs, or somehow data in wrong format needs to be interpreted.

Don’t get obsessed with code coverage numbers. Don’t write tests cases to satisfy some random code coverage statistic (and God help your company, if you’re incentivized somehow for high code coverage). It’s easy to write unit tests with no asserts. If you’re on the other side — e.g. a manager, you can counteract the false unit tests that just inflate the code coverage numbers by using mutation testing.

Organization should be very careful when incentivizing developers. In software companies this is actually very hard as it can often backfire and lead to local optimizations that lower the overall output. You should be incentivized by how much customer value you create. Not code coverage, number of bugs fixed, number of features giver to QA for testing. Your job is not over when you hand over your feature to the other team - QA or Operations. Your job begins when the customers start to use the new feature.

Remember that your customers do not know the individual developer or QA names. Nor do they care. They care only for software that brings them value, is reliable and of good quality. You rise and fail together as a team.

“You build it, you test it.” it is a paraphrase of the quote by Werner Vogels, the CTO of Amazon. No more excuses. The buck should stop with you.

The rest of the related posts can be found here:

• Work In Small Increments
• Resources Are Not Free
• Don’t Assume Anything
• Monitor Production
• Solve Problems Immediately
• Continuous Improvement