Seven Habits To Create Reliable Software - Work In Small Increments
This is the first part of a collection of seven blog posts about how to write reliable software. When a new developer joins the company we go over this list, and so I’ve decided to organize my thoughts on the subject a bit and share them with a wider audience.
Small is beautiful. Also small is easy to comprehend, develop, test, deploy, get feedback, diagnose and if needed to roll back. It is essential that you work in small increments.
Big bang releases mean big risk. There are two main problems associated with them. The first one is the fact that customer wants and needs change all the time. Let me tell you a story about that.
Some years ago I was working at a company that had two main products. At some point we decided to develop. Our initial research showed that customers would love this product, and no other products on the market were offering similar functionality. However as the saying goes, too many cooks spoil the broth. There were too many product people with too many ideas how to wow the customer. Every one of them wanted to leave their mark in this product.
It was a product that was using an email client, so it took us a lot of time to add support for all the possible email clients at that time. After one and a half years of development, we were finally done. But when we release it, the customers did not like it. They said it was not what they expected it to be. Also the market has moved both in terms of better, faster technologies and in terms or competition. When we started development we didn’t have any competitors, when we released we had about five. We ended up with total of four customers using the product, two of them actually paying. It was a disaster. Had we delivered small functionalities constantly, we woud have valuable feedback loop and make adjustments when needed.
The second risk with big releases is a technical one. You push 1,000 line code change at once in production. Suddenly the performance of your application drops by 50%. Now think about another other scenario, you push a 5 line code change, again the performance drop in production is 50%. In which of those two scenarios it’s more likely to quickly find the root cause of the problem?
When I talk to developers about the benefits of small code changes they have two main objections. The first one goes like this “The feature I’m writing is too big, need to push either all or nothing”. So this is where I say, “No, you need to find a way to deliver it piece by piece”. Do you know what happens when you code for two weeks on a separate branch (and there no automated integration checks run on that branch)? All hell breaks loose when you integrate it into master after two weeks. If companies such as Facebook or Google with thousands of developers can practice trunk development, so can you. You can hide functionality that’s not ready for release by using feature toggles or branch by abstraction. Push your code and integrate it at leas once per day.
OK, you might say, the above is pretty straightforward advice for someone working on web based software, where changes can be deployed rapidly to production. How about when you develop software that’s installed on customer premises? First, it’s not up to a developer to decide this, but if you want to deliver reliable software, you need to have a way to push functionality and fixes constantly, not just once a year. Think how ofter your OS of choice asks you to update. How about your phone? This was the customer facing aspect. The other aspect is technical — the internal quality of your off-the-shelf software. The principle is still the same — use small batches. By developing in small batches, you can easily spot problems. Imagine that you push 5 lines of code and the CI build brakes. One of those five lines is obviously the problem. Now imagine if you push 1,000 lines of code and the CI build breaks. You can benefit greatly from fast testing feedback loop, especially if you’re able to run all your high level tests (API and UI) after every code change.
Small, iterative steps are also at the core of Toyota’s Improvement and Coaching Kata’s. They are called PDCA - Plan Do Check Analyze. You grasp your current condition, know your target condition. You do not know how to get there in specific details. But you take a small step, evaluate the result, and based on it you take another step, adjusting if needed. It does not matter how do you start, as long as you just start and take small steps toward your established goal.
The rest of the related posts can be found here: