I've noticed that as I've gained experience over the years, I've also adopted many strong opinions - Some of which turn out to be erroneous, and clouded my judgement.
This is an excellent TED talk that gave an extra dimension on communication: http://www.ted.com/talks/celeste_headlee_10_ways_to_have_a_better_conversation
It's well worth taking 12 minutes out to watch this - What things do you notice that you do?
"Approach a conversation, as if you always have something to learn".
The Infrastructure Engineer: A blog about Infrastructure, Systems, DevOps, Automation, Cloud Computing, Python and Tech News.
Friday, 24 February 2017
Wednesday, 8 February 2017
Consistency vs Perfection
I prefer known "bad" code than assumed-perfect new code for knowability reasons (Six Sigma) - You can control, fix and support known(bad, but consistent) processes far easier - And the typical way forward is to bring consistency first.
An example (One that I've trapped myself with in the past):
1. I write one half-good piece of code.
2. I copypasta that in 5 other places (Defect: Now was the time to refactor. Perhaps understandable when you have a large code base).
3. Someone else adds a 2nd similar-but-better piece of code.
4. Someone else adds a 3rd similar-but-even-better piece of code.
5. We have 3x different similar pieces of code that should be consolidated into a single abstract helper method.
6. Perfect code #2 has a bugfix applied because it turned out that we forgot about an old edge case that method #1 covers.
7. Perfect code #3 has a cool enhancement, and diverges it's purpose away from the other two methods.
The two issues with this design pattern are:
1. The refactor becomes harder. The difference necessarily causes difference. Consistency keeps stuff glued together, makes for easier refactors, and therefore more likely to be done.
2. Multiple ways of doing the same thing, with different performance characteristics (two methods having unknown performance characteristics). The old code might be "bad", but it's also probably less buggy and a more known quantity.
This is another case of Perfect being the enemy of Good.
An example (One that I've trapped myself with in the past):
1. I write one half-good piece of code.
2. I copypasta that in 5 other places (Defect: Now was the time to refactor. Perhaps understandable when you have a large code base).
3. Someone else adds a 2nd similar-but-better piece of code.
4. Someone else adds a 3rd similar-but-even-better piece of code.
5. We have 3x different similar pieces of code that should be consolidated into a single abstract helper method.
6. Perfect code #2 has a bugfix applied because it turned out that we forgot about an old edge case that method #1 covers.
7. Perfect code #3 has a cool enhancement, and diverges it's purpose away from the other two methods.
The two issues with this design pattern are:
1. The refactor becomes harder. The difference necessarily causes difference. Consistency keeps stuff glued together, makes for easier refactors, and therefore more likely to be done.
2. Multiple ways of doing the same thing, with different performance characteristics (two methods having unknown performance characteristics). The old code might be "bad", but it's also probably less buggy and a more known quantity.
This is another case of Perfect being the enemy of Good.
Tuesday, 6 September 2016
Poking the Beehive
This week we learnt that Facebook crashed a data centre when stress-testing their services. Why the hell would they do that to themselves?
On the surface it would seem crazy - Why would you poke the beehive when you know it will make the bees angry?
In fact, it's not a new thing; Some teams apparently like to get stung. Netflix most famously have a simian army—Chaos Monkey, Latency Monkey and so on—that poke the beehive in various ways and at varying scale. Netflix worked up to disconnecting (and therefore breaking) entire data centres. Amazon does similar (but planned) "game days", where relevant teams are pulled in to monitor while dummy traffic is turned up to 11 to stress the various systems.
The team at Netflix reasoned as follows: All of the things are going to break one day. They will probably break at the most inopportune time, when there is high demand, a new release, after dinner, Christmas time etc. It's expected that monitoring will send an alert to the appropriate engineers. Engineers will look at the problem, recover service, deep dive and ultimately fix the problem at the source. The result is a more robust system, an Anti-Fragile architecture.
If your systems are going to break eventually (and they will!), and you care about it when they do (you should!), then why not do all of that work up front, when customers are less likely to notice, and build a more robust product.
Front-loading this operational work is also beneficial from a planning and resource management perspective. You could plan a 20% buffer for poking the beehive, and work through a lot of the bugs prior to initial release. It's far easier to justify bug squashing at development time, then trying to carve out a lump of time later.
Poking the beehive hive should be a part of your deployment process, and then run at regular, random intervals throughout the year. You have the option of going the full hog (Netflix) and test your operations team and process too, by not informing them. Or you could focus on testing the system instead with an Amazon-style game day where dependent engineering teams are ready to dive into breakages.
Do you and your team poke the beehive on a regular basis? What are your experiences with system stability?
Tuesday, 23 August 2016
Perfect is the enemy of good (Or: The 80/20% rule)
Ah, the age old adage — "Perfect is the enemy of good", and the 80%/20% rule. Most of us know of it, but how many of us practice it?
The 80%/20% rule—aka the Pareto principle—suggests that 80% of the results come from 20% of the effort. The Pareto principle originally referred to land ownership distribution across a population, but it has been observed and recycled for use in many other areas. Put simply, aiming for perfect (100%) is a fruitless goal, and you will get lost trying to hit that last 20%. I would suggest that that last 20% is actually best thought of as complete waste, perhaps even down right dangerous. Here be dragons!
We see it all the time; I'll do Just this one more thing before release. Just one more feature. Just one more bug fix. Just one more unit test. Strangely though, never Just one more piece of documentation!
It seems like a commendable goal on the surface — Why would one not strive for perfection and make the customer ecstatic? The answer is that it delays delivery, and increases opportunity cost:
- Delayed delivery: Trying to hit that last 20% unnecessarily delays delivery of your product to the customer—You actually produce negative value by doing this extra work.
- Increased opportunity cost: The remaining 80% of time that you spend trying to hit that last 20% could be better spent on hitting that valuable first 20% in other projects.
One strategy that you could employ to break the cycle is to put yourself into the client or customer's shoes. Yes, they want the perfect product with all of the features, performance and availability and none of the bugs. But above all, they want It, Now.
Another approach you could adopt is to cultivate an agile or iterative atmosphere with your clients. You explicitly agree that improvements will be delivered rapidly and in incremental stages. This can be somewhat of a paradigm shift for IT Managers, Engineers or Program Managers that are used to the classical waterfall model, but the outcome is a productive one: A better relationship with the client, better understanding of requirements and quicker course-correcting, quicker product deliver, and ultimately a happier customer. You typically never work at the wasteful long tail because by that point, you have already agreed with your customer that a more valuable task is to be delivered first.
I still find myself stuck in perfectionism mode from time to time, but manage to find a way to snap myself out of it! Where do you find yourself stuck in perfectionist mode? How do you find your way back out of the rabbit hole?
Subscribe to:
Posts (Atom)


