Is software antifragile? I think so. I recently finished the book Antifragile – Things that Gain from Disorder by Nassim Nicholas Taleb. I liked it a lot, and I think the ideas in it are quite useful when examining various systems and phenomena. What especially struck me when reading it was how much of it applies to software development. These are the main themes:
Software is Antifragile
Antifragility means things that gain from disorder. In the chapter “The Cat and the Washing Machine”, Taleb discusses the difference between the natural and the man-made. For example, the muscles in our bodies get better with use. Left unused, they eventually wither. A washing machine on the other hand does not get better with use. It will eventually wear out and break down. So where is software on this spectrum? The way I see software, it gets better with use. It is not self-healing, but given that somebody is fixing the bugs that are found (which is often the case), then software keeps getting better with use. As Taleb writes, errors are rich in information, and for each bug found and fixed, the software gets a bit better, since it now also handles the situation that lead to the bug. This malleability of software, coupled with the fact that software doesn’t wear out, makes it more like the cat than the washing machine in my opinion. If software only gets better with use, how come there is so much software that doesn’t work very well? The counterbalancing force here is the constant flow of new features added. This is the normal state for most successful software – we want it to do more things. So while we gain quality from the bugs fixed, we lose quality when adding all the new features. For some programs where the functionality is fixed, for example unix commands like cat, grep or ls, I imagine they have become very stable, because eventually all of the bugs have been found and fixed.
Agile Development is Antifragile
A lot of the ideas in Antifragile are exactly the same as in agile development. The tendency to want to stabilize systems by suppressing randomness and variability that Taleb criticizes has its parallel in waterfall development’s desire to have a predictable project. Taleb argues that instead of relying on predictions, it is better to be in a position where you can take advantage of variations. He calls this “more upside than downside”, which is a kind of options thinking (not surprising, given Talebs background as a trader). In one of the first chapters in “Extreme Programming Explained”, Kent Beck used options to illustrate the advantage of being open to change. I was also reminded of agile development later in Antifragile, where Taleb writes about universities and technological progress. He makes the case that a lot of progress comes not from the application of theory, but rather from tinkering and trial and error. When the result is described later, it is tempting to make the development process seem more ordered and logical than it really was. This describes my experience of how a lot of software development works. You add bit by bit, and revise and change as you gain more insights, both from the act of coding, and from user feedback. I would never have been able to predict the end result at the beginning – the whole process is much more organic.
The Flaneur as Exploratory Tester
In Taleb’s opinion, modern life removes too much randomness and variability. He illustrates the difference by comparing a tourist to a flaneur. The tourist has a set schedule, which is followed to the letter. This means that the tourist sees the major sights, but never has a chance to serendipitously see or experience anything that wasn’t planned from the beginning. The flaneur on the other hand does not have a fixed schedule, only a starting-point, but is continuously evaluating what to do next based on newly acquired information. So as a flaneur, if you notice for example a market, you may choose to check it out instead of sticking to the schedule. When talking to people there, you may get a recommendation for a good restaurant, and so on. By adapting to new information, you are able to find interesting places you would never have thought of visiting during the planning of the trip. To me, this is a great metaphor for exploratory testing. A definition of exploratory testing given in Explore It! is: “Simultaneously designing and executing tests to learn about the system, using your insights from the last experiment to inform the next”. This is the flaneur as tester (and the tourist is the automatic checking by scripts).
Skin in the Game
At the end of the book, there is a chapter on ethics that Taleb calls “skin in the game”. To have skin in the game, you should share both in the upside and downside. Taleb quotes the 3,800 year old Hammurabi’s code: “If a builder builds a house and the house collapses and causes the death of the owner – the builder shall be put to death”. It is interesting to view this from a software development perspective. I have never worked on software where people’s lives were in danger if the software failed, but I would not be willing to submit to Hammurabi’s code if I did. But I think a little less extreme form of skin in the game is actually very good. Being on call for example. If the software you wrote fails, you may get called in the middle of the night to help fix it. I have been on call at most of the places I have worked in the past, and I think it has a lot of benefits. It gives you an incentive to be very thorough, both in development and testing. It also forces you to make the software debuggable – otherwise you yourself will suffer. Another way of introducing skin in the game is dog-fooding – using the software you are developing in your daily work. I have never worked on software that we have been able to dog-food, but I think that is another great practice.
Antifragile Software Development Career
You can also apply the concept of antifragility to your career as a software developer. A couple of months ago I switched jobs, from developing telecom applications in Java to developing financial risk-management software in Python. Part of the reason for doing that is that I value variation. Having to learn a new language, a new code-base and a new domain is stressful in the beginning. But the benefit is that I broaden my knowledge as a developer. Staying too long at the same place can make you vulnerable – Taleb labels someone in their fifties who has worked their whole career in the same company as “completely unhireable”.
What Do You Think?
If you haven’t read Antifragile yet, I can really recommend it. There is a lot of food for thought there, not only in relation to software development. If you have read it – what are your thoughts on software development and antifragility? Let me know in the comments.
I gave a response on the HN submit. I like how people can discuss Taleb without resorting to inflammatory nonsense. I’ll give a proper response later I just added some food for thought. I also will subscribe via rss or twitter from now on as I continue learning and developing.
Hacker News discussion: https://news.ycombinator.com/item?id=7867311
Very interesting post, I’ve been working in the software industry for almost 3 years now and I feel the points you mentioned represent the reality of these days software and its organic way of growing. In order to develop secure and reliable software I feel we should use TDD (or any other methodology that enforces testing). These methodologies allow us to put our “skin in the game” and still be able to sleep at night. It’ll also put our work on higher standards.
Excellent discussion. One thing about the Agile/DevOps continuous deployment approach is SW is never finished, merely enhanced and ongoing attention not only drives out imperfections but lets in the innovations in performance and functionality that bring it closer to the
Pingback: Antifragility and Software Development | Henrik Warne's blog | COPY SOFT
I agree that software systems may have a measure of Antifragility, but as software developers our goal is to make our systems more Antifragile:
I like this post.
Came across an interesting interview with Taleb a while ago: http://www.theguardian.com/books/2012/nov/24/nassim-taleb-antifragile-finance-interview
So many thoughts running through my head right now. Currently restructuring my department’s systems, workflow and processes. How do I build antifragile elements into this? Making decisions based on dynamic variations sounds more like being reactive rather than proactive. How are forecasting, statistics and analytics valuable with an agile, antifragile approach? I know there are answers out there…I’m just fairly new to this but want to understand it.
I think you’ve missed the point. The whole point of antifragility is to gain from disorder. Software is neither antifragile or fragile. Reacting to a production failure by modifying the code and adding a unit test would be an example of antifragile development – the source code has benefited from the feedback the production break has provided. Similarly perhaps the software has introduced a standardisation and in fact it is the client who should react with a change.
The dog-food or skin-in-the-game approaches both incentivise an approach of withholding changes out of fear of breaking something – this ultimately leads in the long term to functionally robust systems which handle multiple edge cases (which may not have been required). However this approach also slows development and leads to the source code itself becoming fragile due to an inability to refactor the code out of fear of causing a break. How can you benefit from a break if your goal is to not cause a break? This is the premise of antifragile.
I’m not against dog-food or skin-in-game approaches in general – but I don’t believe they assist in achieving antifragility.
Thanks for commenting. I think we agree. In the section “Software is Antifragile” above, I write about the case you describe – failures in production leads to bug fixes being added to the code, i.e. gaining from disorder.
As for dog-fooding: my understanding of the reason for using your own product is to make sure it gets excercised as much as possible, so *you* find the bugs instead of the customers (later on). Therefore it is not about withholding changes out of fear of breaking something. And skin in the game to me means the incentive to do a good job is that you are not woken up when you are on call. Not having skin in the game would for example be a separate support organization that handle support calls, and you (as a developer) is not affected.