Thoughts about living and working in a bold new world

Most people need to feel like there is some point to their existence. It seems to be a basic human drive, and it’s responsible for the creation of art, businesses, religions and empires.

We can agree that food, shelter, health and safety are also really important. And how you prioritise these needs depends on your circumstances. So if you’re really hungry, you’re probably not going to be driven to paint a masterpiece until you find some food.

I can remember visiting my Dad after school at the police station where he worked as a detective and wondering why he wasn’t able to get up from his desk to drive me home so I didn’t have to walk all the way. But I accepted that he was very busy doing something that was more important than saving me some legwork.

This is not one of those sad stories, “Cat’s In The Cradle” style. My Dad was great. It’s more about observing the way he went to work at a certain time, finished at a certain time, and there was no possibility to break from that to do anything else. I imagine that if there was a medical emergency, he could have gotten away but certainly not because he had a flash of inspiration for a hobby he was working on, or because his 9-year-old didn’t appreciate the value of exercise.

So we can make the observation that “people who work normal jobs are not free to wander while at work”. If that sentence makes you think, “Well, duh!” then we’re in good shape to continue.

A lot of the way we think about business and work comes from the Industrial Revolution. During that period, factories arose which needed large numbers of people to do repetitive tasks for long periods of time.

Naturally, most people weren’t excited or motived to do these tasks, and given the choice they would do other things. But employers were able to offer them the choice between starvation or being able to feed their families. Most people chose the lesser of two evils. Over the following couple of centuries this model became the norm.

Fast forward to now. The industrial revolution undoubtedly raised the standard of living for most people in modern societies, so we can’t be too quick to knock it. But things are changing really, really fast. It’s increasingly becoming the case that the repetitive, boring tasks are able to be done by technology. Robots can perform amazingly complex tasks that until quite recently could only be carried out by humans.

And so more and more people are finding themselves doing jobs that humans are better at than machines, such as designing things for other humans to use, nursing people or providing emotional counselling. Things that require more abstract thinking. This is the case in our industry, which is Software Design.

When a human is needed to do a simple physical task, such as stamping shapes out of metal ingots on a production line, it works quite well to use a “carrot and stick” approach. It doesn’t matter how happy or content the person is who’s doing the job, you get good productivity by paying them and threatening to fire them if they don’t perform. The person is driven along by the stick and pulled along by the carrot. And if you give them a further away carrot, such as an annual bonus, even better.

But lately, the things that need doing are more creative in nature or require problem solving. We’ll use Software Design as an example since it’s the most familiar, but there are many others. A lot of software companies have followed the standard model for businesses, and so you find that if you want to work in the industry you’re compelled to live in a particular city in a particular country, go in to the office at a particular time and sit at a particular desk. Then you are required to attend particular meetings and finish work at a particular time. The carrot and stick are still the same - if you’re not productive you’ll get fired. If you are, you’ll get paid. And if you are very productive you’ll get a bonus at Christmas time.

Here’s the snag though. The humans in this case are not doing mindless, repetitive tasks. They’re doing artistic, creative things. For example, trying to imagine what other, very different people might be thinking when they look at a computer screen. Or trying to find a way to get a computer to do something that hasn’t been done before. If you are bored or unhappy, you can still keep doing a mindless, repetitive task. You don’t need to think about it or even really concentrate.

But when the task requires problem-solving, it’s very different. If you are stressed about whether your spouse is about to break up with you, it’ll destroy your ability to get into the head of an unknown future user or imagine new ways to approach your programming. In the Industrial Revolution-inspired model, it doesn’t matter whether you’re happy or fulfilled. The job will still get done. But in a modern, creative job it matters hugely.

So emerges the modern phenomenon that we’re so familiar with: “The Office”. The crazy situations where office workers battle each other to establish hierarchies, hoard knowledge to make themselves indispensable or bully and belittle each other. Where people spend hours surfing news sites or youtube watching cat videos after clocking in just to bide time until they can clock off.

These outward signs are just the tip of the iceberg. The actual disfunction in modern companies extends far beyond to affect the emotional wellbeing of the employees and their families.

That’s not to say that all companies are dysfunctional. Some find ways to mitigate the effects and build relatively healthy cultures where people don’t feel miserable. But the general trend is towards dysfunction and inefficiency. The point is that it’s widely recognised that the traditional model of business is rapidly failing in the modern era.

If you’re with me on the idea that people need to be happy and content in order to perform creative tasks, then we need to step back and ask, “How do people become happy and content?” That’s a really big and complex question, not made easier in that it varies from person to person.

We’ve made a good start just by recognising that it’s not irrelevant, but the dawning realisation is that it’s actually more important than anything. In fact it’s the very reason that businesses, technology and human endeavour exist at all. We’re all trying to make life “better” which, now that the other basic needs have been met, translates to “happiness and fulfilment”.

The next proposition is that 'people are not machines'. By this, I mean that we are by nature quite chaotic and unpredictable, whereas machines are designed to be be reliable, precise and predictable. We are soft and weak, whereas machines are hard and strong. We think and function in mysterious, fuzzy ways, whereas machines are well understood and can easily be fixed if they malfunction. Humans are much harder, if not impossible, to fix if they break.

Our current work model recognises the huge success of machines - due to their strength, precision, reliability and ability to be improved and repaired - and says, “Those characteristics are great. People should be like that too.” If you described somebody as “Strong, reliable and precise”, most people would agree that describes a pretty awesome person. If you witness someone who is amazingly good at something, you’d say “He’s a machine!” It pervades our thinking and culture so deeply that humans should try to emulate machines, that we’re not even aware of it.

But here’s the thing: humans are not machines. And it causes us a great deal of stress to try and emulate them. I’d suggest that it’s actually a cause of a lot of the discontent and unhappiness we see in the modern workplace. The deeply ingrained expectations that we are going to start and stop working at precise times regardless of how we’re feeling. That we will perform tasks at a predetermined rate, and that we will not have anything else going on in our lives that can distract us from performing our tasks. As much as we aspire to be machines, we’d do ourselves a big favour to accept that we are not.

This brings us back to that hard question: “How do people become happy and content?” In the context of work, a good answer would seem to be, “Allow them to be humans and not machines.” At first glance that would seem impossible. The hidden assumption is that people are naturally lazy and will not work unless forced to. And that’s partly true. We are lazy - sometimes. And people are easily distracted, so they won’t concentrate unless they have to. Well, that’s true too - sometimes.

But we can also apply ourselves ruthlessly at other times. We are forgetful and dreamy at times, and at others we have clarity and recall that a supercomputer couldn’t hope to emulate. We move between theses states - idleness and productivity, fatigue and exertion, vacuousness and concentration. To look at modern society from a height you’d be forgiven for thinking only the latter of each of those word-pairs exists, but it’s an illusion. They’re the opposite sides of the same coin, we just want to be machines so badly that we choose to only look at one side.

I have observed over time that everything has a counterpart and in order for us to be happy there needs to be balance. I’ll give you an example: I love being in the wilderness. When there is natural beauty and greenery around me and the only sounds are the wind, water and animals, I have a huge feeling of well-being. Unless I’m hiking across a lake and I fall through the ice. Or a storm blows up with howling winds and snow and I’m stuck in a tent. Then it sucks.

And I love the city. You don't have to worry about the weather killing you. The excitement and energy that you feel meeting with people, building cool technology, dining at amazing restaurants and going to live comedy are experiences I would not want to miss. But I hate sitting in traffic and large crowds of people stress me out.

So you’d think, “I enjoy this thing, so if I want to be happy I need to do it all of the time.” People get fed up with the rat-race and buy a lifestyle block in the country, then work themselves into the ground trying to maintain it. The tongue-in-cheek name I’ve heard used many times is “life sentence block”. Is that weird? Well, it’s understandable if we accept the flip-flop aspect of human nature. To be happy, we need contrast. That’s why home is sooooo wonderful when you get back after a long trip, and why the countryside seems bristling with life and adventure after you’ve been cooped up inside for a long time.

That’s what I think Yin and Yang is about. The concept seems to be that each aspect of life has opposing sides that need to be in balance in order to achieve contentment and happiness.

The catch is that it’s dynamic - things keep moving. It’s a mistake to take a snapshot of your life and work out where your balance is, then expect everything to stay like that. You have to keep constantly re-evaluating where you are between those extremes and adjusting accordingly. So I might need to spend 80% of my time in the country and 20% in the city now, but when I was younger it was probably the other way around.

And it applies to every activity you undertake, all the time. The amount of time you spend with your partner vs doing things on your own. How much time you spend working vs relaxing. How much time you spend sleeping vs being awake. If you think about people who live at the extremes, they rarely seem healthy or happy. People who work all the time and never relax. Those who sleep all the time and never get up. People who become hermits and never socialise. Those who party all the time and can’t bear to be alone.

'Agile Development' is a term that has rapidly grown in prominence in recent years. It’s become one of those terms that is bounced around so much that most people have heard it and are aware that it’s a thing, even if they might not know what kind of thing it is. Companies talk about “Going Agile”, so it’s reasonable to guess that it’s a process of some sort. That’s most people’s perception, though what it actually involves is somewhat mysterious still.

So what is Agile? Well, for a start it’s not a process. Rather, it’s a way of looking at the world. You can’t “Go Agile” any more than you can “Go Buddhist”. Even if the CEO comes to believe that Buddhism might be more effective for the company, it’s just not going to work to stand up and declare, “Guys, we’re going Buddhist!”. Buddhism isn’t in and of itself going to make every team more effective. It’s not a matter of whether Buddhism is “better” or not - it’s just a different way of thinking about things that may yield some advantages if everybody on the team is on the same page. And you can say the same for any other world view. The analogy works for “Christianity”, “Atheism”, “Science”, “Marxism”, “Capitalism”, “Socialism” or any other ideology that a group of people share.

Some ideologies have come to have a positive or negative connotation, depending on who you talk to. But ultimately, none of these ideologies are “good” or “bad”. Any more than nuclear power is good or bad, or guns are good or bad. They are tools that are suited for different purposes, and how they are interpreted and used depends on the groups of people who use them. And those uses can vary widely.

OK, so Agile is not a process. It is an ideology. It’s not “good” or “bad”, it’s just a new way of thinking about things. What is it?

It starts by recognising that the nature of the tasks we’re doing these days is inherently unpredictable. As we move out of the Industrial Age and into the Information Age, human workers are increasingly being asked to solve problems that haven’t been solved before and to get into the heads of other people to try and make things that they want to use. But nobody knows at the outset what the end result is going to look like. The traditional approach of using tried and tested methods to make predictable products on a large scale requires things to be thought out and mapped very thoroughly to avoid waste. So large amounts of resources can be put into the planning, with confidence that it will pay off. If you are are going to make a large building, you could just get a group of labourers together and start pouring concrete, but pretty soon you’re going to find that you forgot to put the drainage pipes in the foundation and you either have to pull the work down and start again or - more commonly because you can’t afford to do that - tack them onto the outside of the building. Obviously, since it is well known how to build a building, it would be worth the investment to have an architect draw up plans.

This is what’s know as “Waterfall Development”. Where a large investment is made in up-front planning so that the end result can be predicted, which gives everyone confidence to go ahead. And Waterfall Development has been hugely effective, so companies naturally follow that blueprint without really questioning it. Until it stops working.

The reason it’s starting to fail is that modern companies are undertaking tasks that are not well known and cannot be mapped out. Not only that, but the goalposts keep moving. There may be a vague idea of what the business goal is, but it’s very fuzzy, such as: “There’s an opportunity for a better ride share app”. But in modern information business, whether that app succeeds or not usually comes down to some very fine details which can be easily overlooked, if not impossible to spot at the outset.

It’s natural to think that it’s just a matter of planning harder, thinking things through more thoroughly. Surely the only reason the project is failing is because the designers were lazy or the programmers incompetent? And sure, that’s been the attitude for a lot of companies. Teams have been fired and reinstated only to find that the project and the company keeps failing.

So a key change in Agile thinking is to accept that this unpredictability is an inherent part of the problem, not a failing on the part of the people trying to solve it. The natural effect of that is to take a more experimental approach to product development. That is, try something small first that is not too expensive and that people don’t get too heavily invested in, and test it in the real world. Put it in the hands of people who are actually wanting to use it. Then take what you learned and iterate on it. Even be prepared to throw it away and start again. Lots of small iterations and figuring out the solution - and clarifying the goal - as you go. Agile companies often find that they end up making something completely different from what they envisioned as they realise what customers actually need. It’s humbling and somewhat miraculous to go through this process as you discover how far off you were at the start and how much better the end result is than what you could have imagined.

The great thing about this is that humans are uniquely suited to doing chaotic jobs where the path is unknown and the goal posts keep shifting. It’s the kind of thing that Artificial Intelligence can only hope to emulate and one area where machines still can’t compete with us. I’d even suggest that it’s a much more natural fit for how we actually operate when we’re allowed to be ourselves. So it seems like a wonderful answer. However, there’s a catch.

In the early days when Agile Development was gaining traction in Silicon Valley, a lot of startups began to take it to the extreme, in “If a little is good, then more must be better” fashion. So in pushing back against the traditional Waterfall approach, they started to make things with no plan whatsoever. They just made what they thought would be cool and made it up as they went along. There were some huge successes where small teams of 1 to 3 people built the likes of Facebook and Google. But as the sizes of the teams grew, there also came some big failures. When there are only a few people, it’s possible to be buzzing around chaotically and still be able to see what each other is doing. So Bob doesn’t need to tell you, “I’ve had a new idea and I’m following up on it” because you can look over and see what he’s doing and say, “Hey Bob, that looks cool. What are you up to?” Whatever’s happening it won’t be long before other teammates notice and catch on, and the team tends to overall stick together. But as the team grows, it becomes harder and harder to see the big picture. You can think you’re working on part of a masterpiece only to find out that 2 weeks ago the plan changed, and the work you were doing is no longer relevant. This leads to frustration, feeling like your contribution was a waste of time, resentment and finger-pointing that breaks down the team morale.

So we’ve mentioned 2 different kinds of Agile failure: the first is when an existing company which thinks traditionally tries to force their employees to “Go Agile”. And the second is when a startup initially has success thinking agile but it breaks down as the team grows. They are 2 different problems: the first is caused because the team doesn’t understand what Agile is, and the second is caused by lack of fluid communication in a larger team. Of course, if you did take a large team and manage to bring them all around to an agile way of thinking, you’d still then have to deal with the second issue.

There is so much confusion about Agile and failure in trying to adopt it, you’d be forgiven for thinking it was all just smoke and mirrors. Except that it’s not. The answer, like most things in life, lies in the balance.

When it comes to Agile, you do need to accept the uncertainty, take some risks and trust that solutions will emerge. But it’s a mistake to think you can scale it up without planning. You do need planning, just very different planning from what we’d normally expect with a Waterfall approach. If your team have to document every thought and send emails to everybody else in the company for every decision they make, and then wait for higher-ups to make decisions before proceeding, things will grind to a halt in the resulting molasses. There are just too many decisions to be made, and any of them could be critical. And the goal will shift before anybody looks up and notices it, or an obstacle will roll across the path undetected until you crash into it.

Agile thinking involves a higher level of tolerance for mistakes. That doesn’t mean that incompetence should be rewarded, but just that we want to shift the balance away from making people terrified to make a single mistake. The team members need to be empowered to make decisions and keep moving in confidence that their colleagues will be able to see what they are doing, and support them in changing direction if their decision doesn’t work out. If one extreme of the spectrum is being ostracised for any small error, and the other extreme is allowing ongoing incompetence, it’s possible to live somewhere in the middle, recognising that errors are part of the process. We need to try things out in order to move forward, and find ways of making those mistakes as inexpensive as possible. The natural inclination to build a team of experts who never makes a mistake is a fallacy. We need to be empowered to take some risks, trust that our team-mates are competent and get moving.