Data Driven, Data Informed or Data Decorated?
The Difference Between Using Data Well and Hiding Behind It
Hi I’m Fiona, Welcome to the Turquoise Bit my bi-weekly look at stuff that can change how you think, decide and lead. If this is your first read, please consider signing up to receive future posts directly. Read time 4 min
Learning to use data also showed me when it wasn’t enough.
In the early days of advertising at Amazon, I remember the big debates about what went on the homepage - the premium ‘real estate’ of world’s largest online shop. Was it a Prime product, a major deal or any other thing we believed customers wanted to see? Or was it advertising space, sold to the highest bidder? It is a challenge other online platforms face too.
The data at that moment could tell you which one made more money at any particular time, but it could not tell you which one was right, nor what matched the long-term. The customer was always meant to be at the centre of that answer, and yet the customer isn’t always represented in the data served up. It depends entirely on how you set up the metric and what you consider matters.
Why I care?
It’s perhaps worth sharing why I’ve spent over a year writing a book about this type of decision. Writing may look like a strange detour from a career spent leading, but data and judgment are with me every day as well as where they go once a decision has been made. And as a leader it is always about helping others see. I kept seeing something that didn’t sit right, and the things that don’t sit right are usually worth a second look, so I dived in.
I am a huge fan of data. I led for 20 years with relatively little, then ten more with an abundance. I see how it has changed what we’re capable of deciding, and changed it for the better. But data doesn’t take you all the way. And if it isn’t set up correctly, I’m not sure it helps you at all necessarily.
The muscle we are forgetting to train
When I look at how companies spend on data. Billions go into infrastructure, into joining up systems, some of which were never designed to talk to each other, and into data capability of every kind. Everyone is aiming to be ‘data driven’. Alarmingly I think, a fraction of the investment that goes into developing data, goes into developing judgment.
We are building the data muscle and just assuming that the judgment muscle will grow alongside it. It doesn’t. It atrophies or doesn’t even get built. Use it or you lose it very much applies here.
I spent six years working inside one of the most data-driven companies in the world. But judgment was never absent there. Amazon is in many cases data driven, but it is also unquestionably data informed. Data is used consistently to augment the decision, it doesn’t make all decisions alone. Judgement is what interrogates it, audits it and leverages it.
Take investment judgment. I spent years pushing to make things faster and cheaper for customers. The cost of that sat clearly in the data, in black and white, without ambiguity. We understood the inputs, but we discussed the opportunities and the adjustments. We did things because we believed it was right for the customer and sometimes before we could prove it back to the business.
Take a conscience related judgment. We made product recalls or put stock on hold. Extra guardrails that costs margin, yet protects someone who will never know it was actually done necessarily. That’s not a question of how much we would consider spending to fix something, rather it’s a question of what’s right.
The idea that judgement is not needed is simply wrong and short sighted. But so too is the idea that everyone knows how to use data to the max too.
Data alone doesn’t help
Some leaders hide from data, not willing to listen or learn to what data has to say or leaning on instinct and call it experience. Others hide behind it, waving a dashboard to justify a budget, build prestige or win an argument with a peer. Neither one is leadership.
Then there’s a third failure worth mentioning because it isn’t data’s fault. I call it data decorated — when data gets reached for after judgment has already made the call, purely to dress the decision up and make it look defensible. When that happens, data hasn’t let anyone down. It was never asked to inform anything. It was asked to perform or protect and it can end up misleading.
The two together, data informing and judgment deciding, is where the real strength lives. Either one alone, and something goes missing.
We see this outside work too, we do the homework on the price of a house, the mortgage the repayments, the costs etc. yet we let the best one pass because ‘ we didn’t have a good feel about it when we walk in.’
Or the job offer we get after much negotiation and all the attributes in the job descriptions line up well, but we meet the team or the manager, and decide the culture or vibe is not for us.
The final mile
Data narrows the field. It does that brilliantly, better than it has ever done before. But judgment is still needed for the final mile, on the calls that actually count. Sometimes that final mile is commercial, and you can model your way close to an answer. But sometimes that final mile needs conscience. And conscience is something data will never see.
As we bring more AI into how we work, I don’t think judgment disappears. I think it concentrates. The decisions left for us become the ones the data genuinely cannot answer, which makes them harder, not easier, and more consequential than before. So surely now, more than ever, is the moment to invest in judgment the way we’ve invested in data?
I’m not writing this as an academic. I’m writing it as a practitioner who led for three decades, some of it with rich data, some of it with almost none. And the honest answer is that when the data is in front of me and the call is still hard, I ask myself a short, unglamorous set of questions. What does my conscience say? What do my values say? What’s right for the people on the other end of this, the ones a metric will never quite capture. Would this decision still hold up if I had to explain it to all of them, not just the ones who’ll never ask me.
We have a long way to go in our relationship with data. It isn’t the enemy of judgment. It’s the thing that should be standing right beside it partnering - together stronger.
It is time to invest in judgement too to get the best of both.
Look again this week: Be aware of where you use data this week and ask are you narrowing the field or letting go of the responsibility to choose?
Have a great week