Joe Leech

Product catastrophe: The top four scenarios and how to deal with them

The four major reasons for catastrophe. Up the proverbial creek? Let me pass you a paddle.

Part of what I do now is to help organisations recover from a crisis. I’ve been involved in some monumental screw-ups over the years, been party to some avoidable mistakes and have made some bad choices myself over the last 15 years. I thought I’d share the most common situations, the symptoms, causes and what to do to both before and after it goes wrong.

mr wolf the fixer
Mr Wolf the fixer from Pulp Fiction

1. We re-platformed and redesigned and our metrics dropped

We needed to re-platform. The contract was up with our current provider and the renewal was expensive. It cost us 6 figures and 18 months to replace our tech stack so we thought it was a good opportunity to redesign. We’ve launched and our metrics have dropped. Help!

I warned against this a few years ago. Re-platforming is a risk, redesigning is a risk. Doing both together just compounds that risk. Once the new product is launched and the metrics drop it’s hard to pin-point where the problem lies and it’s extremely hard to roll back both aspects of this large change. Not too mention any SEO risks associated with it too.

Don’t let this happen to you. If you are planning to re-platform keep the current design and information architecture, re-platform then iterate and optimise the design on on the new, shiny, platform.

But it feels like a good to time to redesign! We need to justify the large, “invisible” cost to stakeholders

But it’s the most dangerous time to redesign. It will cost more to the business in the long term

It’s too late, we’re live!

As hard as it is to hear the most reliable course of action is to recreate the old design and IA as much as possible on the new platform, get back to the baseline or as close as you can. Then look to make improvements.

It’s never nice to go back but this is the only sure course of action, to restore the experience and as close to possible your numbers.

2. Post a big launch and our users hate it

We’ve did everything right, we’ve user tested the site, had an open beta for users to test, we’ve A|B tested and it’s on par with the current version but our users HATE it, worst of all our CEO is ‘concerned’.

This is scariest scenario. It comes down to one of the biggest issues in product, specifically the psychology of design, users hate change.

This is particularly true for two types of product:

  1. Expert systems like call centre, intranets and tools employees use for hours a day
  2. Complex, frequently used consumer apps like internet banking, grocery shopping or commonly used ecommerce sites

Historically both of the above haven’t demonstrated the best in design and are often tightly coupled to legacy (read complex) backend systems. They don’t get updated often, every 5 – 10 years, and when they do get updated it’s a wholesale, from the ground-up redesign to bring them into the modern world.

In my talk at Mind the Product I talk about the psychology of why users hate change.

Big changes in the way a product works means the user has to relearn everything about using it.

If your job relies on you being quick with that call centre system while an angry customer is on the phone, your kids are screaming at you as you are doing the weekly online shop – you don’t have the mental energy to learn something new.

Even if the new version is demonstrably better than the old, the user would have learnt how the old crappy one worked. Invested time to get to know it as they had too, either because it’s their job or they have no, easy alternative.

Relearning how to use something complex, even if it’s easier to use than before, still requires cognitive load. More mental effort, more time, more frustration.

97% OF ALL PEOPLE WANT THE SEARCH BOX BACK ON THE LEFT SIDE. SO WHEN WILL THIS FINALLY BE FIXED ? Search on the left please repeated many times
Read the comments on Wikipedia when they changed the design of the site

Your new users will have no issues, they’ll love it but your existing users, well they won’t be happy. They resent having to learn something new and will complain. Your numbers will drop. Even worse, the users complaining will probably be your most loyal, most long term, most committed customers. Soon enough they complain loudly enough and your CEO hears. This is when I get called.

What to do?

The immediate reaction is to go back to the old version. And when this happened to me early in my career (👋 Which?). Confidence was lost in the new product and the team, the new site was canned and ¼ million £ investment written off.

Normally, after 3-6 nail-biting weeks users will have learnt the new app, will have put the effort in and will realise it’s better. Hang fire, hold off, be patient, the numbers will go back to something approaching normal.

You can A|B test the new version if you have a sufficient number of users and the time, technology and resources to do it.

There is very little you can do to prevent users hating your new product, you can only mitigate against it. Warn people, warn users, warn bosses, bosses of bosses, be prepared!

Optional beta versions rarely work as many people won’t ever try the new version and before you know it you have to support two systems long term. You will have to rip that band-aid off at some point.

The big guys, companies like Booking.com, know that you don’t do a huge redesign all in one go. Small, incremental design changes that may irk but are quickly forgotten.

3. The relationship between product team and management is breaking down

Product teams

It’s so frustrating we can’t ship anything and when we do it’s a shadow of what we initially designed

Managers

They never seem to launch anything

Or

They just build things, but not always the right things

The break down of trust is more often than not a break down in communication an understanding. Due to the parties not having shared, common knowledge and experience in business and product design and agile delivery.

Let’s talk about the product team. Perhaps the product team aren’t aligned with the business vision / strategy, or worse there is not a coherent business vision or strategy so they are building stuff that’s not what the business wants, or thinks it wants.

I help to coach product teams to be business focused and to work better with their stakeholders. Building and shipping product is straight forward if there is a shared vision and strategy. You can up-strategise but it’s hard.

As for those organisations with a vision / strategic vacuum. I suggest leaving this book in the executive bathroom: Good Strategy / Bad Strategy.

Many senior managers and C level execs don’t have digital product experience. They know the business inside and out but their domain knowledge is lower. They are lost when it comes to apps, websites, agile and modern tech delivery approaches and lets face it, we do use complex language to describe what we do.

This lack of knowledge leads to a lack of confidence to be sure their team, division or company is doing the right thing. Symptoms include: micromanaging and bike-shedding. Made famous by execs spending longer discussing the design of the bike shed than the nuclear reactor it was attached to.

I coach execs who want to make better digital decisions and work better with modern tech delivery teams. There’s no quick fix for this, management consultancies try and help here but come with their own series of challenges.

There are a variety of tech basics courses that can also be useful for execs. This is a good, remote tech basics course from Lynda / LinkedIn

4. Management Consultancy / Agency have screwed up

We’ve spent £120,000 on a piece of work with [redacted] consulting. They’ve left us with a report, some spreadsheets and prototype but we can’t be sure it’s the right thing to do. They’ve quoted us ½ mil to build it for us.

Or

Our tech agency suggested some design changes and we made them and our conversion has dropped by 20%.

This is probably the most common reason I get called in to help and often I’m asked to do a UX review of the prototype or product design changes. A bad user experience might be the symptom but it’s not cause of the underlying problem.

It’s worth exploring the major reason; Lack of experience and over confidence from the consultancy or agency.

That’s both experience in the digital domain but also experience in ecommerce or complex consumer facing apps and websites; and it’s almost always consumer facing stuff where the mistakes are made.

Warning signs to watch-out for in working with a consultancy / agency:

  • Bait and switch – The pro team pitch, the b-team deliver
  • 100% confidence in everything they do – Younger, talented, confident designers and consultants who don’t know their limits
  • Competent but not experienced staff – pretty designs, impressive PowerPoints but little large scale delivery experience and little experience in complex interactions
  • Planning, designing and strategising but no actual, making, launching or running a product experience . “Here’s our report / prototype, good luck building it!”

I’m not saying I’m a better designer or management consultant but I do know my limits, I know how to use user research, A|B testing, to launch and relaunch products and most important of all ways to mitigate risk and give some kind of certainty.

I’ve also seen a ton of poor and failing products, apps, websites, projects teams and organisations.

Having a trusted third party to question the thinking, offer some experience and just call bull-sh!t when needs be can give you the confidence all is going as well as expected.

Some general advice in working with a management consultancy or agency

  • Use consultancy / agency smarts for the right stuff
    • Don’t get a tech agency to design your new product. Get them to build it.
    • Don’t get you management consultancy to design your product, get them to define the business case, organisational process, jobs and needs.
  • Don’t be fooled by that cool UX design agency they just bought and want to introduce you to. They will probably be stretched by the new pipeline of work and many of the best people will have left following the purchase
  • Ask that the team delivering the work pitch and ask them about their personal expedience. Personal experience is very different from the organisation’s experience
  • Give them a tester engagement for a much smaller fee to learn how they work and if they are up to a larger challenge

There’s lots more to be aware off: cheap initial engagement and very expensive follow-up, unfavourable contracts, little commitment, find and replace engagements, consultant lock-in and other things but those are all subjects for another day.

The common thread amongst all of these issues is the need to take a step back and question. Are we doing the right thing? Are we approaching this is in the right way?

If you’ve got a niggling feeling that the answer might be no to either of these questions, stop and look for another way and of course get in touch if you want some quick, free advice.

Would you like me to write more about any of these subjects? Please email me or leave a comment down there.

Leave a comment

Leave a Reply

Your email address will not be published. Required fields are marked *