If you already have experience and you know the right answer, then the cost of building it right the first time is low. This is huge. Having the right answer is extremely valuable. The cost of finding problems early on is important. This is why hiring is important. This is why training people is important.

In this article, I want to illuminate the cost of a flawed design during different stages of building a product and then I will extend the idea into figuring out the best way to build products in today’s world.

Cost of Flawed Design at various stages

Design Phase

The cost of finding a problem with the design of a bridge or a Skyscrapper early on is important. Having good engineers find the problem during design process is cheap. Some work will be redone but shouldn’t usually cause too many delays in project. People with experience still makes mistake (It’s the difference between experienced and expert), but finding those mistakes is still easy in the beginning.

Building Phase

Now imagine finding a problem in design during construction. It’s a project management disaster. Some stuff might have to be scrapped. Walls might have to be taken down, beams might have to be readjusted. This is costly financially, usually demoralizing and takes off a project off its plans. This usually happens but the mistakes that are found are small (have to move structural beam a little bit). It is still costly. People with experience can usually come up with solutions to these problems quick. The cost of fixing is large but coming up with solutions is usually not too expensive.

Post Construction/Production Phase

The worst scenario is when a project is complete, but we realize in retrospect a major flaw in design. Sometimes the design flaws are aesthetics such as the Chelsea Wateside Park . Sometimes its a major structural/architectural flaw. There are two types, flaws that are discovered early such as the Ottowa-Gatinuea Bridge and those are not discovered until a disaster happens such as the Florida Bridge Collapse at University of Miami.

The cost of fixing a problem that is discovered after completion of project is extremely high. Sometimes there is no way to fix it and it must be torn down, thus cost is all that wasted money and effort as well as money and effort to demolish. It is not easy fixing a bridge and cost and effort are high to fix something afters it designed and built.

If the problem goes unnoticed, the cost is not just money, but also lives. And human lives are the most expensive.

The Importance of good design

I choose to use the bridge metaphor because bridges are foundational structures of our modern society. Governments, societies and regulators spend a great deal of time regulating the engineering industry, putting a really high bar on becoming a certified engineer in order to design and build bridges. Despite that bridges fail and lives are lost.

I want to make it clear that having good systems to detect problems early is the only way to eliminate the problems of flawed design. There are two things I advocate for: hiring good people and building testing systems to verify things work.

Back in the old days only the smartest were given the tasks for building bridges, because they are important, because a mistake costs lives. But we didn’t just trust those smart people, once a bridge is built, they would roll a huge piece of rock on the bridge to make sure it is structurally sound.

I don’t want to dive any deeper into topics of bridges or structures. The takeaway here is doing it right the first time is important and reduces the risk. The best builders build it right the first time.

Building Your Product: Building It Right The First Time

When you are building a new product, be it a software product or anything else, building things right the first time is very important. I am not advocating over-engineering or overthinking our products. I am simply advocating that building it right the first time is something we should aspire to do, and that it should be a calculated risk.

If you are not risking lives, or a lot of money, you are more free to experiment and break things more. Sometimes you don’t know the answer right off the bat.

But I am not here to talk about experiments. I am here to talk about the cases where we know how to build something. There is a lot of things we know how to build. We know how to build bridges. We know how to build electrical systems. We know how to implant a mechanical device in your heart to keep your heart working. We know how to build software to get us to the moon, and back!

I am going to be make two claims:

1. that a lot of what we do today, we already know how to do
2. For those tasks we should get it right the first time

The other claim I am making here, is that there are things we don’t know how to do. We don’t know what product will stick, or how people will interact with new technology X. I believe that those are the things we should be spending our times focusing on and not things we already know how to do.

Example of the App that Can’t Scale

I heard the story many many times. Some people come up with a great idea and they decide to build an App/software to do it. It’s great; people love it; and it becomes super popular. They get funding and decide to expand. NOW HERE IS THE CRITICAL POINT. We have some funding; we need to move fast and deliver value. So here is where the mistake is made.

A lot of companies would choose to not invest a lot in making things work from the get go. We know how to build apps. We know how to build services. There are many tools that do these things. SO MANY! So why is it that Apps crash when they become successful now.

Examples below of bugs inability to scale:

• Pokemon go launch disaster
• Fornite Battle Royale Servers down
• Microsoft’s Wunderlist outage

How to Fix It: Hire the Experts

Is scaling so hard? It is, but also it is not. There is so many experts in the area. I think there is a mentality in software that anybody can do anything. That an engineer in embedded system design would know everything about networking or whatever.

It’s WRONG. People specialize and just like any other field, we should strive to find people are are experts in the field. We should hire them; we should develop them ;we can avoid a lot of these outages that way.

I think great software developers know when they don’t know something. Just having a great developer who will speak up, admit what they don’t know, and ask for more resources from management is a great asset. It reduces your cost significantly. It reduces the cost of breaking things in the future. It reduces the cost of failed design.

Every server crash is costing your product a lot of money, as well as engineering effort in order to fix it.

Why can’t we build it right the first time?

How To Fix It: Test It To Make Sure It Works

Sometimes you have some really amazing people that work on your product. And they can go figure out anything. I truly believe those people exist. They are the same people who know when they don’t know something and they go figure it out. They go investigate and learn.

My problem with learning is its not enough. You are a novice now in that area. You are no expert yet. And even experts make mistakes.

So how do we prevent ourselves from making mistakes. You test your stuff. Engineers test their bridges. Using modern software simulations, today’s engineers put their bridges, skyscrapers and submarines under heavy simulated computer testing to make sure their structures are sound. An inexperienced engineer will build something that would break easily, and have to redesign it several times. An experienced engineer will design it right from the get go (usually) or just require minor modifications to make it work.

But even great engineers test their things.

Sometimes a test is just having someone look at it. In construction, multiple people usually need to sign off on a project before its approved for building.

When you are building a product, make sure you are testing it. Make sure you test all the assumptions. test it for user experience. Test it for compatibility. Test the colors of your design. If you expect scale, test your scaling!!!!

The cost of the Flawed Design

We spend a lot of time figuring out the new innovative part of our software. Sometimes its the product itself thats innovative and we spend all our time focusing on testing that (which is good), but we might forget to test the things that we think we know well. well because we all make mistakes.

If you are a founder, or a manager or the lead for a product, you probably spend a lot of your time making sure that the product vision and the innovation behind what you are doing is right.

But don’t forget to hire the right people! And don’t forget that those right people always test their stuff, no matter how small it is!

Future work:

In a set of series articles, I am going to publish what I consider “doing it right” the first time means for specific software engineering topics. I hope by doing so, I can help educate others about my approaches. But I also hope that others with more experience can poke holes in my design.