Thursday 16 August 2018

Adapt or die: don't try to rewrite your tech history - work around instead

There's more to startup choices than PC versus Mac, but racing ahead can leave 'tech debt'

Mark Zuckerburg used PHP to build the first version of Facebook, and when it was no longer fit for purpose they kept the language and rewrote the foundation - something most start-ups won't be able to do
Mark Zuckerburg used PHP to build the first version of Facebook, and when it was no longer fit for purpose they kept the language and rewrote the foundation - something most start-ups won't be able to do

Richard Rodger

This is the third part of a three-part series on the technology choices that we have made as a startup in our first year of existence, and the consequences of those choices. In part one I discussed our key early decision: don't build any technology and just use existing tools. In part two I discussed our decision to build a Minimum Viable Product (MVP) without worrying about long-term architecture.

But now that our MVP is launched and has been up and running since March, and has proven a valuable sales tool in generating private early-adopter trials, we need to start thinking longer term.

One way that we've made our lives easier, and given ourselves cover to make more strategic technology choices, is by not opening public registrations yet.

Instead, we've focused on growing our community and working with them to build a better product - the proof of that pudding won't come out of the oven until later this year.

Building a business is all about taking on debt. There's no other way to do it. You have to invest in a market proposition before you can reap the rewards.

A technology startup takes on 'debt' in various forms (ironically, traditional business loans are almost never seen - startups are too risky). Angel investors and later, venture capitalists, invest in the company because they hope that it will generate outsize returns.

Their 'debt' is repaid in the rising value of the shares of the company.

It is also repaid in the respect that the executive team must accord to these new shareholders, who must be brought on board (and onto the board) as valuable members of the leadership team. This debt is not easy to settle, and interest will accumulate rapidly if either side breaks trust.

But another form of debt is known as 'technical debt'. It is just as real as financial debt, and has a far greater impact on the business. Technical debt is the balance between high-quality engineering, and short-term business needs.

Facebook provides us with a good example. The original version of the system was built using a programming language called PHP (originally an abbreviation of 'Personal Home Pages').

This language is not seen as acceptable for serious enterprise software development. It has many design flaws compared to something like Java, or C# (C 'sharp'), both of which were designed very carefully by talented software engineers at Sun Microsystems and Microsoft, respectively. But it was good enough for Mark Zuckerberg to build the first version of Facebook.

Later, Facebook was stuck with millions of line of code written in PHP, and no way to reverse out the accumulated 'technical debt' that they represented. They faced performance and scaling challenges because PHP was weak where they needed it to be strong. A solution was urgently needed.

At this point, there is often a call in the engineering team for a 'rewrite'. Throw all the old stuff out, and rebuild from the ground up. It will all be so much better the second time around now that we know what we really need - the promised land awaits! If you've ever been involved in rolling out a new software system to replace an old one, you'll know how painful that can be. Most of these projects end in abject failure, and many column inches are written on the hundreds of millions of euro, dollars and pounds wasted.

As an entrepreneur running a startup, you will face this choice one day, and you must reject it sternly. Such a project will always end in failure.

The esteemed IBM engineer Fred Brooks, who led the team that designed many of their original mainframes, calls it the 'second system effect'. The architects get so excited by the possibility of building something new and shiny that they get lost in a maze of their own complexity.

If something useful does emerge, it will be years late, far over budget, and barely functional. A large corporation, or a government body can survive such a catastrophe, but a startup will be long dead.

You are in direct competition with other startups (probably better funded) for market share. If they can execute faster by delivering better technology sooner, you lose.

What did Facebook do about PHP? They kept the language and rewrote the foundation. They create a new internal dialect of PHP called HipHop that was faster and more scalable, and could take them up to billions of users.

Yes, they could hire a lot of PhDs to do this, and you won't be able to do that, but you can learn a basic lesson: don't rewrite. Instead, adapt.

Facebook are stuck with PHP, and they will always pay a price for the sloppiness of the language, but they have mitigated most of the problems by figuring out a workaround.

This was the better business decision, and allowed them to keep adding features and outcompeting and growing. Growth and revenue are great solutions to technical problems!

Knowing that you will face this moment, is there anyway to make it less painful, to be slightly more flexible, so that technical debt does not cost you so much that it puts your ability to execute in danger? Unfortunately, to quote Fred Brooks, there is "No silver bullet". This is a fundamental trade-off - a business decision you can't avoid. Instead, you must ask what the criteria are for the decision?

Do you (a) build as much as possible, as quickly as possible, technical consequences be damned, to gain dominant market share, or do you (b) build a platform that gives you many years of flexibility, but means your market entry is delayed so much that others will probably get ahead of you?

With the first option you might gain an early advantage, but end up losing in the long-term when more robust competitors slowly build up more defensible and deeper technical features (eg Google - a late comer to the search engine space). Or, you could also be lucky, and, like Facebook, end up so dominant that you can't be dislodged, or with sufficient market share to justify a decent acquisition by a larger company.

With the second, you might build the best technology, but never break into the market because you can't get customers to move to your solution (eg Betamax vs VHS - that old classic). On the other hand, if your tech is very much better, and you can continue to deliver at high speed (because you have low technical debt), then you can break monopolies and take over the market (like the iPhone and Android-killing Nokia).

Both strategies are viable, and I think the determinant has to be whether your market is new and growing, or old and in need of disruption.

Go with (a) if new and growing - an early lead in market share is pretty much all that matters.

Go with (b) if old and in need of disruption - you'll need great innovation and execution to dislodge incumbents.

We are more b) than a). These are extreme endpoints of a strategic spectrum. The middle of that spectrum is not a good place to be - you do have to choose, but there is definitely tactical scope at each end.

In the case of voxgig, our space is technology conferences. There are many incumbents, and we think it's time for a little disruption. But we need to be able to execute more rapidly than the incumbents on the tech side if we are ever to accelerate beyond them. That said, we also need to work with trial customers to understand their needs, so we have to build something.

Our technology choice, post-MVP, is thus to build a platform to reduce technical debt, but not so much as to impact our ability to deliver useful software today. Instead of trying to build the perfect platform, and cover all use-cases and scenarios, we are building a simple platform that is developed in alternating stages with feature development.

On a project management level it works like this: first we build out platform infrastructure. Then we work with trial clients to build features. This work identifies holes in our platform (common functionality that is missing), and then we go back to the platform to improve.

We'll continue this cycle for the next year. But over that year, as the engineering team grows, we'll eventually build both platform and features in parallel, and then move even faster. Our platform will reduce the impact of technical debt and make this possible.

There is still a technical risk here - if we get the platform architecture wrong, we'll end up stuck, as it can't be changed without a rewrite. Then we'll slow down in our feature delivery. This always happens anyway to some extent, but you want to stay flexible enough to keep ahead of the curve.

One technical tactic that we are using is something called the 'microservices' architecture - we break our platform into lots of small replaceable parts. This is very effective at keeping technical debt under control.

Stepping back, the key to making the right technology choices is not about the technology itself at all. It is about understanding the dynamics of your market, and you how you will use technology to cope with those dynamics. Pro tip: if your CTO isn't raising this as a key decision, they're not a CTO.

(Newsletter update: 2,935 subscribers - nearly at 3,000! Open rate 11pc. Next week I'll be doing a full review of our newsletter growth strategy in year one, and the results - and a basic mistake I made as a leader.)

Richard Rodger is the founder of voxgig. He is a former co-founder of Nearform, a technology consultancy firm based in Waterford.

Indo Business

Business Newsletter

Read the leading stories from the world of Business.

Also in Business