Working with technology, it’s rare that you get to build something totally new without having to think about systems or services that already exist.
It’s much more common to have one or more existing systems that support the running of the business – from complex spreadsheets or databases to large back-office systems.
When you set out to develop new digital services, you often have to think about how to feed data into, or get information out of, these existing systems.
But these often have characteristics that make it difficult. Maybe they’ve grown and expanded beyond their original purpose. Or they’re slow and costly to change. They might be difficult to scale, or be tied up with mission-critical tasks that limit their availability.
On the other side, your new digital services need to be responsive to change so you can meet user needs. You need to move quickly and try out new things. Your services need to scale to meet demand and need to be available at all hours.
These are two very different cultures, both technically and operationally. How do you make the old and new work together?
There are many ways of connecting systems together, from syncing data between old and new, to developing APIs to allow information from one to update the other. That’s the kind of thing that technologists often call an “integration”. But where do you start?
It’s tempting to try and solve the problem in one go with one solution – particularly if the existing system is difficult or costly to change. But doing this comes with its own set of problems.
Often, existing systems have grown and become more complex over time. Designing and implementing one single monolithic integration for everything will take a long time, and your new digital service won’t work until it’s all done.
It’s likely you don’t know everything that your new digital service will need from day one, so you’re going to be making guesses which might not be correct and will necessitate changes later on. The more larger and complex the integration, the longer it’ll take to change later.
The alternative is to start small and design integrations based on user needs – in exactly the same way as you would develop the new digital service.
Developing integrations based on user needs means:
- you can start small and think about what the right type of integration is for each problem
- you can start quicker – you don’t need to model the whole universe in one go, just the bits you need
- you get an opportunity to think about whether the existing thing makes sense – maybe it needs to be tweaked or changed
- you can also work out where you might need to put in more effort, and spend it in the right places
- you can isolate each part of your integration, making it easier to iterate, improve and scale as needs change
In 2015, I was working with the UK’s Driver and Vehicle Licencing Agency (DVLA) to develop new digital services for buying and selling vehicles.
They had to knit together new and old systems that held and maintained the official record. They were responsible for printing official documents and responded to queries from the police. These systems were difficult to change.
At least one old system ran complex batch-processing overnight, which meant it was unavailable for 6 hours a day. A lot of data was captured and stored, even though it wasn’t necessary to run the service. Most of it wasn’t relevant to buying and selling a car.
Designing an integration to account for all those things could have taken years as each issue was complex and had its own rules.
We started small. We developed an integration that dealt with selling a car to a car dealer. We followed it with a second version that covered buying a car from a dealer (which was a little more complex). Finally, we looked at dealing with private sales between individuals (the most complicated scenario).
The result: we didn’t develop integrations that weren’t needed. We were able to change processes and think about collecting data in a different way. We were able to improve bit by bit, without waiting for a big bang.
Old IT systems aren’t necessarily a bad thing. Tackling them in today’s internet era doesn’t mean tearing everything down, but it does mean applying some thought to the process before you start. Breaking the problem, and then the work, down into smaller pieces is the best way to start.