“We have to rewrite it!”
I’ve done a lot of interviews recently, and a common theme among them — and among prior interviews over the years too — is companies who want to go from an existing “legacy” system to their shiny “new” system: They’ve concluded that the existing “legacy” system isn’t meeting their needs, and that a “new” system is necessary.
I call these “version 2.0” projects, because quite a lot of them involve taking an “original” system that’s been keeping the company alive since its inception, and making a replacement for it. I’ve been on several teams doing “version 2.0” projects over the years, and I’ve even started a couple of those projects, and there’s one truth that has been consistently valid among every single one of those exercises:
Don’t do it.
It’s not like this is new advice. Fred Brooks wrote about it in the “Second System Effect.” Joel Spolsky wrote about it as “Things You Should Never Do.” Anybody who’s led a software team or been on a software team throughout these kinds of projects knows how bad they can get: They’re always way over time and way over budget and usually buggy and glitchy and make your customers unhappy. And yet — we all keep doing it.
Now, if you’re dead-set on replacing your old system outright, don’t say I didn’t warn you. It’s going to go over budget and over time. My experience is that whatever time and money you budgeted, you’re going to need 3x to 5x that much time and money to replace your old system, if not more. The new system is going to be buggy and glitchy. You’re going to have new features, sure, but you’re going to miss features and lots of corner cases that the old system supported and that your customers depended on. Your customers are going to complain about the new system a lot more than they ever complained about the old system. You’re going to grow to hate the code long before you release it, and hate it even more after it’s released.
Don’t do it.
I’ve said it for a decade now to everyone who will listen: The new system is the new legacy system.
The new system is the new legacy system.
“But our old software is so ancient and awful!” you say. “The programming team hates it! It’s impossible to hire for it! How can we possibly get to modern code without just replacing it?”
That’s what I really want to address here today. Not why — there’s an infinity of cautionary tales to convince you why you should never replace your old system — but how? How can you realistically stick with your legacy system? What can you do to make it less awful? What can you do to modernize it, for real?
Real Refactoring
Refactor is a funny word in the software industry. In some circles it’s feared. Often, it’s treated as a fancy modern way of saying “rewrite the code.” But that’s not what it means, and not what it has ever truly meant. So let’s talk about it a little bit.
Martin Fowler wrote a whole book Refactoring, and if you’re a professional software developer and you haven’t read this book, go to your nearest bookstore, buy it, and read it, and memorize the important parts.
Refactoring doesn’t mean rewriting. Let’s talk about what it really is. Refactoring is:
- A well-defined mechanical operation, to —
- Change the code —
- Without changing its behavior.
You’ve probably already done this, but you didn’t even think about it as a refactor. If I gave you this code —
function f(x) { return x + 1; }
function g(y) { return y * 2; }
and you reordered the functions to read like this —
function g(y) { return y * 2; }
function f(x) { return x + 1; }
that is an actual refactor. It changes the code. It’s a mechanical operation, so well-defined that the computer could conceivably do it for you (and in some IDEs, the IDE can do it for you!). But it doesn’t change anything about the code’s behavior — it just moves some stuff around.
If you read Fowler’s book, you’ll learn a long list of different refactoring techniques. There are simple-seeming things like rename a variable or rename a function. There are more sophisticated techniques like extract function or inline function. The most critical part of all of these, though, is that they’re all mechanical, not just “I’m going to change this code,” but predictable, well-defined steps that result in a specific, predictable, well-defined change. Notably, every true refactor has an inverse operation: Rename x to y can be undone by renaming y to x. Extracting a function can be undone by inlining that new function back to where its contents came from. You can always go forward, and you can always go back.
True refactors like this are always safe. You can always perform them on any code base, in any language, because by definition, they don’t change the code’s behavior — only its structure.
And what that means is that your existing code base, that legacy nightmare disaster mess, can be changed. It can be improved. It will take time, but there’s no such thing as code that is tangled into such tight knots that it can’t be untied. Applied correctly, refactors can turn anything into anything else.
Refactors can turn anything into anything else.
Now I’m going beyond the refactors listed in Fowler’s book, because the principles involved can apply more broadly. For example, let’s say you have a horrible SQL database schema that everyone hates. You can perform rename table or rename column on it, as long as you update the software that works with it to respect the new names. You can reorder columns. You can add an empty nullable column. You can safely delete a column that contains all NULL values. You can add a column that perfectly duplicates the data of another column but uses a SQL calculated expression to express it in a different way. As long as you mirror changes in your code, you can transform your data model from anything to anything too.
There’s no code so legacy that it can’t be turned into something modern. Even if your backend is a hideous monster written in the most inscrutable MUMPS code, there are ways to transform it into code written in Cava, using Power Data 3000 and the Dynamic Web Excellence 7 framework. It’ll take time, to be sure, but as long as it keeps running your business — as long as you don’t rewrite it! — you have all the time you need.
I Don’t Believe You
I’ve done this. Lots of times. I once took 50,000 lines of hacky VB.NET code from 2006 and turned it into modern C#, and I did that in under a month. The resulting C# code wasn’t pretty, but it was modern, and it was bug-for-bug, feature-for-feature, drop-in-replacement-identical to the original, such that after I did it, the original could be completely retired. Others went in afterward and made even more refactors and cleanups to that C# code, and the resulting library still powers a multimillion-dollar system to this day*.
(*Probably. I haven’t worked there in some years, but I know that system is still alive and crunching data.)
But because you don’t believe me, let’s run through an exercise that’s similar to that one.
We don’t have enough blog space to cover every possible technique, but let’s do something substantial and realistic. We’ll take a small chunk of old VB.NET WinForms code and turn it into a C# Web API, presumably against which a React or Angular frontend could then be added. This is the sort of problem faced by lots of teams dealing with a legacy codebase, and it will demonstrate many techniques you can use in the real world.
A Sample Exercise
Let’s first look at the original code for our example exercise. Somebody clicks a button on a WinForms app, and the program saves name changes to a database:
Public Class Form1
Private Sub btnSave_Click(sender As Object, e As EventArgs) Handles btnSave.Click
Dim FirstName As String = editFirstName.Text
Dim LastName As String = editLastName.Text
Dim ID AS integer = Convert.ToInt32(HIDDENIdField.Text)
USING con As New SqlConnection("server=.;database=MyDatabase;integrated security=true")
Dim Cmd As SqlCommand = New SqlCommand("update PERSONS Set FirstName='" & FirstName & "', lastname = '" & LastName & "' where ID = " & CStr(ID))
con.Open()
cmd.ExecuteNonQuery()
End USING
End Sub
End Class
This code is recognizably hideous — just like a lot of real-world “organically-grown” code. It’s a horrible, inconsistent mess, tied directly to the “Save” button. It uses bad practices, and maybe even crashes the program every once in a while. But it’s running our business! How do can we possibly modernize it?
The first step is to identify what’s worth keeping here and what isn’t. In this case, the UI team is building a pretty, modern web-based UI in front of this business logic, so the WinForms part of the code is going to end up on the chopping block by the time we’re done. We could try to salvage the UI code if the business really wanted it, but they hired a designer and some ReactJS devs to make something pretty: So our goal is just to turn this into a REST API, and we’re going to focus on the business logic. We’re going to start by using an extract function or method refactor here to split this code in half; this change doesn’t do anything except move some logic out of the button handler:
Public Class Form1
Private Sub btnSave_Click(sender As Object, e As EventArgs) Handles btnSave.Click
Dim FirstName As String = editFirstName.Text
Dim LastName As String = editLastName.Text
Dim ID AS integer = Convert.ToInt32(HIDDENIdField.Text)
Save(FirstName, LastName, ID)
End Sub
Private Sub Save(FirstName As String, LastName As String, ID As Integer)
USING con As New SqlConnection("server=.;database=MyDatabase;integrated security=true")
Dim Cmd As SqlCommand = New SqlCommand("update PERSONS Set FirstName='" & FirstName & "', lastname = '" & LastName & "' where ID = " & CStr(ID))
con.Open()
cmd.ExecuteNonQuery()
End USING
End Sub
End Class
It’s still just as hideous as it was before, but now there’s a distinct separation between the parts we want to keep and the parts we’re going to toss. Importantly, when we did this, nothing broke: It still runs as well as it ever did. You can’t ship the Web API yet, but you can still ship the old software.
Because we’re going to do this kind of extraction for every one of the button handlers on the form (“Add Person” and “Delete Person” and “Add Company” and so on), we’re going to end up with a bunch of extracted functions like this. Let’s move them into their own class — into their own source file — so that we keep all of the operational logic code — the parts we’re going to keep — truly separate from the legacy UI code. First, we’ll add the empty class itself in a new source file; and then we’ll add a reference to it in the original form so that the form can use anything inside it:
' In "businesslogic.vb"
Public Class BusinessLogic
End Class
' In "form.vb"
Public Class Form1
Private Logic As BusinessLogic
Public Sub New()
Logic = New BusinessLogic()
End Sub
...
End Class
This doesn’t affect the behavior of the existing code: We’re adding a piece of empty data to it that nothing yet uses or depends on; nothing will even know that it’s there.
Now that we have the new BusinessLogic class, let’s move the business-logic functions into it. This consists of nothing more than cut-and-paste of the functions, and then changing the places where they’re called to now have Logic. in front of them:
' In "businesslogic.vb"
Public Class BusinessLogic
Public Sub Save(FirstName As String, LastName As String, ID As Integer)
USING con As New SqlConnection("server=.;database=MyDatabase;integrated security=true")
Dim Cmd As SqlCommand = New SqlCommand("update PERSONS Set FirstName='" & FirstName & "', lastname = '" & LastName & "' where ID = " & CStr(ID))
con.Open()
cmd.ExecuteNonQuery()
End USING
End Sub
End Class
' In "form.vb"
Public Class Form1
Private Logic As BusinessLogic
Public Sub New()
Logic = New BusinessLogic()
End Sub
Private Sub btnSave_Click(sender As Object, e As EventArgs) Handles btnSave.Click
Dim FirstName As String = editFirstName.Text
Dim LastName As String = editLastName.Text
Dim ID AS integer = Convert.ToInt32(HIDDENIdField.Text)
Logic.Save(FirstName, LastName, ID)
End Sub
End Class
This is the same thing, just with the pieces in a different place. You can run the program and it’ll still work exactly the same. You can repeat this process for every event handler in the form, moving the code out to the new BusinessLogic file, and it doesn’t change anything.
But — !
Look at what we end up with now: All of the business logic for the entire form is now isolated in a separate source file. It’s crappy code, but it’s now all in one place, and there’s no UI directly attached to it.
So let’s turn it into C#.
Switching Languages for Fun and Profit
How do we do that? There are automated tools for VB.NET to C#, but they tend to be a little unreliable. Also, let’s pretend the source language is farther away — maybe it’s an old PHP script instead. We want a technique that covers a lot of ground and can handle pretty wild situations, like “Flash ActionScript becomes modern Rust.”
So rename the file to .cs.
I’m not kidding. Copy the file into your C# project, rename it to BusinessLogic.cs, and then open it in a plain text editor like Notepad++ — don’t use a big fancy IDE, which will cause a lot of trouble. Just use something that’s a good but plain text editor, preferably with with syntax highlighting, so you end up with a C# file that looks like this:
Public Class BusinessLogic
Public Sub Save(FirstName As String, LastName As String, ID As Integer)
USING con As New SqlConnection("server=.;database=MyDatabase;integrated security=true")
Dim Cmd As SqlCommand = New SqlCommand("update PERSONS Set FirstName='" & FirstName & "', lastname = '" & LastName & "' where ID = " & CStr(ID))
con.Open()
cmd.ExecuteNonQuery()
End USING
End Sub
End Class
Now: Let’s pretend for the rest of this exercise that there are a bunch more functions in here that we need to clean up — it’s not just one business-logic function, but a hundred. We don’t want to manually edit anything if we can avoid it — the computer needs to do most of the work, or we’re going to be working on this for weeks.
That code is awful, but we can make it closer to C# with some initial renaming. First, let’s just fix the casing, and some of the symbols, using strict Ctrl+H search-and-replace:
Public—>publicClass—>classUSING—>usingAs—>asDim—>dimEnd—>endSub—>subString—>stringInteger—>int&—>+
That’s a little closer:
public class BusinessLogic
public sub Save(FirstName as string, LastName as string, ID as int)
using con as new SqlConnection("server=.;database=MyDatabase;integrated security=true")
dim Cmd as SqlCommand = new SqlCommand("update PERSONS Set FirstName='" + FirstName + "', lastname = '" + LastName + "' where ID = " + CStr(ID))
con.Open()
cmd.ExecuteNonQuery()
end using
end sub
end class
Let’s fix the declarations. How the heck do we do that?
Regular expressions to the rescue! We’re still going to do search-and-replace, but we’re going to do fancy search-and-replace. Notepad++ is going to make this a lot more C-sharpy with just a few simple transformations. Here’s what we’re going to search for and what we’re going to replace it with:
public class (.*)—>public class \1 {public sub (.*)—>public void \1 {using (.*)—>using (\1) {(\w+) as (\w+)—\2 \1end (using|sub|class)—>}
Just those four changes gives us this result:
public class BusinessLogic {
public void Save(string FirstName, string LastName, int ID) {
using (new as con SqlConnection("server=.;database=MyDatabase;integrated security=true")) {
dim SqlCommand Cmd = new SqlCommand("update PERSONS Set FirstName='" + FirstName + "', lastname = '" + LastName + "' where ID = " + CStr(ID))
con.Open()
cmd.ExecuteNonQuery()
}
}
}
It’s actually starting to look like C#! It’s not exactly right — the using is weird, and we have a dim and a CStr() in there and we’re missing semicolons, but now it’s almost something that’s syntactically valid C# code.
Let’s do a few more automated updates. We’ll add a semicolon to any line that ends in ). We’ll use a more complicated regex to fix any using directives. And CStr() needs to become .ToString(). We can do all of these with regex search-and-replace.
dim—>\)$—>);using \(new as (\w+) (\w+)(\.*)—>using (\2 \1 = new \2\3CStr\(([^)]+\)—>(\1).ToString()
public class BusinessLogic {
public void Save(string FirstName, string LastName, int ID) {
using (SqlConnection con = new SqlConnection("server=.;database=MyDatabase;integrated security=true")) {
SqlCommand Cmd = new SqlCommand("update PERSONS Set FirstName='" + FirstName + "', lastname = '" + LastName + "' where ID = " + (ID).ToString());
con.Open();
cmd.ExecuteNonQuery();
}
}
}
Holy crap, that’s actually valid C#. It’s ugly C#, but it’s valid, and it does the same thing the original VB.NET code did — line-for-line, bug-for-bug, feature-for-feature. In real-world code, you probably need to do more to accommodate For Each and If and so on, but the basic pattern is pretty straightforward overall.
In really complicated code-bases, you might also need a secondary exercise to finish the job. When I converted 50,000 lines of ancient VB.NET to modern C#, after the automated conversions, there was a manual cleanup exercise before it was truly valid C#. That manual cleanup took two or three days, as I recall — but it wasn’t months, or even a week. “A few days” is something you can realistically do on a normal development schedule.
Nicing Up the New Code
Since it’s now valid C#, let’s switch to a proper IDE! Save it in Notepad++, open it in Visual Studio, and then let’s have VS reformat it for us. Ctrl+E, Ctrl+D.
public class BusinessLogic
{
public void Save(string FirstName, string LastName, int ID)
{
using (SqlConnection con = new SqlConnection("server=.;database=MyDatabase;integrated security=true"))
{
SqlCommand Cmd = new SqlCommand("update PERSONS Set FirstName='" + FirstName
+ "', lastname = '" + LastName + "' where ID = " + (ID).ToString());
con.Open();
cmd.ExecuteNonQuery();
}
}
}
Our code standard says we should be using consistent naming. If you have a tool like ReSharper 9 or higher installed, you can do this in bulk; Alt+Enter and “Bulk Rename” to fix our code!
public class BusinessLogic
{
public void Save(string firstName, string lastName, int id)
{
using (SqlConnection con = new SqlConnection("server=.;database=MyDatabase;integrated security=true"))
{
SqlCommand cmd = new SqlCommand("update PERSONS Set FirstName='" + firstName
+ "', lastname = '" + lastName + "' where ID = " + (id).ToString());
con.Open();
cmd.ExecuteNonQuery();
}
}
}
So far, we haven’t done anything manually, and we have C# code that we can actually compile and run. In a more complex code base, you might need to manually massage a few lines, but the obvious goal would be to avoid that wherever possible.
Let’s do a refactor that fixes what are likely a whole lot of duplicate SQL connection declarations; this is probably repeated a lot in this file, inside every business-logic function you see:
using (SqlConnection con = new SqlConnection("server=.;database=MyDatabase;integrated security=true"))
We’re going to highlight the new SqlConnection(...) part and Ctrl+R, Ctrl+M to extract a method named Connect():
using (SqlConnection con = Connect())
...
private SqlConnection Connect()
{
return new SqlConnection("server=.;database=MyDatabase;integrated security=true");
}
Now, everywhere else, let’s use regex search-and-replace to get rid of the duplicate SQL logic. First, cut that return new SqlConnection line to the clipboard, and then apply this, which will turn every new SqlConnection(...) into a call to Connect() instead:
new SqlConnection\([^)]*\)—>Connect()
Paste the clipboard contents back on the same line they came from so that at least one new SqlConnection(...) call still exists.
We’re looking pretty good, but we haven’t addressed the fact that every time this code does new SqlCommand(), it’s supposed to be applying a using directive — the GC finalizer is the only reason the old code actually worked.
This one takes some slightly fancier doing — regex will help here, but we’re going to have mismatched curly braces. We don’t actually know where in the code the SqlCommand instances should be disposed of.
…but we can make a pretty good guess. They’re not valid without a SqlConnection, so wherever that gets disposed of, we can manually add a closing curly brace. This isn’t as nice as the rest of the changes we’ve made, but it’s still pretty straightforward. Even if there are a hundred methods, we can add the using automatically, and only have to add a hundred closing curly braces manually — maybe a five or ten-minute task total. Here’s us adding the using declaration itself —
^(\s*)(SqlCommand[^;]*);\s*$—>\1using (\2) {
public class BusinessLogic
{
public void Save(string firstName, string lastName, int id)
{
using (SqlConnection con = new SqlConnection("server=.;database=MyDatabase;integrated security=true"))
{
using (SqlCommand cmd = new SqlCommand("update PERSONS Set FirstName='" + firstName
+ "', lastname = '" + lastName + "' where ID = " + (id).ToString())) {
con.Open();
cmd.ExecuteNonQuery();
}
}
}
— and then we manually add the } after cmd.ExecuteNonQuery(). Conveniently, Visual Studio will be happy to reindent the code for us as soon as we do:
public class BusinessLogic
{
public void Save(string firstName, string lastName, int id)
{
using (SqlConnection con = new SqlConnection("server=.;database=MyDatabase;integrated security=true"))
{
using (SqlCommand cmd = new SqlCommand("update PERSONS Set FirstName='" + firstName
+ "', lastname = '" + lastName + "' where ID = " + (id).ToString()))
{
con.Open();
cmd.ExecuteNonQuery();
}
}
}
}
Meeting the Goal
We could keep going and clean this up more, but we have code that works now, in the target language. Let’s finish the original goal by putting an ASP.NET REST API in front of the existing business logic. The only thing this needs to do is take in a web request and proxy it to the existing code. We can’t make this part automatically (see below, though), but it’s dirt-simple code and doesn’t do anything more than copy a few names from the existing, ported code and the basic structure from ASP.NET example code:
public class MyAppController : Controller
{
public class SaveViewModel
{
public int Id;
public string FirstName;
public string LastName;
}
private readonly BusinessLogic _businessLogic;
public MyAppController()
{
_businessLogic = new BusinessLogic();
}
public IHttpActionResult SavePerson(SaveViewModel saveInfo)
{
_businessLogic.Save(saveInfo.FirstName, saveInfo.LastName, saveInfo.Id);
return Ok();
}
}
If we had to do this a lot, it might be worthwhile to make a little script in Python or Ruby that reads in the BusinessLogic.cs source file, finds every line with public void DoSomething(...) on it, and spits controller code for it. The script doesn’t need to be great — you’re only going to run it once, and it doesn’t even need to emit perfect output because you can always manually fix it a little later — but if we had to do this for thousands of files, spending ten minutes writing a script that applies some regexes to the method signatures would probably pay off.
Looking Forward
Of course, we haven’t addressed everything here. There’s a nasty SQL injection in the Save() method; we should definitely be using a parameterized query instead! And if we wanted to move from plain SQL to, say, Entity Framework, there’s some considerable effort to pull that off. Our goal was to modernize the code enough to meet a specific need, and we did that. “Switch from raw SQL to EF” is the sort of thing that’s a great followup exercise — once you have customers using the new code and it’s paying for the time to do it.
Importantly, whatever weird business logic we had in the original application was actually able to be copied over wholesale. The new web application interacts with the exact same database in the exact same way — bug-for-bug, feature-for-feature — and we did the conversion in minutes, not months.
Conclusion
Some of the things we did here were true, Fowler-esque refactors, things you could find in the textbook. Some were cleverness involving attacking the code with regexes. What they all shared in common was that they were automatic. We turned ancient crap into relatively nice code in a surprisingly short time, and kept all of the original mechanics.
You don’t have to rewrite code from scratch to bring it forward. You can turn an old PHP 4 file into modern TypeScript. You can turn classic Visual BASIC into Rust. There’s no combination of things that can’t be made to work with a little creativity. Don’t throw out your legacy code — the knowledge embedded in it is incredibly valuable, and if you throw it out, you’re just going to dig a deep hole that you then have to climb out of. Instead, refactor — make the computer do automated things to convert that old code into your new technologies. The baby may be ugly, but you should never, ever, ever throw it out with the bath water.
