COVID-19 response systems for Maryland and Delaware

Context

From the spring of 2020, the State of Maryland needed several new systems at once. Its Department of Health had to reach every contact of every positive case, book residents into vaccination sites that did not exist yet, and run testing in half its counties. It had to produce letters that people could show their employers. And the state had to keep its website online while the Governor spoke on live television.

The state had Salesforce, a health department under very heavy load, and deadlines set by the pandemic.

Delaware came later. In the middle of the pandemic, the original prime contractor for the Delaware Division of Public Health went bankrupt. The state gave us an emergency contract directly, and contact tracing continued without interruption.

What we built

We delivered eight areas of work over three years. One decision applied to all of them: reach residents by text message, instead of asking them to fill in a web form. The people who most needed an appointment were the least likely to complete a form: older residents, people without broadband, and people who will not download an app for a one-time task.

A resident sends a text, gets a choice of sites and times, picks one, and receives a confirmation. There is no account, app or browser. The same text channel then carried contact tracing conversations, weather closures, site changes, cancellations and test results. All of it connected to the state's Salesforce, so the health department kept working in the system it already knew.

Maryland and Delaware ran as separate systems, with separate data, credentials and message queues. Heavy use in one state could not slow down the other.

The eight areas of work

The work covered eight areas.

Contact tracing

A text message and email system that guided people through contact tracing and quarantine. It connected to the state's Salesforce to send automated messages and letters. About 1.5 million cases processed. Later extended to Delaware under the emergency award.

MassVax: mass vaccination scheduling

An appointment system for the state's mass vaccination sites, which residents could use entirely by text. It sent notices about weather delays and site changes, let residents change or cancel a booking, and tracked each appointment in Salesforce.

Community testing

A portal and appointment tracking for testing sites in half of Maryland's counties.

The Governor's website

We moved maryland.gov from Windows to Linux, secured it to the Center for Internet Security (CIS) Level 1 standard, and tuned it to handle 150,000 requests a minute. It took four working days, and it was ready before the Governor's press conference.

Work excuse and release letters

Millions of letters for patients to give their employers. Employers could check that a letter was genuine without calling the health department.

covidLINK

We set up and maintained covidLINK.maryland.gov, the state's public COVID-19 information site.

Infrastructure and disaster recovery

Amazon Web Services (AWS) infrastructure configured for HIPAA data and secured to CIS Level 1, with central security logging and single sign-on. Load balancing and automatic scaling handled traffic spikes, and we tested encrypted database restores on a schedule.

Advising the state

We found problems together with the state's team, designed the fixes, and coordinated with the other contractors, including Ernst & Young, Enovational and NORC. We also gave the state's Department of Information Technology direct help with text-message compliance in Qualtrics and with Twilio.

Why it was hard

The scheduling logic was the easy part. The hard part was protecting health data under HIPAA, on infrastructure that had to pass a state security review, while we built at emergency speed.

Load was the other problem. Demand came in spikes after eligibility announcements and press conferences. In the same week, we rebuilt maryland.gov to handle 150,000 requests a minute, on Linux and to CIS Level 1, in four working days.

A failure here meant a person could not book a vaccine appointment.

Results

About 1.5 million contact tracing cases processed. More than a million vaccination appointments scheduled, with up to about ten thousand a day at peak. 4.2 million vaccinations processed. Testing in half of Maryland's counties.

Across both states, about 45 million text messages sent and received. Millions of work excuse and release letters, which employers could verify. And a state website that handled 150,000 requests a minute.

The infrastructure was System and Organization Controls 2 (SOC 2) compliant, configured for HIPAA data, and secured to CIS Level 1, with central security logging and single sign-on.

What we learned

Reach people on the channel they already use. Every hour we spent on the text message path did more good than the same hour on the web form. Many residents were never going to come to a website, so we went to them.

In an emergency, skipping compliance work is what slows you down. We passed the state security review because we built the infrastructure for it from the first week. A rebuild on that timeline would not have been possible.