True positives10 May 2021
When they work, Covid-19 tests are good for one yes or no question. If only deciding how to use them were so simple. Repeatedly, mass testing programmes using lateral flow tests have foundered on the risks of false negatives, but the more reliable PCR tests remain too costly to implement on a wide enough scale. Abi Millar speaks to Niraj Jha, co-founder of AI company NeuTigers, Timothy Plante from the University of Vermont and Melanie Ott, director of the Gladstone Institute of Virology, about using machine learning, wearables and even CRISPR to find a middle path out of the pandemic.
As the Covid-19 vaccination roll-out continues, many countries remain stuck in what feels like indefinite lockdown. Restrictions are poised to continue in most places until some critical threshold of immunisation has been reached – with bars, restaurants and even workplaces stubbornly out of bounds and population-wide curfews in place.
Austria, however, has hit on a different solution. Rather than waiting for the EU’s centralised vaccine programme to provide enough doses to inoculate its population, the small European nation, which suffered particularly badly in the second wave, is banking on extensive Covid-19 testing as the best way to reopen public life. It has said it will make 3.5 million free Covid tests available to its 8.8 million citizens each week – freeing them to get their hair cut or visit a ski resort, on the proviso of a recent negative test.
“We are on the way to becoming the testing world champion,” announced the Austrian Chancellor Sebastian Kurz in late February. “Our goal is to be able to control the incidence of infection, or at least mitigate any growth in infection numbers, as best we can, by testing as much as possible.”
Austria isn’t the first country to think along these lines. In June 2020, China completed a mass testing programme in Wuhan, the centre of the original Covid outbreak and the city subjected to the world’s most severe lockdown, to locate asymptomatic carriers and prevent a resurgence of the virus. Almost 11 million people were tested, with the very few positive cases isolated and their contacts traced. Cases and deaths in the city have remained largely static in the months since.
Despite the costs and complexity of the Wuhan approach, it has proved compelling. In October 2020, Slovakia followed suit, testing 3.6 million people (97% of the eligible population) over the course of a single weekend. Those testing positive (around 38,000 people) were asked to quarantine along with their households, while the rest retained their freedom of movement and were able to go to work.
Smaller-scale testing programmes are now being rolled out elsewhere. The UK government is relying on mass testing as a way to keep children in face-to-face education. Since England’s schools opened on 8 March, schoolchildren have been getting tested twice a week. This mirrors similar programmes in Cyprus, which has also rolled out testing in workplaces, and Germany, which now entitles each person to a weekly test.
Supposedly, testing everyone at a school or workplace should catch any pockets of infection and stop outbreaks in their tracks, as well as enabling everyone whose results come back negative to go about their lives almost normally, reducing the disastrous social and economic consequences of prolonged lockdowns. Still, mass testing programmes – particularly when they have to screen for asymptomatic Covid – remain an article of some controversy. Questions have been raised about the reliability of the types of tests used across large populations, which come with a significant risk of false positives and false negatives. These errors could either force households to self-isolate, potentially losing income in the absence of any real infection, or embolden people at risk of spreading Covid to act as if they didn’t have it.
The UK’s school-testing programme relies on controversial lateral flow tests, designed by Innova, which are cheap and can return results in 30 minutes. These tests have only moderate sensitivity and aren’t very good at picking up asymptomatic infection. When the tests were trialled in a community pilot study in Liverpool, 60% of infected symptomless people went undetected, including a third of those with high viral loads who were at risk of infecting others. Doubts have been raised at how effective they will be at suppressing infection in schools.
Slovakia also used lateral flow tests, which were designed by Roche and SD Biosensor. The initiative is thought to have cut infection rates by around 60% in the absence of a strict lockdown. Unfortunately, this was not enough to prevent a devastating resurgence of the virus, with the country reporting one of the highest per capita death rates in the world by February.
It is clear that if mass testing programmes are ever going to fulfil their purpose, two conditions need to be satisfied. First, testing cannot be the only thing a country is doing to suppress the virus. Second, the tests need to reach at least some minimum threshold of accuracy while remaining affordable – which may mean thinking beyond lateral flow and polymerase chain reaction (PCR) tests.
“PCR tests cost $100 in the US,” points out Niraj Jha, co-founder of the AI company NeuTigers. “If we go with PCR and there are eight billion of us on this planet and we test every day, that would cost $800bn every day so that the economy can open up. It isn’t feasible.”
With their sensitivity and specificity exceeding 95%, PCR tests are regarded as the ‘gold standard’ for detecting Covid-19 infections. But as Jha stresses, while they’re suitable for testing those with symptoms, they require specialist lab equipment, and are therefore too slow and costly to be deployed in a mass-screening approach.
Apps in the deep
Jha’s hope is that his company’s new rapid screening app, CovidDeep, could provide an alternative. The app, which uses biometric data, could be used to triage those who need a PCR test. NeuTigers says it is more than 90% accurate in predicting whether a person is virus positive or virus negative – nearly twice as effective as current triage tools like temperature checks and questionnaires.
In conjunction with wearable devices, the app collects data like blood pressure, blood oxygen levels and skin temperature, and checks these against the telltale markers of Covid-19. During the app’s development process, the company collected hundreds of thousands of data points from research participants. It then used neural networks to learn how the virus affected these metrics. The algorithm can spot physiological patterns of Covid even in those who don’t perceive themselves as having symptoms.
“To manage the pandemic while going to work and leading a normal life, we need to get tested every day, but with current technologies that isn’t possible,” says Jha. “CovidDeep requires only two minutes of data collection and is accurate. It opens up the possibility we can tackle the pandemic and open up the economy at the same time.”
CovidDeep might be a useful solution for those who want to test themselves regularly in a non-invasive way, but users would need access to an EmpaticaE4 wristband, as well as an off-the-shelf device that can measure blood pressure and blood oxygen readings. It’s unlikely that many individuals would be willing to shoulder this cost, so the app may be better suited to venues that need to test multiple people as a condition of entry. NeuTigers says it is adapting the app to work with connected consumer health products from Fitbit, Apple and others.
Asymptomatic Covid infections missed by Innova’s lateral flow tests in a community mass testing pilot in Liverpool last December.
Number of Covid-19 tests Austria is making available to its citizens every week.
CovidDeep is not the only AI-based tool in development. AI-COVID, designed by Biocogniv, could be used as a triage tool in hospitals. It uses physiological data taken from Complete Blood Count and Complete Metabolic Panels – common, cheap lab tests ordered by emergency departments in the US – and assesses markers of infection like blood clotting and inflammation. The tool generates results almost instantly, and can be run on top of the blood tests for minimal added cost.
“We took everyone who’d had a PCR test for Covid... and we told the computer to find patterns. It came out with this tool that was almost too good to believe. The performance is far above and beyond what we’re used to in epidemiology.”
With a very high sensitivity of 95%, along with moderate specificity of 49%, AI-COVID can identify Covid-negative patients with high accuracy, allowing hospitals to allocate PCR tests to the patients most likely to need one. It was tested in collaboration with researchers at the University of Vermont and Cedars- Sinai Medical Centre in California, who are excited by its potential.
“We took everyone who’d had a PCR test for Covid, regardless if it was positive or negative, and had also had all these routine blood tests, and we told the computer to find patterns,” says Dr Timothy Plante, lead study author and assistant professor at the University of Vermont. “It came out with this tool that was almost too good to believe. The performance of this tool is far above and beyond what we’re used to in epidemiology.”
He is excited about the prospects of AI-COVID, not just as a predictive tool for Covid-19, but as an example of the kind of technology that may soon be ubiquitous in medicine. “The complexity that computers can assess is far and beyond the complexity that we can see as humans,” he says. “Computers can assess really subtle patterns that correspond with certain outcomes. So, that’s to say that AI has yet to have its heyday in medicine. It’s in the future, and it’ll help with complex decision making down the road.”
Another technology being hyped for Covid-19 testing is CRISPR, the gene-editing tool. Several companies are developing CRISPR-based tests that can be used to detect viral RNA in a sample.
Dr Melanie Ott, director of the Gladstone Institute of Virology, is developing one such test together with colleagues at the University of California, Berkeley. In proof-of-concept studies, it has been able to return results within five minutes using a simple smartphone based device. While not currently as sensitive as PCR, it is suitable for use at the point of care, and is even able to quantify viral load.
“Part of breaking the pandemic is going to be identifying and isolating people who are infectious very early, so that they cannot spread the virus,” says Ott. “It’s in that rapid-testing role that we see our biggest advantage. That’s also where I think knowing whether your viral load is going up or down could be very important, because you’ll know whether you’re going into an infection or at the tail end of an infection.”
Although the device is not yet at the point of commercialisation, Ott hopes it could one day be used in doctors’ surgeries, small businesses, pharmacies, airports and students’ dormitories.
“When it comes to clinical diagnostics, where you want to pick up the last bit of RNA that you can find in a nasal sample, I think PCR will potentially stay the standard,” says Ott. “But when it comes to rapid, inexpensive, deployable testing – and also when you want to quantify the virus – I think CRISPR could replace the PCR. It’s a very good supplement.”
Clearly, none of these technologies amounts to a magic bullet for ridding the world of Covid-19. However, over the months ahead we may start to break the stranglehold of PCR and lateral flow tests, with a variety of different triage and testing tools to choose from. Factor in the roll-out of vaccine programmes, and it’s easy to imagine a future in which these technologies play a role in helping us all stay much safer. Time – and the experiments in Austria and elsewhere – will tell.