“You couldn’t even really tell that there’s a pandemic here,” says Steven Woloshin from his home office in rural New Hampshire. Behind him, each of the three walls is filled with overflowing wooden bookshelves. Woloshin is a professor of medicine at the Dartmouth Institute, an Ivy League research university flanked by the White Mountains of New Hampshire and the Green Mountains of Vermont.
Despite spending lockdown in this idyllic and relatively undisturbed setting, he has invested much of his time in thinking about Covid-19 and the implications of false negative tests. As labs across the world scramble to meet mass testing demands in time for winter, it is more important than ever that both the accuracy of tests and the strategy with which they are rolled out are carefully examined.
Quality control
In recent months, there has been growing concern over inaccurate Covid-19 diagnostic tests reaching the public. The danger of this, explains Woloshin, is twofold. While false positive results, which mistakenly label a person as infected, can lead to unnecessary quarantines and time off work, false negative results create erroneous reassurance, increasing the likelihood that an infected individual will expose others to the virus.
The sensitivity and specificity of new Covid-19 tests are determined by comparing them with those that have already been authorised. The problem, says Woloshin, is that “we don’t have true clinical sensitivity, which would be determined by assessing the performance of the test against a clinical gold standard”. In reality, he adds, many samples can’t be used to detect the virus because they’re taken too early in the infection’s incubation period, or there isn’t yet enough of the virus in the nasopharynx, where the majority of PCR tests are carried out.
Woloshin believes it is vital to start considering exactly what should constitute a gold standard. However, he accepts that cost is a major barrier to progress. “It’s easy for me to say, ‘Oh, we have to develop a gold standard’,” he says, “but that’s a lot of work and a lot of investment.” In addition, he says, the pre-test probability of infection – based on factors like where the individual lives, the local prevalence of disease and whether social distancing is being enforced – must be studied. He is adamant that it is only by combining the two that an accurate picture begins to emerge of the likelihood of infection.
“We really need to develop a way of characterising people at their risk level so we can understand their pre-test probability,” he explains. “Of course, we then need to look at how well the tests work by having a better handle on the clinical sensitivity and specificity, which we really don’t have at the moment. Instead, we have this next best thing of laboratory sensitivity and specificity, which is important, but it doesn’t tell the whole story.”
For Bruce Tromberg, director of the National Institute of Biomedical Imaging and Bioengineering (NIBIB) at the National Institutes of Health (NIH), “no test is perfect – it’s mathematically impossible”. However, like Woloshin, he believes it is vital to understand the population in order to determine the types of error that can be tolerated.
“If you have a vulnerable population that’s very high risk, let’s say people with underlying conditions at an advanced age, the test [you use] should be the most rigorous with the highest sensitivity, because the risk of not getting it right is too high,” he explains. “At the other extreme, let’s say I’m a 21-year-old soldier on a troop ship and there’s definitely presence of the virus on board. I’d want to get tested as fast as possible, like with one of these,” he continues, holding up a portable lateral flow test. “It might not be the most accurate test, but if it’s wrong it’s not that bad because it’s not a super highrisk population.”
Tromberg is currently leading the NIBIB’s Rapid Acceleration of Diagnostics (RadX) innovation programme, providing researchers across the US with $500m of funding to speed up Covid-19 test development. The goal? To make six million daily tests available to the US public by the end of the year. He concedes this is ambitious, describing the “tsunami of tests” needed to reach this point, but is confident that a dramatic upscaling of capacity can be achieved. The programme is developing an array of tests for various user communities, from rapid portable point of care tests that are inexpensive and accessible, to platforms that are able to test for Covid-19 and influenza simultaneously. In order to ensure the new tests can be easily integrated into existing healthcare systems, RadX has developed a stringent evaluation process whereby experienced healthcare professionals assess the most promising tests.
Despite vociferous complaints from leading scientists that the FDA’s Emergency Use Authorisation has led to inaccurate Covid-19 tests flooding the market, Tromberg emphasises that vital lessons have been learned. “We have actually benefitted from the mistakes, or at least the challenges that we’ve seen from other tests,” he says. “It’s given us more insight, and that’s kind of expected when you only have a certain limited number of tests available to meet the vast needs of at least half a dozen – if not more – different user communities.”
On this point, Woloshin agrees. Of course, it’s easy to criticise the regulators, he says, but, ultimately, they have an extremely difficult job. If they are careful they will be accused of moving too slowly, while if they move quickly there is the inevitable risk that inaccurate tests will reach the public. At the start of the pandemic, he continues, the idea of fasttracking testing platforms made sense. Now, though, is the time to start raising the bar and ensuring the tests available are given a more rigorous assessment.
Game plan
Since the outbreak of Covid-19, governments around the world have struggled to put in place comprehensive testing plans to keep up with the rapid spread of the virus. So far, there have been problems with availability, access and lengthy turnaround times. Another issue, adds Woloshin, is developing tests that people find acceptable. “Right now, with a lot of the tests you have to have this thing stabbed up your nose into your brain, which people find very unpleasant,” he explains. “We need to increase access to tests that are accurate but less invasive.”
For Woloshin, a carefully thought-through testing plan will be critical for the safe reopening of society, especially as winter looms, bringing with it the prospect of a deadly resurgence of the virus. As schools and universities start to resume operations in the US, he believes strong leadership is needed to coordinate the various testing strategies and carry out trials to determine which should be implemented. “It’s not just a matter of individual tests, but how to apply the tests, how frequently to carry them out and [how to combine them with] information about people’s exposure histories, temperatures and where they’re living,” he says.
“[These are] things that individually might have very low sensitivity, but in combination may form the basis of a more rational testing strategy.”
Michael Mina, a Harvard epidemiologist at the centre of this debate, recently suggested test sensitivity is secondary to frequency and turnaround time, and that carrying out cheap, daily, selfadministered Covid-19 tests could be as effective as a vaccine for controlling the spread of the virus. “We keep trying to use these diagnostic tools that just tell us what’s going on [with an individual] once every couple of months when they may be tested,” he told the Harvard Gazette in August. “It’s doing nothing to stop the transmission chains.” Instead, he hopes to see a shift in strategy towards rolling out at-home pregnancy-style tests – much like the portable point of care tests being developed by RadX – which are simple to use and give results within minutes.
Contact tracing is also an important component in disrupting the widespread transmission of infection. However, in many countries – the US and UK being two notable examples – rolling out an effective system has been plagued with technical issues. What’s more, there are still some elements of the virus that are not fully understood, making effective tracing difficult. “If the disease behaved differently, if everyone became symptomatic, it would be easier,” explains Woloshin. “But because there’s so much asymptomatic transmission, it really undermines efforts to constrain the virus.”
As for introducing widespread asymptomatic testing, Woloshin reiterates his argument that targeted strategies should be implemented depending on the situation. For places with virtually no overt disease, he says, it’s safe to assume there probably isn’t the need for a lot of testing. On the other hand, in Covid hotspot areas where people are likely to be infectious, it could be wise to carry out universal testing. “The question is the grey zone in the middle; what’s the smart strategy?” he says. “That’s what I’m saying needs to be studied – we need to develop feasible, evidence-based strategies for testing.”
Beyond the test
If you’re searching for a silver lining, the pandemic has brought widespread public attention to the value of medical diagnostics. Debates have been triggered about how tests should be deployed and what could be achieved if different diagnostics were given the same level of attention. On this point, Woloshin is wary, having contributed to research on overdiagnosis in cancer screening. It is crucial, he says, to take a step back and consider the context. Covid-19 is an infectious disease which can have serious or even fatal consequences, especially for vulnerable populations. “I wouldn’t say that because we’re being very aggressive about looking for Covid-19, we should be doing the same [for cancer],” he adds. “The dynamic of the diseases and the harms of over-detection and overtreatment are completely different.”
Contemplating the future, Woloshin admits there are still a lot of unanswered questions about the virus. Until we know more, he says, the prudent strategy is to continue emphasising the importance of social distancing, washing hands and wearing masks. After all, he says, “there are multiple prongs for taking on the virus, testing is just one of the fundamental ones.”
As for Tromberg, the $500m investment is vanishing, but with every dollar spent, RadX gets one step closer to reaching its target of six million daily tests, helping to make new testing strategies a reality. “I’m intrinsically optimistic,” he says, “because I’m representing the thousands of talented people out there who are working incredibly hard on this, and I’m confident that as an engineering and technology community we have the capacity to succeed.”
$500m
RadX programme funding for the development of Covid-19 tests.
NIBIB
Six million
Daily test target for RadX by the end of 2020.
NIBIB
