Detecting COVID-19 infection hotspots in England using large-scale self-reported data from a mobile application: a prospective, observational study
Summary
Background
As many countries seek to slow the spread of COVID-19 without reimposing national restrictions, it has become important to track the disease at a local level to identify areas in need of targeted intervention.
Methods
In this prospective, observational study, we did modelling using longitudinal, self-reported data from users of the COVID Symptom Study app in England between March 24, and Sept 29, 2020. Beginning on April 28, in England, the Department of Health and Social Care allocated RT-PCR tests for COVID-19 to app users who logged themselves as healthy at least once in 9 days and then reported any symptom. We calculated incidence of COVID-19 using the invited swab (RT-PCR) tests reported in the app, and we estimated prevalence using a symptom-based method (using logistic regression) and a method based on both symptoms and swab test results. We used incidence rates to estimate the effective reproduction number, R(t), modelling the system as a Poisson process and using Markov Chain Monte-Carlo. We used three datasets to validate our models: the Office for National Statistics (ONS) Community Infection Survey, the Real-time Assessment of Community Transmission (REACT-1) study, and UK Government testing data. We used geographically granular estimates to highlight regions with rapidly increasing case numbers, or hotspots.
Findings
From March 24 to Sept 29, 2020, a total of 2 873 726 users living in England signed up to use the app, of whom 2 842 732 (98·9%) provided valid age information and daily assessments. These users provided a total of 120 192 306 daily reports of their symptoms, and recorded the results of 169 682 invited swab tests. On a national level, our estimates of incidence and prevalence showed a similar sensitivity to changes to those reported in the ONS and REACT-1 studies. On Sept 28, 2020, we estimated an incidence of 15 841 (95% CI 14 023–17 885) daily cases, a prevalence of 0·53% (0·45–0·60), and R(t) of 1·17 (1·15–1·19) in England. On a geographically granular level, on Sept 28, 2020, we detected 15 (75%) of the 20 regions with highest incidence according to government test data.
Interpretation
Our method could help to detect rapid case increases in regions where government testing provision is lower. Self-reported data from mobile applications can provide an agile resource to inform policy makers during a quickly moving pandemic, serving as a complementary resource to more traditional instruments for disease surveillance.
Funding
Zoe Global, UK Government Department of Health and Social Care, Wellcome Trust, UK Engineering and Physical Sciences Research Council, UK National Institute for Health Research, UK Medical Research Council and British Heart Foundation, Alzheimer’s Society, Chronic Disease Research Foundation.
Introduction
- Flaxman S
- Mishra S
- Gandy A
- et al.
and test-and-trace systems. Despite these efforts, many countries have had increases in infection since re-opening and have often re-imposed either regional
or national lockdowns. Regional lockdowns aim to contain the disease while minimising the severe economic effect of national lockdowns.
- Lavezzo E
- Franchin E
- Ciavarella C
- et al.
Large-scale, population-based testing can indicate regional hotspots, but at the cost of a delay between testing and actionable results. Moreover, accurately identifying changes in the infection rate requires sufficient testing coverage of a given population,
- Peto J
- Alwan NA
- Godfrey KM
- et al.
which can be costly and requires substantial testing capacity. Regional variation in testing access can hamper the ability of public health organisations to detect rapid changes in infection rate. There is a high unmet need for tools and methods that can facilitate the timely and cost-effective identification of infection hotspots to enable policy makers to act with minimal delay.
- Rossman H
- Keshet A
- Shilo S
- et al.
Evidence before this study
To identify instances of the use of digital tools to perform COVID-19 surveillance, we searched PubMed for peer-reviewed articles published between Jan 1, and Oct 14, 2020, using the keywords “COVID-19” AND ([“mobile application”] OR [“web tool”] OR [“digital survey”]). Of the 382 results, we found eight studies that utilised user-reported data to ascertain a user’s COVID-19 status. None of these studies sought to provide disease surveillance on a national level or to compare these predictions with other tools to ascertain their accuracy. Furthermore, none of these papers sought to use their data to highlight geographical areas of concern.
Added value of this study
To our knowledge, we provide the first demonstration of mobile technology to provide national-level disease surveillance. Using over 120 million reports from more than 2·8 million users across England, we estimated incidence, prevalence, and the effective reproduction number. We compared these estimates with those from national community surveys to understand the effectiveness of these digital tools. Furthermore, we showed that the large number of users can be used to provide disease surveillance with high geographical granularity, potentially providing a valuable source of information for policy makers who are seeking to understand the spread of the disease.
Implications of all the available evidence
Our findings suggest that mobile technology can be used to provide real-time data on the national and local state of the pandemic, enabling policy makers to make informed decisions in a quickly moving pandemic.
Methods
Study design and participants
- Drew DA
- Nguyen LH
- Steves CJ
- et al.
which also contains a preliminary demonstration of how symptom data can be used to estimate prevalence.
Procedures
- Pouwels KB
- House T
- Pritchard E
- et al.
is a longitudinal survey of individuals selected to be a representative sample of private households (excluding eg, care homes and student accommodation), which began on April 26, 2020. Individuals in the ONS survey are supervised while they self-administer nose and throat swabs. The results give estimates of prevalence and incidence over time. Data are released weekly, with each release covering 7–14 days before the release date, with the first release on May 10, 2020. The ONS survey swab 150 000 participants per fortnight. The REACT-1 study began on May 1, 2020, and is a cross-sectional community survey, relying on self-administered swab tests from a sample of the population in England.
- Riley S
- Atchison C
- Ashby D
- et al.
,
- Riley S
- Ainslie KEC
- Eales O
- Walters CE
- Wang H
The sample is randomly selected in each round of data collection. Data releases are intermittent and cover periods of several weeks. The UK Government swab test data are made up of two so-called pillars of testing: pillar 1 covers those with clinical need and health-care workers, and pillar 2 testing covers the wider population who meet government guidelines for testing.
Guidance: COVID-19 testing data: methodology note.
We used the ONS and REACT-1 surveys to compare our national estimates of incidence and prevalence, and we used the UK Government testing data to validate our geographically granular list of hotspots.
We describe two methods for estimating prevalence. The first is symptom-based, primarily making use of self-reported symptoms and a predictive, symptom-based model for COVID-19. The second is both symptom-based and swab-based, and seeks to further integrate the information from swab test results collected in the app.
- Menni C
- Valdes AM
- Freidin MB
- et al.
to predict whether a user is SARS-CoV-2 positive on the basis of their reported symptoms (appendix p 2). For a given day, each user’s most recent symptom report from the previous 7 days is used for prediction. If a user reports a positive COVID-19 test in that 7-day period, the test result is used to override the user’s symptom-based estimate. The proportion of positive users is used to estimate prevalence. A user who is predicted to be COVID-19-positive for more than 30 days is considered long-term sick and no longer infectious, and they are then removed from the calculation. We sought to extrapolate these prevalence estimates to the general population. As noted in a previous study,
- Bowyer R
- Varsavsky T
- Sudre CH
- et al.
there is a disparity in COVID-19 prevalence between regions of higher index of multiple deprivation (IMD), a measure of the relative deprivation of geographical regions,
- McLennan D
- Noble S
- Noble M
- Plunkett E
- Wright G
- Gutacker N
and those of lower IMD. We stratified users by Upper Tier Local Authority (UTLA), IMD tertile, and age bands (in decades), and we predicted the percentage prevalence per stratum. We then multiplied our predicted percentage of positive cases per stratum with that stratum’s population size, according to census data, to estimate cases per stratum. These estimates were then summed to produce our population prevalence estimate, which we term PA. We examined the sensitivity of PA to health-seeking bias by removing all users reporting sick at sign-up from the analysis.
These prevalence estimates make use of the swab test results but lack geographical granularity, being per NHS region. We can increase the granularity by taking the symptom-based estimates, which are calculated per UTLA, and rescaling all the estimates that make up an NHS region such that the total prevalence across those UTLAs matches the estimated per-NHS region prevalence. We term this hybrid method, which uses both symptom reports and swab tests, as PH. It is possible to produce granular incidence estimates by applying the model of recovery to these granular prevalence estimates; we term these estimates IH.
- Bettencourt LMA
- Ribeiro RM
Briefly, we used the relationship It + 1=It exp(μ (R(t) – 1), where 1/μ is the serial interval. We modelled the system as a Poisson process and used Markov Chain Monte-Carlo to estimate R(t). In our probabilistic modelling, we assumed that the serial interval was drawn from a gamma distribution with α=6·0 and β=1·5 as in the study by Nishiura and colleagues.
- Nishiura H
- Linton NM
- Akhmetzhanov AR
By sampling successive chains from the system, we obtained a distribution over R(t), which allowed us to report a median and 95% credible intervals. These estimates of uncertainty do not account for the uncertainty in the estimate of incidence, which we found to be mostly systematic and smaller than the other forms of uncertainty modelled.
A hotspot is defined as a sudden increase in the number of cases in a specific geographical region. We produced two rankings of UTLAs in England. The first ranks each by their estimated prevalence, PH. This ranking has the advantage of being preregistered; a list of the top ten UTLAs according to PH has been published online since July 23, 2020. However, it does not allow the direct identification of areas of concern; ie, areas with a large number of new cases, and we therefore also report a second ranking using IH.
We compared our rankings to those obtained by ranking according to government testing data. England contains 149 UTLAs, each containing a mean of 370 000 people. We used the government data to produce daily reference rankings of each UTLA, based on 7-day moving averages of daily cases per UTLA. We included all tests done on a given day to produce the ranking for that day, even if that test took several days to have its result returned, to produce the most accurate gold standard ranking that we could. We used 7-day moving averages of PH and IH to produce our predicted rankings of each UTLA. We then evaluated these predictions against the historical reference using two metrics. The first, recall at 20, is the number of UTLAs in our top 20 that appear in the reference top 20. The second, the normalised mean reciprocal rank at 20, measures the agreement between ranks of our top 20 list. We estimated uncertainty by drawing 100 samples of PH, IH, and the government testing data for each UTLA and day, making use of errors calculated using the Wilson interval approximation for the binomial distribution. These samples were ranked and metrics re-computed to produce 95% CIs for each metric.
Statistical analysis
- Murray B
- Kerfoot E
- Graham MS
- et al.
The COVID Symptom Study app is registered with ClinicalTrials.gov, NCT04331509.
Role of the funding source
Zoe Global developed the app for data collection. The funders had no role in the study design, data collection, data analysis, data interpretation, or writing of the report. All authors had full access to all the data and the corresponding author had final responsibility for the decision to submit for publication.
Results
TableCharacteristics of all app users in England who signed up between March 24, and Sept 29, 2020
Data are mean (SD) or n (%).
- Pouwels KB
- House T
- Robotham JV
- Birrell P
(figure 1A), we included two estimates from the ONS: the official reports, released every week, and the results from time-series modelling. The reports represent the best estimate of the ONS at the time of release, whereas the times-series model can evolve, leading to revision of previous estimates in response to new data (appendix p 7). The government values are consistently lower than other estimates because they are not a representative figure for the population. To account for this, we looked at the number of people who reported classic symptoms (fever, loss of smell, and persistent cough) for the first time between July 7 and Aug 5, 2020, and who did not get tested; we found the number to be 50 499 (62%). We used this percentage to scale the government data by a factor of 2·5, our best estimate of the systematic undercounting of new cases.
Figure 1Daily incidence in the UK since May 12, 2020, compared with daily laboratory-confirmed cases and the ONS study (A) and daily prevalence in the UK compared with the ONS and REACT-1 studies (B)
ONS data are taken from the report released on Oct 9, 2020, and ONS report dates are taken as the midpoint for the date range covered by the estimate. For the Symptom Study and REACT-1, the shaded areas represent 95% CIs. For the ONS, the shaded areas and error bars represent 95% credible intervals. ONS=Office for National Statistics. REACT-1=Real-time Assessment of Community Transmission.
Taking this factor into account, PA is slightly higher than ONS and REACT-1 data. On Sept 28, 2020, we estimated a prevalence of 0·53% (95% CI 0·45–0·60).
Figure 2Empirical probability density function of days to recovery with a gamma fit (A) and empirical cumulative density function of days to recovery with the same gamma fit (B)
The bars represent empirical findings and the red line is the gamma fit.
The R number and growth rate in the UK.
Estimates for each of the NHS regions are in the appendix (p 12). The estimates agree that R(t) has been above 1 from early-mid-September, and we estimate that R(t) in England was 1·17 (95% CI 1·15–1·19) on Sept 28, 2020. The government estimates are much smoother than our estimates, which is probably because they are derived from a consensus of the R(t) estimates from the models produced by many groups.
Figure 3Estimated R(t) for England between June 24 and Sept 28, 2020
The shaded area for Symptom Study data represents 95% credible intervals and for government data represents 95% CIs. UK Government estimates published every 7–12 days from June 12, 2020. R(t)=effective reproduction number.
Figure 4Performance of our two ranking methods: ranking by prevalence and incidence on two metrics, recall at 20 (A) and the normalised mean reciprocal rank at 20 (B)
Figure 5Agreement between COVID Symptom Study and UK Government case numbers per week and UTLA, against the number of government pillar 2 tests carried out
Discussion
- Allen WE
- Altae-Tran H
- Briggs J
- et al.
Corona Israel,
- Rossman H
- Keshet A
- Shilo S
- et al.
the Facebook Survey, and CovidNearYou. However, to our knowledge we are the first to provide national-level disease surveillance, and we have found good agreement with traditional, representative community surveys.
COVID-19 testing in the UK.
Our results indicate that our case estimates agree best with government estimates in areas with high levels of testing per capita, suggesting that our estimates could prove a valuable resource for forecasting in regions with poor testing provision. There are other reasons why our list might differ from the Government list. First, the two methods might have different uptake in some higher-risk groups, such as students in provided accommodation, thus showing different sensitivity to hotspots based on the demographic make-up of a region. The two methods might therefore be complementary, and we suggest that our hotspot detection could be most beneficial as an additional indication of regions where increased testing might be best focused. The modest reliance on PCR tests suggests our approach could prove valuable in countries where testing infrastructure is less developed, although further work is required to assess our approach in other locations.
- Pouwels KB
- House T
- Robotham JV
- Birrell P
and the REACT-1 study.
- Riley S
- Atchison C
- Ashby D
- et al.
,
- Riley S
- Ainslie KEC
- Eales O
- Walters CE
- Wang H
These studies have the advantage of being more representative of the population, and their design enables the detection of asymptomatic cases. However, they are smaller than the COVID Symptom Study; the ONS and REACT-1 studies currently report 120 000–175 000 participants in England, whereas the app reports over 2 800 000 users in England. The ability to use self-reported symptom data from this large cohort enables us to make predictions of more geographically granular regions than do either the ONS or REACT-1 studies, allowing us to predict COVID-19 hotspots at the UTLA level. Our estimates should therefore be viewed as independent and complementary to those provided by the ONS and REACT-1 studies.
- Bowyer R
- Varsavsky T
- Sudre CH
- et al.
and we have few users reporting from key sites such as care homes and hospitals. We accounted for some population differences when producing prevalence estimates using specific census adjusted population strata, but the number of invited tests does not allow us to do this when calculating incidence. Differences in reported symptoms across age groups
- Zazzara MB
- Penfold RS
- Roberts AL
- et al.
would probably lead to different prediction models of COVID-19 positivity, and the performance of the model will vary with the prevalence of other infections with symptoms that overlap with COVID-19, such as influenza. Furthermore, the app population is less ethnically diverse than the general population.
- Lo C-H
- Nguyen LH
- Drew DA
- et al.
Reliance on user self-reporting can also introduce bias into our results; for instance, users who are very sick might be less likely to report than those with mild symptoms. Other sources of error include collider bias
- Griffith G
- Morris TT
- Tudball M
- et al.
arising from a user’s probability of using the app being dependent on their likelihood of having COVID-19, potentially biasing our estimates of incidence and prevalence. We showed a sensitivity analysis that attempts to understand the effect of health-seeking behaviour, but we acknowledge that there are many other biases that might affect our results—for example, our users might be more risk-averse than the general population—and that our results must be interpreted with this in mind.
We have presented a means of combining app-based symptom reports and targeted testing from over 2·8 million users to estimate incidence, prevalence, and R(t) in England. By integrating symptom reports with PCR test results, we were able to highlight regions which might have concerning increases in COVID-19 cases. This approach could be an effective, complementary way for governments to monitor the spread of COVID-19 and identify potential areas of concern.
Contributors
TV, MSG, TF, MFG, PWF, JW, CJS, TDS, and SO contributed to the study concept and design. SG, JCP, CHS, DAD, LHN, ATC, RD, JW, CJS, TDS, and SO contributed to the acquisition of data. TV, MSG, LSC, SG, JCP, CHS, and BM contributed to data analysis and have verified the underlying data. TV, MSG, and LSC contributed to the initial drafting of the manuscript. ATC, CJS, TDS, and SO contributed to study supervision. All authors contributed to the interpretation of data and critical revision of the manuscript.
Declaration of interests
SG, JCP, RD, and JW are employees of Zoe Global. DAD and ATC previously served as investigators on a clinical trial of diet and lifestyle using a separate smartphone application that was supported by Zoe Global. TF reports grants from the European Research Council, Swedish Research Council, Swedish FORTE Research Council, and the Swedish Heart–Lung Foundation, outside the submitted work. MFG reports financial and in-kind support within the Innovative Medicines Initiative project BEAr-DKD from Bayer, Novo Nordisk, Astellas, Sanofi-Aventis, AbbVie, Eli Lilly, JDRF International, and Boehringer Ingelheim; personal consultancy fees from Lilly; financial and in-kind support within a project funded by the Swedish Foundation for Strategic Research on precision medicine in diabetes from Novo-Nordisk, Pfizer, Follicum, Abcentra; in-kind support on that project from Probi and Johnson and Johnson; and a grant from the EU, outside the submitted work. ATC reports grants from Massachusetts Consortium on Pathogen Readiness, during the conduct of the study, and personal fees from Pfizer and Boehringer Ingelheim and grants and personal fees from Bayer, outside the submitted work. TDS is a consultant to Zoe Global. All other authors declare no competing interests.
Data sharing
Acknowledgments
Zoe Global provided in-kind support for all aspects of building, running, and supporting the app, and provided service to all users worldwide. Support for this study was provided by the National Institute for Health Research (NIHR)-funded Biomedical Research Centre (BRC) based at Guys’ and St Thomas’ NHS Foundation Trust. Investigators also received support from the Wellcome Trust, the UK Medical Research Council and British Heart Foundation, Alzheimer’s Society, the EU, the UK NIHR, Chronic Disease Research Foundation, and the NIHR-funded BioResource, Clinical Research Facility and BRC based at Guys’ and St Thomas’ NHS Foundation Trust, in partnership with King’s College London, the UK Research and Innovation London Medical Imaging and Artificial Intelligence Centre for Value Based Healthcare, the Wellcome Flagship Programme (WT213038/Z/18/Z), the Chronic Disease Research Foundation, and the DHSC. CMA is supported by the National Institute of Diabetes and Digestive and Kidney Diseases (K23 DK120899), as is DAD (K01DK120742). DAD and LHN are supported by the American Gastroenterological Association–Takeda COVID-19 Rapid Response Research Award (AGA2021-5102). The Massachusetts Consortium on Pathogen Readiness and Mark and Lisa Schwartz supported DAD, LHN, and ATC. LHN is supported by the American Gastroenterological Association Research Scholars Award and the National Institute of Diabetes and Digestive and Kidney Diseases (K23 DK125838). TF holds a European Research Council Starting Grant. ATC was supported in this work through a Stuart and Suzanne Steele MGH Research Scholar Award. Investigators from the COVID Symptom Study Sweden were funded in part by grants from the Swedish Research Council, Swedish Heart–Lung Foundation, and the Swedish Foundation for Strategic Research (LUDC-IRC 15-0067). We thank Catherine Burrows (Bulb, UK) for assistance with database querying.
Supplementary Material
References
- 1.
Estimating the effects of non-pharmaceutical interventions on COVID-19 in Europe.
Nature. 2020; 584: 257-261
- 2.
COVID-19: how does local lockdown work, and is it effective?.
BMJ. 2020; 370m2679
- 3.
Suppression of a SARS-CoV-2 outbreak in the Italian municipality of Vo’.
Nature. 2020; 584: 425-429
- 4.
Universal weekly testing as the UK COVID-19 lockdown exit strategy.
Lancet. 2020; 395: 1420-1421
- 5.
A framework for identifying regional outbreak and spread of COVID-19 from one-minute population-wide surveys.
Nat Med. 2020; 26: 634-638
- 6.
Rapid implementation of mobile technology for real-time epidemiology of COVID-19.
Science. 2020; 368: 1362-1367
- 7.
Community prevalence of SARS-CoV-2 in England during April to September 2020: results from the ONS Coronavirus Infection Survey.
Lancet. 2020;
- 8.
REal-time Assessment of Community Transmission (REACT) of SARS-CoV-2 virus: study protocol.
Wellcome Open Research. 2020; 5: 200
- 9.
High prevalence of SARS-CoV-2 swab positivity in England during September 2020: interim report of round 5 of REACT-1 study.
medRxiv. 2020; ()
- 10.
Guidance: COVID-19 testing data: methodology note.
- 11.
Real-time tracking of self-reported symptoms to predict potential COVID-19.
Nat Med. 2020; 26: 1037-1040
- 12.
Geo-social gradients in predicted COVID-19 prevalence and severity in Great Britain: results from 2 266 235 users of the COVID-19 Symptoms Tracker app.
medRxiv. 2020; ()
- 13.
The English Indices of Deprivation 2019: technical report.
- 14.
Real time Bayesian estimation of the epidemic potential of emerging infectious diseases.
PLoS One. 2008; 3e2185
- 15.
Serial interval of novel coronavirus (COVID-19) infections.
Int J Infect Dis. 2020; 93: 284-286
- 16.
Accessible data curation and analytics for international-scale citizen science datasets.
ArXiv. 2020; ()
- 17.
Community prevalence of SARS-CoV-2 in England: results from the ONS coronavirus infection survey pilot.
medRxiv. 2020; ()
- 18.
Prevalence of asymptomatic SARS-CoV-2 infection: a narrative review.
Ann Intern Med. 2020; 173: 362-367
- 19.
The R number and growth rate in the UK.
- 20.
Population-scale longitudinal mapping of COVID-19 symptoms, behaviour and testing.
Nat Hum Behav. 2020; 4: 972-982
- 21.
COVID-19 testing in the UK.
Lancet Respir Med. 2020; 81061
- 22.
Probable delirium is a presenting symptom of COVID-19 in frail, older adults: a cohort study of 322 hospitalised and 535 community-based older adults.
Age Ageing. 2020; ()
- 23.
Racial and ethnic determinants of COVID-19 risk.
MedRxiv. 2020; ()
- 24.
Collider bias undermines our understanding of COVID-19 disease risk and severity.
Nature Commun. 2020; 115749
Article Info
Publication History
Published: December 03, 2020
Identification
Copyright
© 2020 The Author(s). Published by Elsevier Ltd.