Abstract

Objective

To evaluate the predictive power of different malaria case time-series models in the state of Amapá, Brazil, for the period 1997-2016.

Methods

This is an ecological time series study with malaria cases recorded in the state of Amapá. Ten deterministic or stochastic statistical models were used for simulation and testing in 3, 6, and 12 month forecast horizons.

Results

The initial test showed that the series is stationary. Deterministic models performed better than stochastic models. The ARIMA model showed absolute errors of less than 2% on the logarithmic scale and relative errors 3.4-5.8 times less than the null model. It was possible to predict future malaria cases 6 and 12 months in advance.

Conclusion

The ARIMA model is recommended for predicting future scenarios and for earlier planning in state health services in the Amazon Region.

Time Series Studies; Malaria; Decision Support Techniques; Epidemiological Monitoring; Forecasting

Introduction

Malaria is a preventable and curable disease. However, in 2018, around 228 million cases were registered worldwide.¹1. World Health Organization - WHO. World malaria report 2019 [Internet]. Genebra: World Health Organization; 2019 [cited 2020 Sep 28]. 232 p. Available from: https://www.who.int/publications/i/item/world-malaria-report-2019
https://www.who.int/publications/i/item/... Timely prevention of malaria cases allows for the allocation of adequate resources for controlling the disease and planning its elimination.³3. Gomez-Elipe A, Otero A, Van Herp M, Aguirre-Jaime A. Forecasting malaria incidence based on monthly case reports and environmental factors in Karuzi, Burundi, 1997-2003. Malar J [Internet]. 2007 Sep [cited 2020 Sep 28];6(129):1-10. Available from: https://doi.org/10.1186/1475-2875-6-129 https://doi.org/10.1186/1475-2875-6-129
https://doi.org/10.1186/1475-2875-6-129... The initiative called ‘The Malaria Eradication Research Agenda’ analyzed different important aspects for the global elimination of malaria. Predictive modeling of cases was suggested as a tool for helping the Health Surveillance sector to plan infection control actions.⁴4. The malERA Consultative Group on Modeling. A research agenda for malaria eradication: modeling. PLoS Med [Internet]. 2011 Jan [cited 2020 Sep 28];8(1). Available from: https://doi.org/10.1371/journal.pmed.1000403
https://doi.org/10.1371/journal.pmed.100...

Monthly time series of new cases can be statistically adjusted in mathematical functions, using software.⁶6. Hyndman RJ, Khandakar Y. Automatic time series forecasting: the forecast package for R. J Stat Softw [Internet]. 2008 Jul [cited 2020 Sep 28];27(3):C3. Available from: http://dx.doi.org/10.18637/jss.v027.i03
http://dx.doi.org/10.18637/jss.v027.i03... Time series can be broken down according to three basic components: (i) the seasonal component, representing the cyclic pattern of the disease over time; (ii) the linear component, understood as an increasing or decreasing linear trend in the disease over time; and (iii) the stochastic component, regarding intervening factors affecting time series without a specific pattern.

Statistical time series models may be used for predicting future cases. Autoregressive integrated moving average (ARIMA) is a pioneer method for describing and predicting time series.⁹9. Dokumentov A, Hyndman RJ. STR: a seasonal-trend decomposition procedure based on regression. Monash Econometrics and Business Statistics Working Papers [Internet]. 2015 [cited 2020 Sep 28]. Available from: https://robjhyndman.com/publications/str/
https://robjhyndman.com/publications/str... The exponential smoothing model (ETS) represents an alternative to ARIMA.¹⁰10. Cleveland R, Cleveland W, McRae J, Terpenning I. STL: a seasonal-trend decomposition procedure based on Loess (with discussion). J Off Stat [Internet]. 1990;6(1):3-73. Available from: https://www.wessa.net/download/stl.pdf
https://www.wessa.net/download/stl.pdf... Exponential smoothing models for adjustment of complex seasonal patterns (TBATS and BATS) are methods deemed more efficient than ARIMA.¹¹11. Durbin J, Koopman SJ. Time series analysis by state space methods. Oxford University Press; 2013. Another alternative can be the model dividing the seasonal component into subcomponents (STLM).¹²12. Crone SF, Kourentzes N. Feature selection for time series prediction - A combined filter and wrapper approach for neural networks. Neurocomputing [Internet]. 2010 Jun [cited 2020 Sep 28];73(10-12):1923-36. Available from: https://doi.org/10.1016/j.neucom.2010.01.017
https://doi.org/10.1016/j.neucom.2010.01...

The models described above may be deterministic and have statistical structure to break down and adjust seasonal and linear components in time series, but they cannot estimate the stochastic component. Therefore, machine learning computational approaches have been proposed to quantify the effect of the third component: the structural model (StructTS), the neural network model (NNETAR), and machine learning models (ELM, MLP) are examples of such approaches.⁹9. Dokumentov A, Hyndman RJ. STR: a seasonal-trend decomposition procedure based on regression. Monash Econometrics and Business Statistics Working Papers [Internet]. 2015 [cited 2020 Sep 28]. Available from: https://robjhyndman.com/publications/str/
https://robjhyndman.com/publications/str... Models may be compared with a null model, defined by the constant value of the last observation.¹⁷17. Helfenstein U. Box-jenkins modelling of some viral infectious diseases. Stat Med [Internet]. 1986 Jan-Feb [cited 2020 Sep 28];5(1):37-47. Available from: https://doi.org/10.1002/sim.4780050107
https://doi.org/10.1002/sim.4780050107...

The premise of this article is that health surveillance can use predictive models of time series to predict the impact of malaria on a given Brazilian state. Malaria case incidence in the state of Amapá between 2015 and 2018 was, on average, 17 cases per 1,000 inhabitants, one of the highest rates in the country in comparison with the rates found in the states of Acre (37/1,000 inhabitants), Amazonas (17/1,000 inhabitants) and Roraima (19/1,000 inhabitants) in the same period. It is important to identify future time series of malaria cases for planning control measures. This study aims to evaluate the predictive power of different malaria case time-series models in the state of Amapá.

Methods

A statistical and computational approach was applied to the Amapá state health service, complementary to malaria control activities in Brazil as a whole, in accordance with the terms set by the World Health Organization (WHO) for eliminating this disease.²2. Organização Mundial da Saúde - OMS. Estratégia técnica mundial para o paludismo 2016-2030 [Internet]. Geneva: Organização Mundial da Saúde; 2015 [citado 2020 set 28]. 30 p. Disponível em: https://apps.who.int/iris/bitstream/handle/10665/176712/9789248564994_por.pdf?sequence=6
https://apps.who.int/iris/bitstream/hand...

This was an ecological time series study, using the number of malaria cases recorded in the state of Amapá in the period from 1997 to 2016.

Amapá is one of Brazil’s most important endemic malaria regions (Figure 1). Its predominant climate, according to Köppen-Geiger's climatic classification, is tropical monsoon, that is, hot and very humid, with average rainfall indexes of 3,300mm annually. The largest portion of the state (73%; 97,000km²) is covered by native vegetation. In 2019, Amapá's population totaled 830,000 inhabitants, distributed over 16 municipalities.

Malaria data (autochthonous cases, identified through microscopy slides testing positive for Plasmodium – using the thick smear technique) were obtained from the following Health Ministry Health Surveillance Secretariat information systems: National Malaria Control Program Information System (SISMAL) for the period 1997-2003; and the Malaria Epidemiological Surveillance Information System (SIVEP-Malaria) for the period 2003-2016.

Three quantitative discrete variables were used with natural logarithm transformation:¹⁸18. Briët OJT, Vounatsou P, Gunawardena DM, Galappaththy GNL, Amerasinghe PH. Models for short term malaria prediction in Sri Lanka. Malar J [Internet]. 2008 May [cited 2020 Sep 28];7(76). Available from: https://github.com/MVMLima/artigo_RESS_malaria
https://github.com/MVMLima/artigo_RESS_m...

1. number of monthly cases of malaria, from January 1997 to December 2015 (this variable was used to adjust the parameters of the statistical models);

2. Estimated values, from January to December 2016 (predictor variable, used to test the predictive power of the statistical models); and

3. number of monthly cases of malaria, from January to December 2016 (test variable, compared with predictor variable).

The statistical approach used was based on time series models. The first procedure involved the Dickey-Fuller test, increased to a 5% significance level, to test whether the time series was stationary or not. The stationary premise was assumed using statistical models, described later in this text.

The time series encompassed 240 months, from January 1997 to December 2016, divided into two periods: training period, from January 1997 to December 2015; and test period, from January to December 2016. The number of monthly cases of malaria in the training period was used to adjust each one of the statistical models and estimate time component parameters (seasonality; linear trend; stochastic effect). The number of monthly cases of malaria in the test period was used for comparison with the values estimated by the statistical models. The test was conducted with three prediction time horizons: 12 months in advance (January to December 2016); six months in advance (July to December 2016); and three months in advance (October to December 2016). The result of each test was used to assess the predictive power of the models.

The statistical models used were:

1. deterministic models (ARIMA;⁹9. Dokumentov A, Hyndman RJ. STR: a seasonal-trend decomposition procedure based on regression. Monash Econometrics and Business Statistics Working Papers [Internet]. 2015 [cited 2020 Sep 28]. Available from: https://robjhyndman.com/publications/str/
   https://robjhyndman.com/publications/str... ETS;¹⁰10. Cleveland R, Cleveland W, McRae J, Terpenning I. STL: a seasonal-trend decomposition procedure based on Loess (with discussion). J Off Stat [Internet]. 1990;6(1):3-73. Available from: https://www.wessa.net/download/stl.pdf
   https://www.wessa.net/download/stl.pdf... TBATS;¹¹11. Durbin J, Koopman SJ. Time series analysis by state space methods. Oxford University Press; 2013.BATS;¹¹11. Durbin J, Koopman SJ. Time series analysis by state space methods. Oxford University Press; 2013. STLM¹²12. Crone SF, Kourentzes N. Feature selection for time series prediction - A combined filter and wrapper approach for neural networks. Neurocomputing [Internet]. 2010 Jun [cited 2020 Sep 28];73(10-12):1923-36. Available from: https://doi.org/10.1016/j.neucom.2010.01.017
   https://doi.org/10.1016/j.neucom.2010.01... );

2. stochastic models (StructTS;¹⁴14. Hyndman RJ, Athanasopoulos G. Forecasting: principles and practice. 2nd ed. [S.l.]: OTexts; 2018. NNETAR;⁹9. Dokumentov A, Hyndman RJ. STR: a seasonal-trend decomposition procedure based on regression. Monash Econometrics and Business Statistics Working Papers [Internet]. 2015 [cited 2020 Sep 28]. Available from: https://robjhyndman.com/publications/str/
   https://robjhyndman.com/publications/str... ELM;¹⁵15. Morettin PA, Toloi CM. Análise de séries temporais: modelos lineares univariados. 3rd ed. São Paulo: Blucher; 2018.MLP¹⁶16. Antunes JLF, Cardoso MRA. Uso da análise de séries temporais em estudos epidemiológicos. Epidemiol Serv Saúde [Internet]. 2015 jul-set [cited 2020 set 28];24(3):565-76. Disponível em: http://dx.doi.org/10.5123/S1679-49742015000300024
   http://dx.doi.org/10.5123/S1679-49742015... ); and

3. null model.¹⁷17. Helfenstein U. Box-jenkins modelling of some viral infectious diseases. Stat Med [Internet]. 1986 Jan-Feb [cited 2020 Sep 28];5(1):37-47. Available from: https://doi.org/10.1002/sim.4780050107
   https://doi.org/10.1002/sim.4780050107...

Three criteria were considered for assessment of the predictive power of the statistical models:

1. Mean Absolute Percentage Error (MAPE), where At is the real value and Ft is the prediction value. The values are added up for each point predicted in time, and the result is divided by the number of n points. Multiplying by 100% makes MAPE a percentage error, according to the following formula:

1. Relative MAPE scale, in which the null model MAPE is divided by the MAPE values of the remaining models. If the result of such division is less than or equal to 1, the model is classified as bad; if it is greater than 1 and less than or equal to 2, the model is classified as having low predictive power; and if it is greater than 2, the model presents reliable predictive power.

2. Uncertainty coefficient (Theil’s U). This is a relative accuracy measure, intended to eliminate statistical models with high deviations in relation to the average value. Values less than 1 mean the predictive power is reliable.¹⁷17. Helfenstein U. Box-jenkins modelling of some viral infectious diseases. Stat Med [Internet]. 1986 Jan-Feb [cited 2020 Sep 28];5(1):37-47. Available from: https://doi.org/10.1002/sim.4780050107
   https://doi.org/10.1002/sim.4780050107...

A predictive power deemed acceptable was then defined as (i) lowest MAPE value, (ii) values greater than 2 for the null model MAPE scale, and (iii) values less than 1 for Theil’s U.

The analyses were conducted in the R computing environment, through a corrected and validated script that can be reproduced by Municipal and State Health Department surveillance staff in Brazil (Supplementary material).¹⁹19. Lima MVM. Protocolo de programação em ambiente R e banco de dados disponíveis em repositório de acesso público [Internet]. [S.l.]: GitHub; 2020 [citado 2020 ago 5]. Disponível em: https://github.com/MVMLima/artigo_RESS_malaria
https://github.com/MVMLima/artigo_RESS_m...

Results

The total number of malaria cases was 403,832, with an average of 1,771 cases per month and median of 1,518 cases/month, with 1,021 cases in the first quartile and 2,079 cases in the third quartile - standard deviation (SD) = 918 cases. The increased Dickey-Fuller value showed that the time series is stationary (test value = -5.352; p-value <0.01). The minimum number of cases was 487 (May 2014), while the maximum was 5,944 (October 2000), consistent with the seasonal nature of malaria in the state (Table 1).

Table 2 presents the performance of the models according to the criteria for assessing predictive power and the three selected prediction time horizons. In the 12 and six month time horizons, all deterministic models (ETS, ARIMA, STLM, BATS and TBATS) showed reliable predictive power. However, none of the stochastic models showed reliability in predicting future cases of malaria for those time horizons.

The three-month time horizon surprisingly showed itself to be challenging for all the models. Finally, no model was deemed reliable for predicting malaria cases three months in advance (Figure 2).

Discussion

Deterministic models showed themselves to be reliable for predicting the monthly number of cases of malaria in the next 12 or six months in the state of Amapá. This result may be interpreted as follows: the time series being studied has characteristics that enable better performance by deterministic models. This is because time series with (i) a strong seasonal component, (ii) relatively low linear trend (for example, temporal stationarity) and (iii) low or null stochastic effect are satisfactorily adjusted by deterministic models such as ARIMA. Theoretically, statistical models able to detect stochastic effects can present better performance, in comparison to deterministic models, in the case of time series with (i) lack of seasonality, (ii) high linear trend, and (iii) stochastic effects.²⁰20. Baquero OS, Santana LMR, Chiaravalloti-Neto F. Dengue forecasting in São Paulo city with generalized additive models, artificial neural networks and seasonal autoregressive integrated moving average models. PLoS One [Internet]. 2018 Apr [cited 2020 Sep 28];13(4):1-12. Available from: https://doi.org/10.1371/journal.pone.0195065
https://doi.org/10.1371/journal.pone.019... However, none of the assessed models presented reliable results for predicting the number of monthly cases of malaria in the three-month time horizon. Prediction of malaria in short future periods also proved to be unsatisfactory in another study, in which deterministic models were applied in districts of Sri Lanka between 1972 and 2005.¹⁸18. Briët OJT, Vounatsou P, Gunawardena DM, Galappaththy GNL, Amerasinghe PH. Models for short term malaria prediction in Sri Lanka. Malar J [Internet]. 2008 May [cited 2020 Sep 28];7(76). Available from: https://github.com/MVMLima/artigo_RESS_malaria
https://github.com/MVMLima/artigo_RESS_m... That study discouraged using statistical models for predicting malaria cases in near future periods of one month in advance, notwithstanding the period of the study in Sri Lanka being shorter than the shortest period in our study.

A limitation of the present study is the impossibility of predicting malaria cases for short periods in advance, such as 3 months. Another limitation is the fact that, when using the state of Amapá as unit of analysis, local scale information is lost. The advantages of this approach, however, are the possibility of predicting malaria cases 12 or six months in advance and using deterministic statistical models. As these models are easier for health service managers to understand, this makes their implementation in health services more viable.²¹21. World Health Organization - WHO. Malaria epidemics: forecasting, prevention, early detection and control. Lesyin: World Health Organization; 2004 [cited 2020 Sep 28]. 48 p. Available from: https://apps.who.int/iris/handle/10665/70073
https://apps.who.int/iris/handle/10665/7...

The potential use of time series techniques in epidemiological studies, disease surveillance and malaria outbreak prediction has been explored in different studies.¹⁸18. Briët OJT, Vounatsou P, Gunawardena DM, Galappaththy GNL, Amerasinghe PH. Models for short term malaria prediction in Sri Lanka. Malar J [Internet]. 2008 May [cited 2020 Sep 28];7(76). Available from: https://github.com/MVMLima/artigo_RESS_malaria
https://github.com/MVMLima/artigo_RESS_m... For example, a study with similar design, conducted in the north of Thailand from 1999 to 2004, found deterministic models enabling future prediction of malaria and dengue fever one to four months in advance, to the extent its authors suggest using such models for allocating resources in controlling and preventing these diseases.²²22. Sriwattanapongse W, Khanabsakdi S. Modeling and forecasting malaria and dengue hemorrhagic fever incidence and prevalence in Nothern Thailand. J Math Syst Sci [Internet]. 2011 [cited 2020 Sep 28];1(1):52-9. Available from: https://www.semanticscholar.org/paper/Modeling-and-Forecasting-Malaria-and-Dengue-Fever-Sriwattanapongse-Khanabsakdi/b5de80ebedda6ce67bd6e5cdfee666ea2270e98d
https://www.semanticscholar.org/paper/Mo... Another study with deterministic models, conducted in Sudan, between 2009 and 2013, showed that the predictive power of the models used varied according to each state (of that country).²³23. Hussien HH, Eissa FH, Awadalla KE. Statistical methods for predicting malaria incidences using data from Sudan. Malar Res Treat [Internet]. 2017 [cited 2020 Sep 28]:4205957. Available from: https://doi.org/10.1155/2017/4205957
https://doi.org/10.1155/2017/4205957... Deterministic models used for predicting malaria in Afghanistan, between 2005 and 2015, led to reliable predictions for 12 to four months in advance.²⁴24. Anwar MY, Lewnard JA, Parikh S, Pitzer VE. Time series analysis of malaria in Afghanistan: using ARIMA models to predict future trends in incidence. Malar J [Internet]. 2016 Nov [cited 2020 Sep 28];15(1):566. Available from: https://doi.org/10.1186/s12936-016-1602-1
https://doi.org/10.1186/s12936-016-1602-...

Considering that, among deterministic models, ARIMA is the most widely applied in the literature and has predictive characteristics for diseases affected by seasonality,²²22. Sriwattanapongse W, Khanabsakdi S. Modeling and forecasting malaria and dengue hemorrhagic fever incidence and prevalence in Nothern Thailand. J Math Syst Sci [Internet]. 2011 [cited 2020 Sep 28];1(1):52-9. Available from: https://www.semanticscholar.org/paper/Modeling-and-Forecasting-Malaria-and-Dengue-Fever-Sriwattanapongse-Khanabsakdi/b5de80ebedda6ce67bd6e5cdfee666ea2270e98d
https://www.semanticscholar.org/paper/Mo... we recommend its implementation as a protocol for predicting monthly malaria cases for long-term horizons - 12 or six months – on a state-wide scale in the Brazilian Amazon state of Amapá.

References

Publication Dates

History