Use of HPV testing in cervical cancer screening services in Mexico, 2008-2018: a nationwide database study

Uso de la prueba de VPH para la detección de cáncer cervical en México, 2008-2018: estudio de una base de datos nacional

Erika Hurtado-Salgado Eduardo Ortiz-Panozo Jorge Salmerón Nenetzen Saavedra-Lara Pablo Kuri-Morales Eduardo Pesqueira-Villegas Rufino Luna-Gordillo Eduardo L Franco Eduardo Lazcano-Ponce About the authors

Abstract:

Objective:

To describe the methods of a study aimed at evaluating high risk-HPV (hrHPV)-based screening and cervical cytology as triage compared to conventional cervical cytology as primary screening in the detection of grade 2+ cervical intraepithelial neoplasia in the National Cancer Screening Program (NCSP) of Mexico.

Materials and methods:

We will use information originated from the Womens Cancer Information System of Mexico regarding cervical cancer from 2008 to 2018. The database includes cytology results, diagnostic confirmation by histopathology and/or treatment colposcopy. We will then carry out statistical analyses on approximately 15 million hrHPV.

Results:

We will evaluate the overall performance of hrHPV-based screening as part of the NCSP and compare hrHPV-based to cytology-based screening under real-life conditions. To guarantee an unbiased comparison between hrHPV with cytology triage and conventional cytology we will use propensity score matching.

Conclusion:

Decision makers may use our results to identify areas of opportunity for improvement in NCSP processes.

Keywords:
cervical intraepithelial neoplasia; cervical smears; DNA probes, HPV; health informations systems; Mexico

Resumen:

Objetivo:

Describir los métodos de un estudio que busca comparar el beneficio de la introducción de la prueba de VPH de alto riesgo como prueba primaria frente a la citología convencional para la detección de la neoplasia intraepitelial cervical grado 2 o mayor, dentro del Programa de Prevención y Control del Cáncer de la Mujer, para el periodo de 2008 a 2018.

Material y métodos:

Se utilizarán los registros del Sistema de Información de Cáncer de la Mujer, se realizarán los análisis estadísticos con aproximadamente 15 millones de resultados de VPH-alto riesgo, además se utilizarán los resultados de citología, colposcopia, histología y los casos referenciados al centro oncológico para tratamiento. Para comparar ambos grupos usaremos “propensity score matching”.

Resultados:

Se evaluará el desempeño general de la prueba de VPH-alto riesgo, en condiciones reales dentro del Programa Nacional de Prevención y Control de la Mujer y su tendencia en el tiempo.

Conclusiones:

Los resultados de estudio ayudarán a los tomadores de decisiones a identificar áreas de oportunidad para mejorar el programa en México.

Palabras clave:
neoplasia intraepitelial cervical; protis cervical; sondas de ADN de HPV; sistemas de información en salud; México

Introduction

Cervical cancer (CC) is the fourth most prevalent malignant tumor presented in women worldwide, with an estimated nearly 570 thousand new cases in 2018.11. Antoni S, Soerjomataram I, Møller B, Bray F, Ferlay J. An assessment of GLOBOCAN methods for deriving national estimates of cancer incidence. Bull World Health Organ. 2016;94(3):174-84. https://doi.org/10.2471/BLT.15.164384
https://doi.org/10.2471/BLT.15.164384...
The International Agency for Research on Cancer estimates that the global CC incidence was 13.1 per 100 000 women of all ages.11. Antoni S, Soerjomataram I, Møller B, Bray F, Ferlay J. An assessment of GLOBOCAN methods for deriving national estimates of cancer incidence. Bull World Health Organ. 2016;94(3):174-84. https://doi.org/10.2471/BLT.15.164384
https://doi.org/10.2471/BLT.15.164384...
,22. Almonte M, Murillo R, Sánchez GI, Jerónimo J, Salmerón J, Ferrecio C, et al. Nuevos paradigmas y desafíos en la prevención y control del cáncer de cuello uterino en América Latina. Salud Publica Mex. 2010;52(6):544-59. https://doi.org/10.1590/S0036-36342010000600010
https://doi.org/10.1590/S0036-3634201000...
According to Global Burden of Disease data, the age-standardized incidence rate of CC was 12 cases per 100 000 people in Mexico.33. Gómez-Dantés H, Lamadrid-Figueroa H, Cahuana-Hurtado L, Silverman-Retana O, Montero P, González-Robledo MC, et al. The burden of cancer in Mexico, 1990-2013. Salud Publica Mex . 2016;58(2):118-31. https://doi.org/10.21149/spm.v58i2.7780
https://doi.org/10.21149/spm.v58i2.7780...
CC was the primary cause of death by neoplasia in Mexican women up to the year 2005. After that year, breast cancer replaced it as the primary cause of death and CC became the second most frequent cause of death from neoplastic diseases.44. Palacio-Mejía LS, Lazcano-Ponce E, Allen-Leigh B, Hernández-Ávila M. Diferencias regionales en la mortalidad por cáncer de mama y cérvix en México entre 1979 y 2006. Salud Publica Mex . 2009;51:s208-19. https://doi.org/10.1590/S0036-36342009000800011
https://doi.org/10.1590/S0036-3634200900...
However, at a sub-national level, CC remains the primary cause of cancer death in women in twelve of thirty-two states of Mexico.55. Instituto Nacional de Salud Pública. Evaluación del programa de Prevención y Control del Cáncer Cervical en México, 2008-2011. Problemas y Recomendaciones Cuernavaca: INSP, 2011 [cited 2017, Nov 14]. Available from: https://www.insp.mx/images/stories/Centros/cisp/Docs/120412_cneg.pdf
https://www.insp.mx/images/stories/Centr...
CC continues as the most important cancer in a number of low income countries.66. Bray F, Piñeros M. Cancer patterns, trends and projections in latin america and the caribbean: A global context. Salud Publica Mex . 2016;58(2):104-17. https://doi.org/10.21149/spm.v58i2.7779
https://doi.org/10.21149/spm.v58i2.7779...

The Census of Population and Housing estimates a reduction in the mortality rate of cervical cancer in Mexico from 1990 to 2010 (10.2 a 9.7 per 100 000).77. Torres-Sánchez LE, Rojas-Martínez R, Escamilla-Núñez C, Vara-Salazar ED, Lazcano-Ponce E. Tendencias en la mortalidad por cáncer en México de 1980 a 2011. Salud Publica Mex . 2014;56(5):473-91. https://doi.org/10.21149/spm.v56i5.7373
https://doi.org/10.21149/spm.v56i5.7373...
This reduction in mortality is mainly attributable to the decrease in the birth rate and the increase in cervical cytology coverage.88. Lazcano-Ponce E, Palacio-Mejia LS, Allen-Leigh B, Yunes-Diaz E, Alonso P, Schiavon R, Hernandez-Avila M. Decreasing cervical cancer mortality in Mexico: effect of Papanicolaou coverage, birthrate, and the importance of diagnostic validity of cytology. Cancer Epidemiol Biomarkers Prev. 2008;17(10):2808-17. https://doi.org/10.1158/1055-9965.EPI-07-2659
https://doi.org/10.1158/1055-9965.EPI-07...
,99. Torres-Mejía G, Ortega-Olvera C, Ángeles-Llerenas A, Villalobos-Hernández AL, Salmerón-Castro J, Lazcano-Ponce E, Hernández-Ávila M. Patrones de utilización de programas de prevención y diagnóstico temprano de cáncer en la mujer. Salud Publica Mex . 2013;55(Suppl 2):241-8. https://doi.org/10.21149/spm.v55s2.5121
https://doi.org/10.21149/spm.v55s2.5121...
Rural zones still show the highest prevalence and mortality risks, even though the cervical cancer screening coverage has increased from 33.3% in the year 2000 to 49% in the year 2012.99. Torres-Mejía G, Ortega-Olvera C, Ángeles-Llerenas A, Villalobos-Hernández AL, Salmerón-Castro J, Lazcano-Ponce E, Hernández-Ávila M. Patrones de utilización de programas de prevención y diagnóstico temprano de cáncer en la mujer. Salud Publica Mex . 2013;55(Suppl 2):241-8. https://doi.org/10.21149/spm.v55s2.5121
https://doi.org/10.21149/spm.v55s2.5121...

The National Cancer Screening Program of Mexico (NCSP) uses secondary prevention strategies to benefit the most vulnerable population sectors. Since 1974, NCSP has adopted conventional cervical cytology for all women under 65 years old into routine screening in Mexico. In 2008, the hrHPV test was incorporated as a screening procedure for all women older than 35 years. Conventional cervical cytology continued as the strategy for woman younger than 35 years. The positive cases detected by hrHPV-based screening are evaluated with cytology as a triage test. Given the high sensitivity (96%) but low specificity (76%) of the hrHPV screening, a reflex triage testing with cytology is necessary before deciding which women need a colposcopy examination.1010. Ronco G, Zappa M, Franceschi S, Tunesi S, Caprioglio A, Confortini M. Impact of variations in triage cytology interpretation on human papillomavirus-based cervical screening and implications for screening algorithms. Eur J Cancer. 2016;68:148-55. https://doi.org/10.1016/j.ejca.2016.09.008
https://doi.org/10.1016/j.ejca.2016.09.0...
The current clinical guidelines with this approach have demonstrated an increase in negative predictive value of the hrHPV test.1111. Schiffman M, Wentzensen N, Wacholder S, Kinney W, Gage JC, Castle PE. Human papillomavirus testing in the prevention of cervical cancer. J Natl Cancer Inst. 2011;103(5):368-83. https://doi.org/10.1093/jnci/djq562
https://doi.org/10.1093/jnci/djq562...
The hrHPV with cytology triage strategy is an excellent combination for women older than 30 years who may experience greater security when reassured by both tests that they are free of the disease.1212. Lazcano-Ponce E, Lörincz AT, Salmerón J, Fernández I, Cruz A, Hernández P, et al. A pilot study of HPV DNA and cytology testing in 50,159 women in the routine Mexican social security program. Cancer Causes Control. 2010;21(10):1693-700. https://doi.org/10.1007/s10552-010-9598-2
https://doi.org/10.1007/s10552-010-9598-...
,1313. Flores YN, Bishai DM, Lorincz A, Shah KV, Lazcano-Ponce E, Hernández M, et al. HPV testing for cervical cancer screening appears more cost-effective than Papanicolau cytology in Mexico. Cancer Causes Control . 2011;22(2):261-72. https://doi.org/10.1007/s10552-010-9694-3
https://doi.org/10.1007/s10552-010-9694-...
All information about NCSP activities are documented in Women Cancer Information System (SICAM, for its Spanish acronym) database in operation since 2000, which holds around 28 million records registers between 2008 and 2018.

Cervical screening has contributed significantly to the decrease in CC, mainly in high income countries. In Europe, large-scale clinical trials reported that hrHPV-based screening detects 70% more cervical lesions than conventional cytology.1414. Ronco G, Dillner J, Elfstrom M, Tunesi S, Snijders P, Arbyn M. Efficacy of HPV-based screening for prevention of invasive cervical cancer: follow-up of four European randomised controlled trials. Lancet. 2013;383(13):524-32. https://doi.org/10.1016/S0140-6736(13)62218-7
https://doi.org/10.1016/S0140-6736(13)62...
,1515. Leinonen MK, Nieminen P, Lönnberg S, Malila N, Hakama M, Pokhrel A, et al. Detection rates of precancerous and cancerous cervical lesions within one screening round of primary human papillomavirus DNA testing: Prospective randomised trial in Finland. BMJ. 2012;345(7886):1-11. https://doi.org/10.1136/bmj.e7789
https://doi.org/10.1136/bmj.e7789...
In other populations, studies reported 90% and more detected high-grade cancer precursors using this strategy.1616. Franco EL, Duarte-Franco E, Ferenczy A. Cervical cancer: epidemiology, prevention and the role of human papillomavirus infection. Can Med Assoc J. 2001:1017-25. In Mexico, evidence from controlled studies indicated that hrHPV improves the detection of precursor cervical cancer lesions.1212. Lazcano-Ponce E, Lörincz AT, Salmerón J, Fernández I, Cruz A, Hernández P, et al. A pilot study of HPV DNA and cytology testing in 50,159 women in the routine Mexican social security program. Cancer Causes Control. 2010;21(10):1693-700. https://doi.org/10.1007/s10552-010-9598-2
https://doi.org/10.1007/s10552-010-9598-...
Chile has had similar results with hrHPV screening.22. Almonte M, Murillo R, Sánchez GI, Jerónimo J, Salmerón J, Ferrecio C, et al. Nuevos paradigmas y desafíos en la prevención y control del cáncer de cuello uterino en América Latina. Salud Publica Mex. 2010;52(6):544-59. https://doi.org/10.1590/S0036-36342010000600010
https://doi.org/10.1590/S0036-3634201000...
The benefit provided by the introduction of hrHPV testing in the existing screening programs is enormous. When carried out together, hrHPV screening with cytology triage test are capable of timely detection of a greater number of precursor lesions. In fact, screening intervals can be extended when tests are carried out in parallel or sequentially, and operational costs are reduced.1616. Franco EL, Duarte-Franco E, Ferenczy A. Cervical cancer: epidemiology, prevention and the role of human papillomavirus infection. Can Med Assoc J. 2001:1017-25.,1717. Franco EL. A New generation of studies of human papillomavirus DNA testing in cervical cancer screening. JNCIJ Natl Cancer Inst . 2009;101(23):1600-01. https://doi.org/10.1093/jnci/djp392
https://doi.org/10.1093/jnci/djp392...

The efficacy of hrHPV-based screening in NCSP has not yet been evaluated. In addition, there are still inequalities in cervical cancer regarding diverse sociodemographic factors, which are not fully understood. A performance comparison between hrHPV with cytology triage and conventional cytology in the NCSP will provide evidence about the benefits of introducing hrHPV testing in uncontrolled settings. An evaluation of how these variations are present in Mexico and how they influence the heterogeneity of the performance of NCSP is warranted. This work would also contribute to the ongoing efforts towards the creation of a public policy for the management of CC.1818. Reynoso-Noverón N, Meneses-García A, Erazo-Valle A, Escudero-de los Ríos P, Kuri-Morales PA, Mohar-Betancourt A. Challenges in the development and implementation of the National Comprehensive Cancer Control Program in Mexico. Salud Publica Mex . 2016;58(2):325-33. https://doi.org/10.21149/spm.v58i2.7804
https://doi.org/10.21149/spm.v58i2.7804...

Therefore, we describe the methods of nationwide database study. Our work is aimed to evaluate hrHPV screening with cervical cytology as triage, compared to conventional cervical cytology as the primary screening for detection of grade 2+ cervical intraepithelial neoplasia (CIN2+), using information extracted from the SICAM database. Additionally, our study will evaluate potential regional differences in the prevalence of hrHPV infection and CIN +2 among the states of Mexico within the NCSP and generate information about trends of hrHPV infection by age group and geographical zone.

Hypothesis

The hrHPV with cytology triage strategy will detect twice as many CIN 2+ lesions as the conventional cytology strategy when used in primary screening.

Materials and methods

National Cancer Screening Program

Figure 1, in which NCSP actions are summarized, is modified from the Specific Action Program and shows the current care model for CC prevention and control.1919. Guadarrama F. Prevención y Control del Cáncer de la Mujer. Ciudad de México: Centro Nacional de Equidad de Género y Salud Reproductiva, 2013 [cited 2017, Nov 15]. Available from: http://cnegsr.salud.gob.mx/contenidos/descargas/cama/PrevencionyControldelCancerdelaMujer_2013_2018.pdf
http://cnegsr.salud.gob.mx/contenidos/de...
This algorithm is used in the health services provided by the Ministry of Health (MoH) in Mexico. Medical care that is provided by the health services of MoH is mainly directed towards Seguro Popular users (population without access to a social security benefits). However, medical care related to prevention of CC can be provided to anyone who requests it, regardless of his or her health insurance affiliation.

Figure 1
National Cancer Screening Program Actions

Over time, the federal government has tried to unify sanitary service strategies to guarantee effective and timely detection of CC. This has been accomplished by means of the creation of guidelines called the Official Mexican Regulations which are nested in the NCSP, with particular objectives, strategies and lines of action for cancer prevention and control in women.2020. Secretaría de Salud. Modificación a la Norma Oficial Mexicana NOM-014-SSA2-1994, para la Prevención Tratamiento y Control, del Cáncer del Cuello del Útero y Mamario en la atención primaria, para quedar como, NOM-014-SSA2-1994, Para la Prevención, Detección, Diagnóstico, Trata. México: DOF, 1993 [cited 2017, Nov 14]. Available from: http://www.salud.gob.mx/unidades/cdi/nom/m014ssa24.html
http://www.salud.gob.mx/unidades/cdi/nom...
The current government proposes indicators which favor increased coverage of prevention programs and a reduction in morbidity and mortality due to cancer in women. Goal compliance will be verified by means of the National Development Plan from 2013 to 2018 and the Sectorial Health Plan (PROSESA) from 2013 to 2018.2121. Comisión Nacional contra las Adicciones. Programa Sectorial de Salud. México: Conadic [cited 2018, Jun 15]. Available from: http://www.conadic.salud.gob.mx/pdfs/sectorial_salud.pdf
http://www.conadic.salud.gob.mx/pdfs/sec...

SICAM database

The procedure for information integration in SICAM consists of manual processes involving interactions between health providers (users) and the system (SICAM managers). Users enter records of all procedures related with CC screening and follow-up of cases (hrHPV, cytology, colposcopy and biopsy). The information connects online and updates the system in real time from the site where the cervical samples are collected. The SICAM managers implement information validations continuously and are responsible for the final information to users.

Sometimes manual processes result in certain inconsistencies attributable to human error, such as from the user entering data and the patient providing information. Such errors occur (naturally) in large volume information system. Therefore, data cleaning will be a challenge which we will address by applying appropriate algorithms to minimize record duplicity.

Information and Communication Technologies (TIC) in SICAM represents another important challenge due to the more than 300 GB of available information. This large volume requires a high-performance technological infrastructure. The use of TIC in SICAM is necessary due to its various automatized processes that help guarantee information reliability.

In this context, the SICAM information collection and integration framework consists in properly clarifying data input and output based on the information management model in accordance with the current health information systems regulation (NOM-035-SSA3-2012 and NOM-024-SSA3-2012). Information technologies (IT) have been implemented as well to allow access, production, safety, treatment and communication of the information itself so as to guarantee informed decision-making. Figure 2 shows both physical and logical models, which uses state-of-the-art technology to ensure that information remains secure online. The value of this information increases as long it is available and accessible to the authorities responsible for decisions related to health programs.

We will process SICAM information in a virtual space with a basic execution unit of one processing core. Each calculation standard node has 24 cores (2 Intel Xeon E5-2680 v3 sockets at 2.5 Ghz with 12 cores per socket) and 128 GB of shared RAM.

Figure 2
Data collection and integration framework of SICAM

Study design and population

We will analyze the information related to cervical cancer in SICAM during the period from 2008 to 2018 in users of Seguro Popular. There are 28 million records related to cervical cancer in the SICAM database; 21 million correspond to cytology-based screening, and 7 million correspond to hrHPV-based screening. The SICAM administrators will provide us only with variables directly related to the study objectives; thus, preventing access to confidential data. We will establish permanent data cleaning algorithms to avoid duplicates to the extent possible. We will conduct a quasi-experimental design to compare and contrast both strategies of CC screening in real life conditions where the CC screening randomization is impossible to carry out in population level.2222. Bärnighausen T, Røttingen JA, Rockers P, Shemilt I, Tugwell P. Quasi-experimental study designs series-paper 1: introduction: two historical lineages. J Clin Epidemiol. 2017;89:4-11. https://doi.org/10.1016/j.jclinepi.2017.02.020
https://doi.org/10.1016/j.jclinepi.2017....
,2323. Geldsetzer P, Fawzi W. Quasi-experimental study designs series-paper 2: complementary approaches to advancing global health knowledge. J Clin Epidemiol . 2017;89:12-6. https://doi.org/10.1016/j.jclinepi.2017.03.015
https://doi.org/10.1016/j.jclinepi.2017....
,2424. Frenk J, Gómez-Dantés O. Quasi-experimental study designs series-paper 3: systematic generation of evidence through public policy evaluation. J Clin Epidemiol . 2017;89:17-20. https://doi.org/10.1016/j.jclinepi.2017.03.013
https://doi.org/10.1016/j.jclinepi.2017....
,2525. Bärnighausen T, Tugwell P, Røttingen JA, Shemilt I, Rockers P, Geldsetzer P, et al. Quasi-experimental study designs series-paper 4: uses and value. J Clin Epidemiol . 2017;89:21-29. https://doi.org/10.1016/j.jclinepi.2017.03.012
https://doi.org/10.1016/j.jclinepi.2017....

After applying eligibility criteria, we expect to carry out the statistical analyses in a sample of approximately 15 million records within the database. Such registries include results for hrHPV, cervical cytology, colposcopy and biopsy, as well as a referral of confirmed cases to oncological centers. We will use complete records of patients who underwent screening and evaluation for diagnosis confirmation. Specifically, we will use complete results of hrHPV as well as data available on cytology, colposcopy and biopsy to meet our objectives. We will not include records with missing or pending test results. We will analyze separately records identified as repeated measures. Figures 3 and 4 provide details of the study design.

Figure 3
Study design proposed to estimate prevalence of hrHPV

Figure 4
Study design proposed to estimate the relative sensitivity of hrHPV based screening versus cytology screening through regression analysis

In the first phase, we will generate information about the prevalence of HPV infection and trends over time by age group and geographical zone. To estimate hrHPV infection prevalence, we will analyze all hrHPV test records so as to carry out descriptive analysis over the study period. The analysis will be adjusted for covariates. We also expect to determine the hrHPV frequency trend over time and its variations in accordance with the degree of municipal marginalization, depending on sociodemographic and obstetric-gynecologic factors; we provide details in figure 3.

In the second phase, we will compare the performance of hrHPV with cytology triage and conventional cytology. To estimate the relative sensitivity analysis, two groups of women will be analyzed: 1) hrHPV with cytology triage, and 2) conventional cytology. The purpose will be to compare the amount of positive cases undergoing colposcopy for diagnostic confirmation by each strategy. Patients will be reclassified after this assessment and in accordance with the biopsy results; only those results confirmed by histopathology will be considered cases in each screening strategy. Positive results to CIN 2+, which were referred for treatment according to local clinical guidelines.

Given that two different groups will be analyzed, we plan to guarantee the comparability between hrHPV with cytology triage and conventional cytology by propensity score matching, using age, collection date, degree of municipal marginalization and geographical zone as matching variables. We will adjust for the remaining (unbalanced) characteristics in statistical models.

In addition, we expect to estimate variations in relative sensitivity by stratifying for age, geographical zone and degree of municipal marginalization. Identifying these variations may contribute to understanding the performance of NCSP in specific groups. Figure 4 shows details of the approach for this analysis.

These results could demonstrate the benefit of hrHPV testing in a population-based setting and under uncontrolled conditions. Furthermore, we expect this study to contribute to other ongoing studies in the region which are evaluating the performance and cost-effectiveness of different triage tests to detect cervical CIN2+ and the impact of HPV vaccination in a hrHPV based primary screening program to extend the screening interval.2626. Torres-Ibarra L, Lazcano-Ponce E, Franco EL, Shemilt I, Rockers P, Geldsetzer P, et al. Triage strategies in cervical cancer detection in Mexico: Methods of the FRIDA study. Salud Publica Mex . 2016;58(2):197-210. https://doi.org/10.21149/spm.v58i2.7789
https://doi.org/10.21149/spm.v58i2.7789...
,2727. Salmerón J, Torres-Ibarra L, Bosch FX, Cuzick J, Lörincz A, Wheeler CM, et al. HPV vaccination impact on a cervical cancer screening program: methods of the FASTER-Tlalpan Study in Mexico TT - Impacto de la vacunación de VPH en un programa de tamizaje de cáncer cervical: métodos del Estudio FASTER-Tlalpan en México. Salud Publica Mex . 2016;58(2):211-9. https://doi.org/10.21149/spm.v58i2.7790
https://doi.org/10.21149/spm.v58i2.7790...
,2828. Tejada RA, Vargas KG, Benites-zapata V, Mezones-Holguín E, Bolaños-Díaz R, Hernandez AV. Human papillomavirus vaccine efficacy in the prevention of anogenital warts: systematic review and meta-analysis. Salud Publica Mex . 2017;59(1):84-94. https://doi.org/10.21149/7824
https://doi.org/10.21149/7824...

Unfortunately, not all patients who underwent colposcopy adhered adequately to the program. Patients who failed to undergo colposcopy are considered as lost to follow-up. This lack of information will be considered as a possible source of bias.2929. Wentzensen N, Walker JL, Gold MA, Smith KM, Zuna RE, Mathews C, et al. Multiple Biopsies and Detection of Cervical Cancer Precursors at Colposcopy. J Clin Oncol. 2014;33(1):83-9. https://doi.org/10.1200/JCO.2014.55.9948
https://doi.org/10.1200/JCO.2014.55.9948...
This constitutes one of the greatest problems and may result in program failure.55. Instituto Nacional de Salud Pública. Evaluación del programa de Prevención y Control del Cáncer Cervical en México, 2008-2011. Problemas y Recomendaciones Cuernavaca: INSP, 2011 [cited 2017, Nov 14]. Available from: https://www.insp.mx/images/stories/Centros/cisp/Docs/120412_cneg.pdf
https://www.insp.mx/images/stories/Centr...
Another cause of failure in screening programs is delayed and inadequate reporting of screening results to patients and providers. However, we expect the loss to follow-up to be non-differential with respect to the screening modalities being compared.

Inclusion criteria

  • Women between 30 and 64 years

  • Complete screening, follow-up and diagnostic confirmation records

  • Complete basic personal information: age, geographical zone and area of residence

Exclusion criteria

  • Duplicated records, that is to say, those records that have the same collection date and whose patient ID number is the same

  • Divergent results within the same collection date

  • Results that have an interval between tests of one month or less

  • Hysterectomy

Clinical measurements of interest (endpoints)

Table I shows the main study variables. Primary endpoints will be CIN 2+, cytology, colposcopy and biopsy with positive results. Other important variables will be age, geographical zone and municipal marginalization, and age at first sexual intercourse.

Positive hrHPV: cases with a positive result on hrHPV-based screening, regardless of the used DNA extraction method. Results are reported as positive, negative and some cases invalid samples.

  • Positive cervical cytology: ASCUS (Atypical Squamous Cells of Undetermined Significance) or a more severe result in cytology. The guidelines recommend a systematically ongoing monitoring of these patients.3030. James H. Nelson, Hervy E. Averette RMR. Dysplasia, carcinoma in situ, and early invasive cervical carcinoma. Cancer J Clin. 1984;34(6):306-27. https://doi.org/10.3322/canjclin.34.6.306
    https://doi.org/10.3322/canjclin.34.6.30...
    All cytology results of ASCUS or more severe patients must undergo a colposcopy for diagnostic confirmation.

  • Colposcopy: the evaluation will be carried out according to the local clinical criteria. Typically, the biopsy specimen is collected on the cervical lesion with the worst apparent severity.

  • Positive biopsy: CIN 2+ in pathology results, referred for diagnostic confirmation due to having a cytology ≥ ASCUS. Such patients will be considered as cases.

Table I
Principal variables employed in study

Additional information sources

Marginalization

According to Census of Population and Housing, degree of marginalization is formed by four indicators (education, housing, population distribution and income due to work) and is available in different scales and at different levels.3131. Consejo Nacional de Población. Indices de marginación por entidad federativa y municipio. Índices De Marginación. México: Conapo, 2010:73-189.,3232. Consejo Nacional de Población. Índices de Marginación. México: Conapo , 2012 [cited 2017, Nov 19]. Available from: http://www.conapo.gob.mx/es/CONAPO/Indices_de_Marginacion
http://www.conapo.gob.mx/es/CONAPO/Indic...
On this study, we will use public information about the marginalization index at municipal level, which is categorized as very high, high, mid, low and very low. “Very high” category represents the lowest degree of marginalization.

Geographical zone

According to Instituto Nacional de Estadística y Geografía (INEGI) geographical statistics.3333. Instituto Nacional de Estadística y Geografía. Regiones socioeconómicas de México. México: Inegi, 2000 [cited 2018, Jan 31]. Available from: http://www3.inegi.org.mx/sistemas/cntmp/itaeegranjero.aspx
http://www3.inegi.org.mx/sistemas/cntmp/...
The following geographical zones are proposed to describe the prevalence of hrHPV infection. We chose this organization because we hypothesize that the zones are relatively homogeneous in their socio-demographic, economic, and cultural characteristics which in turn may be related to the hrHPV infection.

Zone 1. Northwest: Baja California, Baja California Sur, Chihuahua, Sinaloa, Sonora, Coahuila, Nuevo León, Tamaulipas

Zone 2. Center-North: Aguascalientes, Colima, Durango, Guanajuato, Jalisco, Nayarit, San Luis Potosí y Zacatecas

Zone 3. Center: Ciudad de México, Edo. de México,

Zone 4. Center-South: Guerrero, Hidalgo, Michoacán, Morelos, Puebla, Querétaro y Tlaxcala

Zone 5. South-Southeast: Campeche, Chiapas, Oaxaca, Quintana Roo, Tabasco, Veracruz, Yucatán.

Statistical analysis plan

We will estimate the prevalence of hrHPV infection and CIN 2+ for the period from 2008 to 2018. We will compare the prevalence of hrHPV infection by socioeconomic characteristics, including health insurance status, as well as by year of diagnosis and degree of marginalization of the municipality of residence. To examine the hrHPV infection prevalence trend over time and its variations depending on sociodemographic and obstetric-gynecologic factors and the degree of municipal marginalization, we will use multilevel logistic regression models. The first level will be individual, and the second level will be municipal.

These regression models are justified since the response variable is dichotomous and women from the same municipality share common measurable and unmeasurable characteristics which influence hrHPV infection prevalence.3434. Harvey G. Multilevel Statistical Models. London nstitute Educ Multilevel Model Proj. London: Institute of Education, Multilevel Models, 1999. The specification of the model will be the following for hrHPV, and we will proceed accordingly for the CIN2+ model:

LogitPrYij=1=βo+αj+β´1X´1ij+β´2X´2ji+β3X3ji+β4Tij+ϵij

Where:

Yij is an indicator for the presence of hrHPV

αj is the random intercept at municipal level

X’1ij is the vector of sociodemographic characteristics of women

X’2ij is the vector of obstetric-gynocologic characteristics of women

X’3ij is the degree of municipal marginalization

Tij is the year of diagnosis

єij, is the error term

Odds ratios (ORs) obtained with this model will be the estimators of adjusted prevalence ratios of hrHPV. Furthermore, in order to explore heterogeneous effects, we will evaluate the interaction between year of diagnosis and sociodemographic and obstetric-gynecologic factors of women as well as municipal marginalization. We assume that woman whose information is not available in the SICAM database have on average the same characteristics than women included in our analyses. We also assume that the mechanism by which we obtained the information is similar to a random sampling of women to whom the program is targeted.

We will evaluate the benefit of introducing the hrHPV testing in NCSP by means of testing the relative sensitivity of hrHPV testing with cytology triage compared with conventional cytology, using a procedure based on those described by Pepe.3535. Pepe MS. Oxford Statistical Science. New York: Oxford University Press Inc, 2003. Briefly, we will estimate the ratio between the detection rates of CIN 2+ by hrHPV with cytology triage and conventional cytology. For the hrHPV with cytology triage strategy, we will define the detection rate of CIN 2+ as cases confirmed by biopsy and detected by hrHPV with cytology triage divided by all women with positive cytology and diagnosed with HPV who underwent colposcopy for diagnosis confirmation. For the conventional cytology strategy, we will define the detection rate of CIN 2+ as cases confirmed by biopsy and detected by cytology only divided by all women with positive cytology who underwent colposcopy for diagnosis confirmation.

We will employ a modified multilevel Poisson regression model to estimate relative sensitivity adjusting for individual and municipal characteristics of women. In the first level, we will include individual characteristics such as age, number of pregnancies and geographical zone. In the second level, we will include degrees of municipal marginalization. We will employ this model because the outcome measure will be the number of CIN 2+ confirmed cases and because women from the same municipalities tend towards similar measurable and non-measurable characteristics.3535. Pepe MS. Oxford Statistical Science. New York: Oxford University Press Inc, 2003. In this regression model, the main explanatory variable will be the following screening strategy:

Lnλij=βo+αj+γ1Sij+γ2Mij+δSijMij+β1X1ij+β´2X´2ij+β3X3ij+β4Tij+ϵij

Where:

λij is the detection rate of CIN2+

αj is the random intercept at municipal level

Sij is an indicator for the strategy of detection

Mij is an indicator of effect modifier

X’1ij is the vector of sociodemographic characteristics of women

X’2ij is the vector of obstetric-gynecologic characteristics of women

X3ij is the degree of municipal marginalization

Tij is the year of diagnosis

єij, is the error term

The ratios of incidence rate obtained with this model will be estimators of the adjusted relative sensibility of hrHPV and cytology as compared against with conventional cytology. In addition, with the purpose of evaluating if the sensitivity to detect CIN 2+ varies according to age, geographical zone, and degree of municipal marginalization, we will include interaction terms between the indicator for the strategy of detection (Xlij) and these variables.

Another approach to comparing the performance of hrHPV with cytology triage and conventional cytology will be to select the analytical sample by matching. In this approach, we will obtain a set of matches for each women who participated in the hrHPV with cytology triage strategy from the pool of women who underwent conventional cytology only. We will select the matches based on propensity score matching using age, date of sample collection, geographical zone and degree of municipal marginalization as variables. As a balanced sample is expected within this approach, we will not adjust for covariates in regression models.

Partial verification is a variety of selection bias that could occur in screening test studies due to only positives cases being verified by gold standard (biopsy).3636. Whiting P, Rutjes AWS, Reitsma JB, Glas AS, Bossuyt PMM, Kleijnen J. Sources of Variation and Bias in Studies of Diagnostic Accuracy. Ann Intern Med. 2004;140(3):189. https://doi.org/10.7326/0003-4819-140-3-200402030-00010
https://doi.org/10.7326/0003-4819-140-3-...
Negative cases will not have more invasive procedures for verification of their disease status because this may be inefficient and unethical.3737. Delgado-Rodríguez M, Llorca J. Bias. J Epidemiol Community Health. 2004;58(8):635-41. https://doi.org/10.1136/jech.2003.008466
https://doi.org/10.1136/jech.2003.008466...
The negative test results are differentiated by follow-up in the NCSP.3838. Lijmer JG. Empirical Evidence of Design-Related Bias. JAMA. 1999;282(11):1061-6. https://doi.org/10.1001/jama.282.11.1061
https://doi.org/10.1001/jama.282.11.1061...
,3939. Schmidt RL, Walker BS, Cohen MB. Verification and classification bias interactions in diagnostic test accuracy studies for fine-needle aspiration biopsy. Cancer Cytopathol. 2015;123(3):193-201. https://doi.org/10.1002/cncy.21503
https://doi.org/10.1002/cncy.21503...
It is also possible that the rate of completed colposcopy referrals may differ between screening strategies. We will use the method proposed by Begg and Greenes to correct the partial verification bias.4040. Begg CB, Greenes RA. Assessment of diagnostic tests when disease verification is subject to selection bias. Biometrics. 1983;39:207-15. https://doi.org/10.2307/2530820
https://doi.org/10.2307/2530820...

Ethics and biosafety considerations

We will use the information of women recorded in the SICAM database. It is considered that this is minimal risk research, therefore the use of informed consent is not required to access the information. Biological samples will not be obtained or processed, therefore this protocol does not entail biological risk. The data will be accessed and analyzed only by those responsible for this research. The data used in this analysis will not be traceable to women’s personal information as recorded in SICAM. All procedures used in this protocol will be evaluated by the Institutional Review Board of the National Institute of Public Health.

Conclusion

In summary, we expect that the results of this research project will generate scientific evidence on the effectiveness of the introduction of hrHPV screening in a population-based setting under real-world deployment conditions. This information will also provide evidence about NCSP function and whether its strategies should be improved or removed. Authorities of health programs may use these results to gain a comprehensive perspective of the program and its information system in regards to of the benefit of introducing hrHPV testing in the framework of the NCSP.

Acknowledgements

We would like to acknowledge the National Supercomputer Laboratory of Southeast Mexico (LNS), which belongs to the Consejo Nacional de Ciencia y Tecnología national laboratories registry, for the computer resources provided and their support and technical assistance by means of project number 201801037N.

References

  • 1
    Antoni S, Soerjomataram I, Møller B, Bray F, Ferlay J. An assessment of GLOBOCAN methods for deriving national estimates of cancer incidence. Bull World Health Organ. 2016;94(3):174-84. https://doi.org/10.2471/BLT.15.164384
    » https://doi.org/10.2471/BLT.15.164384
  • 2
    Almonte M, Murillo R, Sánchez GI, Jerónimo J, Salmerón J, Ferrecio C, et al Nuevos paradigmas y desafíos en la prevención y control del cáncer de cuello uterino en América Latina. Salud Publica Mex. 2010;52(6):544-59. https://doi.org/10.1590/S0036-36342010000600010
    » https://doi.org/10.1590/S0036-36342010000600010
  • 3
    Gómez-Dantés H, Lamadrid-Figueroa H, Cahuana-Hurtado L, Silverman-Retana O, Montero P, González-Robledo MC, et al The burden of cancer in Mexico, 1990-2013. Salud Publica Mex . 2016;58(2):118-31. https://doi.org/10.21149/spm.v58i2.7780
    » https://doi.org/10.21149/spm.v58i2.7780
  • 4
    Palacio-Mejía LS, Lazcano-Ponce E, Allen-Leigh B, Hernández-Ávila M. Diferencias regionales en la mortalidad por cáncer de mama y cérvix en México entre 1979 y 2006. Salud Publica Mex . 2009;51:s208-19. https://doi.org/10.1590/S0036-36342009000800011
    » https://doi.org/10.1590/S0036-36342009000800011
  • 5
    Instituto Nacional de Salud Pública. Evaluación del programa de Prevención y Control del Cáncer Cervical en México, 2008-2011. Problemas y Recomendaciones Cuernavaca: INSP, 2011 [cited 2017, Nov 14]. Available from: https://www.insp.mx/images/stories/Centros/cisp/Docs/120412_cneg.pdf
    » https://www.insp.mx/images/stories/Centros/cisp/Docs/120412_cneg.pdf
  • 6
    Bray F, Piñeros M. Cancer patterns, trends and projections in latin america and the caribbean: A global context. Salud Publica Mex . 2016;58(2):104-17. https://doi.org/10.21149/spm.v58i2.7779
    » https://doi.org/10.21149/spm.v58i2.7779
  • 7
    Torres-Sánchez LE, Rojas-Martínez R, Escamilla-Núñez C, Vara-Salazar ED, Lazcano-Ponce E. Tendencias en la mortalidad por cáncer en México de 1980 a 2011. Salud Publica Mex . 2014;56(5):473-91. https://doi.org/10.21149/spm.v56i5.7373
    » https://doi.org/10.21149/spm.v56i5.7373
  • 8
    Lazcano-Ponce E, Palacio-Mejia LS, Allen-Leigh B, Yunes-Diaz E, Alonso P, Schiavon R, Hernandez-Avila M. Decreasing cervical cancer mortality in Mexico: effect of Papanicolaou coverage, birthrate, and the importance of diagnostic validity of cytology. Cancer Epidemiol Biomarkers Prev. 2008;17(10):2808-17. https://doi.org/10.1158/1055-9965.EPI-07-2659
    » https://doi.org/10.1158/1055-9965.EPI-07-2659
  • 9
    Torres-Mejía G, Ortega-Olvera C, Ángeles-Llerenas A, Villalobos-Hernández AL, Salmerón-Castro J, Lazcano-Ponce E, Hernández-Ávila M. Patrones de utilización de programas de prevención y diagnóstico temprano de cáncer en la mujer. Salud Publica Mex . 2013;55(Suppl 2):241-8. https://doi.org/10.21149/spm.v55s2.5121
    » https://doi.org/10.21149/spm.v55s2.5121
  • 10
    Ronco G, Zappa M, Franceschi S, Tunesi S, Caprioglio A, Confortini M. Impact of variations in triage cytology interpretation on human papillomavirus-based cervical screening and implications for screening algorithms. Eur J Cancer. 2016;68:148-55. https://doi.org/10.1016/j.ejca.2016.09.008
    » https://doi.org/10.1016/j.ejca.2016.09.008
  • 11
    Schiffman M, Wentzensen N, Wacholder S, Kinney W, Gage JC, Castle PE. Human papillomavirus testing in the prevention of cervical cancer. J Natl Cancer Inst. 2011;103(5):368-83. https://doi.org/10.1093/jnci/djq562
    » https://doi.org/10.1093/jnci/djq562
  • 12
    Lazcano-Ponce E, Lörincz AT, Salmerón J, Fernández I, Cruz A, Hernández P, et al A pilot study of HPV DNA and cytology testing in 50,159 women in the routine Mexican social security program. Cancer Causes Control. 2010;21(10):1693-700. https://doi.org/10.1007/s10552-010-9598-2
    » https://doi.org/10.1007/s10552-010-9598-2
  • 13
    Flores YN, Bishai DM, Lorincz A, Shah KV, Lazcano-Ponce E, Hernández M, et al HPV testing for cervical cancer screening appears more cost-effective than Papanicolau cytology in Mexico. Cancer Causes Control . 2011;22(2):261-72. https://doi.org/10.1007/s10552-010-9694-3
    » https://doi.org/10.1007/s10552-010-9694-3
  • 14
    Ronco G, Dillner J, Elfstrom M, Tunesi S, Snijders P, Arbyn M. Efficacy of HPV-based screening for prevention of invasive cervical cancer: follow-up of four European randomised controlled trials. Lancet. 2013;383(13):524-32. https://doi.org/10.1016/S0140-6736(13)62218-7
    » https://doi.org/10.1016/S0140-6736(13)62218-7
  • 15
    Leinonen MK, Nieminen P, Lönnberg S, Malila N, Hakama M, Pokhrel A, et al Detection rates of precancerous and cancerous cervical lesions within one screening round of primary human papillomavirus DNA testing: Prospective randomised trial in Finland. BMJ. 2012;345(7886):1-11. https://doi.org/10.1136/bmj.e7789
    » https://doi.org/10.1136/bmj.e7789
  • 16
    Franco EL, Duarte-Franco E, Ferenczy A. Cervical cancer: epidemiology, prevention and the role of human papillomavirus infection. Can Med Assoc J. 2001:1017-25.
  • 17
    Franco EL. A New generation of studies of human papillomavirus DNA testing in cervical cancer screening. JNCIJ Natl Cancer Inst . 2009;101(23):1600-01. https://doi.org/10.1093/jnci/djp392
    » https://doi.org/10.1093/jnci/djp392
  • 18
    Reynoso-Noverón N, Meneses-García A, Erazo-Valle A, Escudero-de los Ríos P, Kuri-Morales PA, Mohar-Betancourt A. Challenges in the development and implementation of the National Comprehensive Cancer Control Program in Mexico. Salud Publica Mex . 2016;58(2):325-33. https://doi.org/10.21149/spm.v58i2.7804
    » https://doi.org/10.21149/spm.v58i2.7804
  • 19
    Guadarrama F. Prevención y Control del Cáncer de la Mujer. Ciudad de México: Centro Nacional de Equidad de Género y Salud Reproductiva, 2013 [cited 2017, Nov 15]. Available from: http://cnegsr.salud.gob.mx/contenidos/descargas/cama/PrevencionyControldelCancerdelaMujer_2013_2018.pdf
    » http://cnegsr.salud.gob.mx/contenidos/descargas/cama/PrevencionyControldelCancerdelaMujer_2013_2018.pdf
  • 20
    Secretaría de Salud. Modificación a la Norma Oficial Mexicana NOM-014-SSA2-1994, para la Prevención Tratamiento y Control, del Cáncer del Cuello del Útero y Mamario en la atención primaria, para quedar como, NOM-014-SSA2-1994, Para la Prevención, Detección, Diagnóstico, Trata. México: DOF, 1993 [cited 2017, Nov 14]. Available from: http://www.salud.gob.mx/unidades/cdi/nom/m014ssa24.html
    » http://www.salud.gob.mx/unidades/cdi/nom/m014ssa24.html
  • 21
    Comisión Nacional contra las Adicciones. Programa Sectorial de Salud. México: Conadic [cited 2018, Jun 15]. Available from: http://www.conadic.salud.gob.mx/pdfs/sectorial_salud.pdf
    » http://www.conadic.salud.gob.mx/pdfs/sectorial_salud.pdf
  • 22
    Bärnighausen T, Røttingen JA, Rockers P, Shemilt I, Tugwell P. Quasi-experimental study designs series-paper 1: introduction: two historical lineages. J Clin Epidemiol. 2017;89:4-11. https://doi.org/10.1016/j.jclinepi.2017.02.020
    » https://doi.org/10.1016/j.jclinepi.2017.02.020
  • 23
    Geldsetzer P, Fawzi W. Quasi-experimental study designs series-paper 2: complementary approaches to advancing global health knowledge. J Clin Epidemiol . 2017;89:12-6. https://doi.org/10.1016/j.jclinepi.2017.03.015
    » https://doi.org/10.1016/j.jclinepi.2017.03.015
  • 24
    Frenk J, Gómez-Dantés O. Quasi-experimental study designs series-paper 3: systematic generation of evidence through public policy evaluation. J Clin Epidemiol . 2017;89:17-20. https://doi.org/10.1016/j.jclinepi.2017.03.013
    » https://doi.org/10.1016/j.jclinepi.2017.03.013
  • 25
    Bärnighausen T, Tugwell P, Røttingen JA, Shemilt I, Rockers P, Geldsetzer P, et al Quasi-experimental study designs series-paper 4: uses and value. J Clin Epidemiol . 2017;89:21-29. https://doi.org/10.1016/j.jclinepi.2017.03.012
    » https://doi.org/10.1016/j.jclinepi.2017.03.012
  • 26
    Torres-Ibarra L, Lazcano-Ponce E, Franco EL, Shemilt I, Rockers P, Geldsetzer P, et al Triage strategies in cervical cancer detection in Mexico: Methods of the FRIDA study. Salud Publica Mex . 2016;58(2):197-210. https://doi.org/10.21149/spm.v58i2.7789
    » https://doi.org/10.21149/spm.v58i2.7789
  • 27
    Salmerón J, Torres-Ibarra L, Bosch FX, Cuzick J, Lörincz A, Wheeler CM, et al HPV vaccination impact on a cervical cancer screening program: methods of the FASTER-Tlalpan Study in Mexico TT - Impacto de la vacunación de VPH en un programa de tamizaje de cáncer cervical: métodos del Estudio FASTER-Tlalpan en México. Salud Publica Mex . 2016;58(2):211-9. https://doi.org/10.21149/spm.v58i2.7790
    » https://doi.org/10.21149/spm.v58i2.7790
  • 28
    Tejada RA, Vargas KG, Benites-zapata V, Mezones-Holguín E, Bolaños-Díaz R, Hernandez AV. Human papillomavirus vaccine efficacy in the prevention of anogenital warts: systematic review and meta-analysis. Salud Publica Mex . 2017;59(1):84-94. https://doi.org/10.21149/7824
    » https://doi.org/10.21149/7824
  • 29
    Wentzensen N, Walker JL, Gold MA, Smith KM, Zuna RE, Mathews C, et al Multiple Biopsies and Detection of Cervical Cancer Precursors at Colposcopy. J Clin Oncol. 2014;33(1):83-9. https://doi.org/10.1200/JCO.2014.55.9948
    » https://doi.org/10.1200/JCO.2014.55.9948
  • 30
    James H. Nelson, Hervy E. Averette RMR. Dysplasia, carcinoma in situ, and early invasive cervical carcinoma. Cancer J Clin. 1984;34(6):306-27. https://doi.org/10.3322/canjclin.34.6.306
    » https://doi.org/10.3322/canjclin.34.6.306
  • 31
    Consejo Nacional de Población. Indices de marginación por entidad federativa y municipio. Índices De Marginación. México: Conapo, 2010:73-189.
  • 32
    Consejo Nacional de Población. Índices de Marginación. México: Conapo , 2012 [cited 2017, Nov 19]. Available from: http://www.conapo.gob.mx/es/CONAPO/Indices_de_Marginacion
    » http://www.conapo.gob.mx/es/CONAPO/Indices_de_Marginacion
  • 33
    Instituto Nacional de Estadística y Geografía. Regiones socioeconómicas de México. México: Inegi, 2000 [cited 2018, Jan 31]. Available from: http://www3.inegi.org.mx/sistemas/cntmp/itaeegranjero.aspx
    » http://www3.inegi.org.mx/sistemas/cntmp/itaeegranjero.aspx
  • 34
    Harvey G. Multilevel Statistical Models. London nstitute Educ Multilevel Model Proj. London: Institute of Education, Multilevel Models, 1999.
  • 35
    Pepe MS. Oxford Statistical Science. New York: Oxford University Press Inc, 2003.
  • 36
    Whiting P, Rutjes AWS, Reitsma JB, Glas AS, Bossuyt PMM, Kleijnen J. Sources of Variation and Bias in Studies of Diagnostic Accuracy. Ann Intern Med. 2004;140(3):189. https://doi.org/10.7326/0003-4819-140-3-200402030-00010
    » https://doi.org/10.7326/0003-4819-140-3-200402030-00010
  • 37
    Delgado-Rodríguez M, Llorca J. Bias. J Epidemiol Community Health. 2004;58(8):635-41. https://doi.org/10.1136/jech.2003.008466
    » https://doi.org/10.1136/jech.2003.008466
  • 38
    Lijmer JG. Empirical Evidence of Design-Related Bias. JAMA. 1999;282(11):1061-6. https://doi.org/10.1001/jama.282.11.1061
    » https://doi.org/10.1001/jama.282.11.1061
  • 39
    Schmidt RL, Walker BS, Cohen MB. Verification and classification bias interactions in diagnostic test accuracy studies for fine-needle aspiration biopsy. Cancer Cytopathol. 2015;123(3):193-201. https://doi.org/10.1002/cncy.21503
    » https://doi.org/10.1002/cncy.21503
  • 40
    Begg CB, Greenes RA. Assessment of diagnostic tests when disease verification is subject to selection bias. Biometrics. 1983;39:207-15. https://doi.org/10.2307/2530820
    » https://doi.org/10.2307/2530820

Publication Dates

  • Publication in this collection
    26 Aug 2019
  • Date of issue
    Nov-Dec 2018

History

  • Received
    11 July 2018
  • Accepted
    26 Oct 2018
Instituto Nacional de Salud Pública Cuernavaca - Morelos - Mexico
E-mail: spm@insp3.insp.mx