This study evaluated a National External Quality Scheme Program for early infant diagnosis of HIV. Fourteen laboratory technicians participated and nine testing panel cycles were sent between 2011 and 2014. The response rate was 100% for the first eight panels, and the number of technicians with a test score of 100% increased during the first three panels. Based on the evaluations of the technicians, the quality of testing for early infant diagnosis of HIV improved over time in the laboratories.
Early infant diagnosis (EID) of HIV infection assures early access to antiretroviral therapy for infected children, which significantly improves survival rates and provides substantial benefits.
In settings that lack adequate infrastructure and cold-chain transportation systems to process whole blood, dried blood spot (DBS) samples offers many advantages. Dried blood spot testing has simplified HIV screening in newborns, because samples are easy to collect often via heel prick or finger stick and no cold chain transportation system is required as DBS samples are stable at room temperature.
Participation in an external quality assurance (EQA) programme has been shown to detect possible errors during testing, trace possible corrective actions and improve the quality of testing, and is thus an important tool for laboratory quality assurance.
Since 2006, Mozambique has been using PCR-based methods with DBS samples to diagnose HIV in exposed newborns. In 2011, the
The ethics committee of the
Between 2011 and 2014, all laboratory technicians performing EID for HIV using DBS were informed about the availability of proficiency testing panels provided by the National EQA Program of Mozambique and were encouraged to enroll and participate. The panels were provided free of charge and sent to each technician for testing within four weeks of receipt. To maintain confidentiality, a unique code was assigned to each technician.
The proficiency testing panels consisted of 20 blinded DBS samples a mixture of negative and positive samples. Negative DBS samples were prepared from a negative blood bag collected by the Blood Bank at
Four series were prepared, and each series consisted of 20 blinded DBS samples in a different sample arrangement. Negative and positive specimens were prepared separately, dried overnight, then encoded with a sample and serial number. Each series contained 20 DBS specimens packed into a sealable plastic storage bag stored at 2°C – 8°C until shipment. All four series were sent to each laboratory, and each laboratory technician had to test one of the series sent.
To validate the panel, each series was selected randomly and all 20 DBS samples within the series were retested using the Amplicor HIV-1 DNA PCR test, version 1.5 (Roche, Germany) according to the manufacturer´s instructions, by two different technicians. The results were crosschecked and both results had to match. Following validation, the proficiency panels were sent and made available to all the laboratories across the country performing EID diagnosis.
A carrier company shipped the proficiency panels at room temperature three times during 2011 and twice between 2012 and 2014. The technicians had to report the results within 30 days after panel reception. All technicians that did not report their results within 30 days after receiving the panels were considered to be non-responders.
Test scores were reported in percentages according to the number of correct results out of the 20 tests per cycle. Technicians were evaluated according to the concordance of their results with expected results established previously. After each panel, each technician was sent a report containing (1) the test scores (percentage of concordant results), (2) reported results, as well as expected results and possible causes in cases of discordant results. For technicians with results below 100%, corrective actions were conducted, including technical support through laboratory visits, emails or telephone calls.
Frequencies and proportions were used to describe the data. Trends between 2011 and 2014 were studied by means of graphs or tables. To study the association between the proportion of technicians scoring 100% on a panel and concordance among technicians, we use a beta-binomial regression with a logit link. We chose this regression to keep the fitted concordance within the 0 to 1 range and to respect the uncaptured geographic heterogeneity (over dispersion) of the data. Based on the Akaike information criterion and Bayesian information criterion, the proportion of technicians scoring 100% was included as linear and quadratic transformed. The quadratic terms were included to detect U-shaped relationships in the data. Stata 14 (StataCorp. 2015. Stata: Release 14. Statistical Software; StataCorp LP, College Station, Texas, United States) was used to perform all analysis.
From 2011 to 2014, nine proficiency panels were sent to each technician. The overall number of technicians that participated increased from 10 in 2011 to 12 in 2014 (
Overall number of technicians that participated in the National External Quality Assessment Program for polymerase chain reaction of HIV DNA across the nine panels, Mozambique, 2011–2014.
Panel number | Panel name | Number of technicians that participated | Response rate % |
---|---|---|---|
1 | 2011-A | 10 | 100 |
2 | 2011-B | 11 | 100 |
3 | 2011-C | 12 | 100 |
4 | 2012-A | 13 | 100 |
5 | 2012-B | 13 | 100 |
6 | 2013-A | 13 | 100 |
7 | 2013-B | 13 | 100 |
8 | 2014-A | 14 | 100 |
9 | 2014-B | 14 | 78.5 |
- | Average | 12.6 | 97.6 |
%, percentage.
For Laboratory D, 18 errors were observed, and 100% of the technicians had at least one error during the nine panels (
Error characterisation of technicians per laboratory during the nine cycles.
Laboratory | Error types |
|||||
---|---|---|---|---|---|---|
False positives | False negatives | Total invalids/indetermined | Total errors | Number of technicians enrolled by 2014 | Percentage of technicians with at least one error during the nine cycles | |
Laboratory A | 4 | 6 | 2 | 12 | 4 | 20 |
Laboratory B | 0 | 2 | 0 | 2 | 3 | 16.60 |
Laboratory C | 0 | 8 | 2 | 10 | 4 | 75 |
Laboratory D | 11 | 4 | 3 | 18 | 3 | 100 |
The number of false positives increased during the first three panels, while no false positives were observed during the last three panels (2013-B, 2014-A and 2014-B) (
Percentage of discordant results among all participating technicians. Discordant results included false positives, false negatives and invalids.
Percentage of technicians scoring 100% versus their concordance during each test panel from 2011–2014. The line is a fit of a beta-binomial regression with linear and quadratic terms of technicians scoring 100%.
During the expansion of EID testing and treatment programmes in Mozambique, it was necessary to optimise the quality testing for HIV DNA PCR. We report the successful implementation of the HIV-1 EID EQA Program in Mozambique during the period of 2011 to 2014. Our results show that the quality testing slightly improved over time after the EID EQA program was implemented. If progress is to be maintained, EID EQA program should be sustained.
The number of technicians participating increased during the first eight panels of the programme, with a response rate of 100%, suggesting a positive response and growing interest in the programme. Initially, training on EID for HIV conducted by the laboratory that provided the proficiency testing panels and the clear explanation of the objectives and benefits of the programme to its participants may have accounted for the increased participation. For each panel, technicians with discordant results had direct site supervision and indirect technical support (via email or telephone). The decrease in the response rate during the last panel was most likely related to annual leave during the period that the panel was sent. As the number of samples tested in the country for EID HIV-1 infection increased, the number of technicians participating in the proficiency testing increased over time.
Although corrective actions such as technical support, training and discussions with the laboratory manager were performed, the number of errors for Laboratory D, which accounted for most of the errors during the nine panels, did not decrease. For Laboratory C, most technicians had at least one error over the nine panels. During the retraining and visits, simplification of the procedures was observed, because of the large number of samples being tested. However, for Laboratory A, retraining a technician who was observed making errors helped to overcome some practical issues.
Corrective actions after each EQA panel and retraining of all technicians contributed to a significant reduction of errors over time. Technical assistance decreased in 2013 because of logistical problems faced by the reference laboratory. As a consequence, the number of errors during this period increased, showing the importance of technical assistance to ensure quality testing.
These results reinforce observations made by other studies that training and experience significantly affects the accuracy of proficiency panel testing over time.
One of the limitations of our study is that we were not able to follow up with the technicians during all the panels and new technicians were enrolled later in the programme, which might have contributed to the increased number of errors. No sociodemographic data, such as gender, education or how long the technician had been familiar with the procedure and number of panels tested were collected. Thus, our study could not assess the association of such factors with errors observed over the time that the programme was evaluated.
A continuous national quality assessment control programme that includes international panels is important for control of the analytical aspects of EID testing for HIV. This EQA programme ensures quality testing at both the technician and laboratory levels. As indicated by this evaluation, improvement in the quality of testing for EID of HIV for the laboratories from 2011 to 2014 was observed. Our results show that both the National EQA Program for EID of HIV testing model of evaluating technicians, as well as the laboratory, are successful in ensuring the quality of results.
The authors would like to thank all the laboratories that participated in the National Quality Scheme program for Early Infant Diagnosis of HIV. In addition, they would also like to thank the staff members of the Molecular Virology Laboratory and the National Programme of External Quality Scheme of the
The authors declare that they have no financial or personal relationships which may have inappropriately influenced them in writing this article.
None.
N.I. supported the planning and implementation of the programme and wrote the manuscript; O.A. was responsible for the statistical analysis, and revised the manuscript; A.V. and S.O.V. supported the implementation of the programme and made conceptual contributions; F.M. performed the data analysis; P.C. and N.M led the implementation of the programme and reviewed the manuscript.