Standardisation of procedures for performing cellular functional assays across laboratories participating in multicentre clinical trials is key for generating comparable and reliable data.
This article describes the performance of accredited laboratories in Africa and Europe on testing done in support of clinical trials.
For enzyme-linked immunospot assay (ELISpot) proficiency, characterised peripheral blood mononuclear cells (PBMCs) obtained from 48 HIV-negative blood donors in Johannesburg, South Africa, were sent to participating laboratories between February 2010 and February 2014. The PBMCs were tested for responses against cytomegalovirus, Epstein Barr and influenza peptide pools in a total of 1751 assays. In a separate study, a total of 1297 PBMC samples isolated from healthy HIV-negative participants in clinical trials of two prophylactic HIV vaccine candidates in Kenya, Uganda, Rwanda and Zambia were analysed for cell viability, cell yield and cell recovery from frozen PBMCs.
Most (99%) of the 1751 ELISpot proficiency assays had data within acceptable ranges with low responses to mock stimuli. No significant statistical difference were observed in ELISpot responses at the five laboratories actively conducting immunological analyses. Of the 1297 clinical trial PBMCs processed, 94% had cell viability above 90% and 96% had cell yield above 0.7 million per mL of blood in freshly isolated cells. All parameters were within the predefined acceptance criteria.
We demonstrate that multiple laboratories can generate reliable, accurate and comparable data by using standardised procedures, having regular training, having regular equipment maintenance and using centrally sourced reagents.
Clinical trials related to HIV, malaria and tuberculosis have been conducted in Africa for many years.
The International AIDS Vaccine Initiative (IAVI) has partnered with local institutions and established Good Clinical Laboratory Practice (GCLP)-compliant laboratories across Africa, Europe and India to conduct safety and immunogenicity assessments in support of clinical trials of HIV vaccine candidates.
Cellular functional assays have been used for assessing the immune response to many vaccines.
It is worth noting that most of the laboratories performing end-point IFN-
The study protocol was approved by the ethics committees of Kenyatta National Hospital, University of Nairobi, Kenya (reference numbers P81/3/2010 & P298/7/2011), the Uganda Virus Research Institute, Entebbe, Uganda (GC/127/10/08/31 & GC/127/11/09/12), Projet San Francisco (PSF), Kigali, Rwanda (006/RNEC/2011), Zambia EMORY HIV Research Project, University of Zambia (008-03-10), Emory University (REC-270606-013) and South African National Blood Service (IRB00041163) and was reviewed by the responsible regulatory authorities in each country.
Each participant provided written informed consent before undertaking any study procedures. At the South African National Blood Service, blood donors signed informed consent after the procedure was explained to them. Blood collected from these donors was not labelled with donor names, but blood bag identifiers. The blood bag was further labelled at clinical laboratory services (CLS) with a project number linked to the blood bag on the laboratory information management system. Each participant in the HIV-1 clinical trial study provided written informed consent before blood samples were collected. The samples were de-identified and labelled with a study number.
Six of the seven participating laboratories were IAVI-supported laboratories: (1) Kenya AIDS Vaccine Initiative-Institute of Clinical Research (KAVI-ICR) University of Nairobi, Nairobi, Kenya, (2) IAVI Human Immunology Laboratory, Imperial College London, United Kingdom, (3) Uganda Virus Research Institute (UVRI), Entebbe, Uganda, (4) Kenya Medical Research Institute Centre for Geographical Medicine Research Coast (KEMRI-CGMRC), Kilifi, Kenya, (5) Zambia EMORY HIV Research Project (ZEHRP), Lusaka, Zambia, and (6) Projet San Francisco (PSF), Kigali, Rwanda. Clinical laboratory services, Witwatersrand University, Johannesburg, South Africa, is the only laboratory not supported by IAVI but contracted to coordinate the sourcing, testing and shipping of peripheral blood mononuclear cells (PBMCs) for ELISpot proficiency testing. Clinical laboratory services is accredited for both ISO 15189 and GCLP and followed the same operating procedures as the IAVI-sponsored laboratories. Clinical laboratory services also participated in ELISpot proficiency testing and is included in this report.
Comprehensive training programmes, calibrated and maintained equipment and quality control measures were integral in establishing IAVI’s sponsored laboratories (
Summary of the process of establishing a clinical trial laboratory under good clinical laboratory practice guidelines.
Good clinical laboratory practice guideline | Process |
---|---|
Development and qualification of collaborating laboratories | Assessment of laboratories’ needs, development of required infrastructure, transfer and qualifications of assays |
Qualification and validation of equipment and assays | Develop qualification and validation plans, generate data and review compared against predefined criteria |
Equipment service and maintenance | Develop calibration plans and document calibration and maintenance of all critical equipment |
Development of essential documents | Develop and review standard operating procedures and other supporting documents describing safety and immunogenicity assessments |
Reagent and consumable procurement | Critical reagents are purchased from approved vendors according to an approved standardised specification |
External quality assurance programme | Develop quality assessment programme covering all safety testing parameters, processing, storage and shipment of PBMCs and the ELISpot assay |
Training programme | Good clinical laboratory practice and technical trainings to ensure compliance with international standards for conducting clinical trials |
Evaluation and accreditation | Good clinical laboratory practice compliance and acceptable technical performance monitoring by a comprehensive audit programme |
On-going technology transfer | Transfer of new assays to African clinical trial laboratories and the establishment of separate research programmes. |
PBMCs, peripheral blood mononuclear cells; ELISpot, enzyme-linked immunospot assay.
To minimise potential failures in the IAVI laboratories, quality control systems and operating procedures were put in place and corrective actions were instituted whenever a laboratory encountered a technical or assay failure to prevent future re-occurrence. Two laboratories (HIL and CLS) were designated to provide laboratory support and quality control management as these two laboratories were based in ideal locations to support a global clinical trial programme. Both locations are major international hubs in Europe and Africa, with direct flights to and from the IAVI-supported laboratories, thereby reducing time and cost to transport samples as well as reduce damage risk to samples while in transit.
International AIDS Vaccine Initiative GCLP laboratories were enrolled in an IFN-
Peripheral blood mononuclear cells used in ELISpot proficiency testing were obtained from the buffy coat (South African National Blood Service, Johannesburg, South Africa) by Ficoll-Paque gradient centrifugation. Blood donors were screened for HIV, hepatitis B, and syphilis. Briefly, whole blood was diluted with sterile phosphate-buffered saline (PBS, Sigma-Aldrich. St. Louis, Missouri, United States) at a ratio of 1:1 and layered gently over 20 mL Ficoll density gradient (Ficoll-Hypaque PREMIUM; GE Healthcare, Uppsala, Sweden) at a ratio of 2:1 in a 50 mL Falcon tube (Greiner Bio-One, Stonehouse, United Kingdom). The tubes were centrifuged at 750 ×
Peripheral blood mononuclear cells obtained from clinical trial participants was isolated from heparinised blood by density gradient centrifugation using Histopaque 1077 (H8889, Sigma). Briefly, 20 mL of blood was layered onto 20 mL of Histopaque in a 50 mL tube (Falcon 357522, Sarstedt, Nümbrecht, Germany) using a sterile serological pipette (Falcon, Sarstedt). The blood was then centrifuged at 400 ×
In a separate study, PBMC from clinical trial samples were obtained from heparinised blood from a healthy HIV-negative placebo and vaccine recipients participating in clinical trials of two prophylactic HIV vaccine candidates at KAVI-ICR, UVRI-IAVI, PSF and ZEHRP.
Cryopreserved PBMCs were shipped to the participating laboratories as non-infectious human specimens –biological substances, category B and UN337, packed in compliance with the International Air Transport Association (IATA) packing instruction 650. Before shipping, CLS notified the receiving laboratories of the PBMC proficiency panels’ shipping itinerary so that they can be ready to appropriately store the samples upon receipt. The proficiency PBMCs were shipped on a temperature-controlled dry shipper (MVE Jencons, United Kingdom). Dry shippers were calibrated for 7 days to ensure they are in good condition for shipment of PBMCs. Briefly, on day 1, empty dry shippers were weighed and filled with LN to the brim and left overnight to adsorb. The next day, excess LN was decanted; the weight and temperature of the shipper were recorded. For the next 5 days and at the same time as the second day, the weight and temperature of the dry shipper were measured and recorded to determine the weight and temperature loss. For each dry shipper to pass calibration, its average weight and temperature loss in 24 h over the 5 days should be no more than 0.66kg (manufactureres specification is 0.6kg + 10%) and less than –190 °C. A day before shipment, the calibrated dry shippers were filled with LN and left overnight. The next day, the excess LN was decanted after which the weight and temperature of the shipper were recorded. The PBMCs were loaded onto a pre-cooled canister and placed into the shipper. The shippers were fitted with temperature loggers which were activated only after the samples were loaded to monitor the temperature of PBMCs while in transit. Once the paperwork was completed the dry shippers were collected by the IATA certified shippers (World Courier). Upon arrival at the laboratory, the loggers were removed and temperature data was downloaded and recorded. Quality checks were then done for the PBMCs against the shipment manifest that accompanied the samples and samples cryopreserved in the LN tank until use.
This plate reader system is regularly maintained by use of a master lot plate supplied alongside the reader system, which contains artificial spots designed to test the performance of the reader system. As part of the internal quality control programme, each laboratory read this control plate once a week to assess the performance of the ELISpot reader using predefined spot parameters.
The developed plates were imaged on the AID ELISpot reader system. The ELISpot data were expressed as the numbers of spot-forming cells (SFC) per million PBMC (Supplementary Document 1 Figure 2). The acceptance criteria for each assay are: (1) the average SFC in the mock wells (peptide and PHA free) must be less than 50 SFC per 106 PBMC; (2) the average SFC in the negative antigen control wells (cell free) must be less than 5 SFC per well; (3) the average SFC in the assay positive control wells (peptide free + PHA) must be more than 50 SFC per 106 PBMC in the positive control PHA. The plates were read and raw SFC data were submitted to HIL for evaluation by a senior scientist and results shared on the access restricted CLS website.
This study aimed to compare the assay results obtained from the analysis of the same (frozen) PBMC samples provided to seven laboratories. We analysed the inter-lab and inter-operator variability and investigated cell recovery, viability and processing time. Outcome measures include the recovery and viability rates of frozen PBMCs, and ELISpot counts for mock, cytomegalovirus and CEF stimuli. For uniformity of data, cell recovery data after thawing and resting overnight from one clinical trial with PBMCs frozen at 15 million cells per mL/vial were normalised to 10 m cells per mL.
For each sample, four replicate ELISpot plate wells per peptide pool were assayed and the arithmetic mean was used for analysis. Results based on fewer than 4 replicate counts were assumed to be less accurate and excluded from the analysis. A total of 1751 assays were performed of which 50 were excluded (i.e. about 2.9%). For peptide pool repeated measures, the Poisson regression model was fit to background-subtracted (except mock) ELISpot counts, with counts from the same volunteer assumed to be correlated. The resulting least-squares parameter estimates are presented together with their 95% confidence intervals adjusted for multiple comparisons using the Bonferroni method. Each model included volunteer, laboratory and month as covariates. Pair-wise comparisons between laboratories are shown as the ratio of the least-squares estimates of the mean count with corresponding adjusted (Bonferroni) 95% confidence intervals. Statistical significance was defined as a 95% confidence interval for the ratio that excludes unity (i.e. entirely above or below the value 1).
Almost all (1733/1751; 99%) of the ELISpot proficiency assays performed had data within acceptable ranges with low responses to mock stimuli within the acceptance criteria of less than 50 SFC per million cells across the seven laboratories over time (
Distribution of ELISpot responses across laboratories in Kenya, Uganda, Rwanda, Zambia, South Africa and the United Kingdom, 2010–2014. Responses against mock, cytomegalovirus and cytomegalovirus, Epstein-Barr virus, and influenza virus stimuli from a panel of six peripheral blood mononuclear cells tested over 6 months, (a) first 24 months and (b) second 24 months. Box plots represent the quartiles, horizontal line the median and whiskers the maximum and minimum values. Each point represents average spot-forming cells per 106 peripheral blood mononuclear cells from replicates per donor at each laboratory. The laboratories are color-coded as follows: Kenya AIDS Vaccine Initiative-Institute of Clinical Research (red); Uganda Virus Research Institute (blue); Projet San Francisco (green); Zambia EMORY HIV Research Project (purple); Kenya Medical Research Institute Centre for Geographical Medicine Research Coast (yellow); Clinical Laboratory Services (cyan); Human Immunology Laboratory (black).
When we compared the ELISpot responses against CEF peptide pools across laboratories, KEMRI-CGMRC had significantly higher counts than other laboratories (
Comparison of ELISpot responses across laboratories in Kenya, Uganda, Rwanda, Zambia, South Africa and the United Kingdom, 2010–2014. The graphs show which site pairs are significantly different (blue lines) and which are not (red lines). For each comparison, a line segment, centred at the least-squares-means in the pair, is drawn. The segment length corresponds to the projected width of a confidence interval for the least-squares mean difference. Each line corresponds to the pair of labs with reference lines that cross at the midpoint. Shown here are the pair-wise least-squares means and their statistical significance, on a natural log scale, for mock, cytomegalovirus, Epstein-Barr virus, and influenza virus and cytomegalovirus stimuli. Differences for alpha = 0.05 (Bonferroni Adjustment); Red line denotes not significant while blue line denotes significant. (a) Mock, (b) cytomegalovirus, Epstein-Barr virus, and influenza virus and (c) Cytomegalovirus.
The performance of three operators from KAVI-ICR in ELISpot testing during the study period was analysed. PBMCs from 12 volunteers were analysed by the three operators on a rotational basis with each operator conducting the same set of samples at monthly time points. ELISpot counts were obtained for mock and background-subtracted cytomegalovirus and CEF responses and the data analysed using the repeated measures Poisson regression model and comparison-adjusted against the sample, operator and month. The geometric mean ELISpot counts for mock were 9–12, 368–393 for CEF and 538–598 for cytomegalovirus stimuli (Supplementary Document 1 Table 4). We found no significant difference in ELISpot performance between the three operators (
Comparison of inter-operator ELISpot responses from three operators at Kenya AIDS Vaccine Initiative-Institute of Clinical Research, Kenya, 2010–2014. The graphs show which operators are significantly different (blue lines) and which are not (red lines). Shown here are the pair-wise least-squares means and their statistical significance, on a natural log scale, for mock, cytomegalovirus, Epstein-Barr virus, and influenza virus and cytomegalovirus stimuli. For each comparison, a line segment, centred at the least-squares means in the pair, is drawn. The length of the segment corresponds to the projected width of a confidence interval for the least-squares mean difference. Segments that fail to cross the 45° reference line correspond to significant least-squares mean differences. None of the pairs of operators is significantly different (all lines cross the 45° reference line). Differences for alpha = 0.05 (Bonferroni Adjustment); Red line denotes not significant while blue line denotes significant. (a) Mock, (b) cytomegalovirus, Epstein-Barr virus, and influenza virus and (c) Cytomegalovirus.
A total of 1297 PBMCs isolated from clinical trial samples at the four aforementioned laboratories supporting two IAVI-sponsored HIV clinical trials were analysed for cell viability, recovery and cell yield per mL of blood. Of the 1297 PBMCs processed, 1220 (94%) freshly isolated PBMCs had viability above 90% with a median of 95% (range 81% – 100%) and those with viability below 90% had a median of 88% (range 81% – 90%) (
Performance of laboratories in peripheral blood mononuclear cell (PBMC) processing, Kenya, Uganda, Rwanda, Zambia, 2010–2014. (a) The percentage cell viability of freshly isolated PBMC, (b) cell yield per mL of blood, (c) percentage cell viability from frozen PBMCs, (d) cell recovery of frozen PBMC (PBMCs were cryopreserved at a final concentration of 10–15 m cells per mL; here data were normalised to 10 m cells), and (e) duration of PBMC processing (in hours). Each dot represents a sample and the horizontal line represents the median with interquartile range. The long horizontal line shows the acceptance cut-off.
The duration of PBMC processing from blood draw to cryopreservation has been shown to affect the integrity of cells.
We compared data generated in multiple laboratories over time to assess their performance in the processing of PBMC samples and ELISpot testing. Some of these laboratories supported clinical trials of HIV prophylactic vaccine candidates and to standardise their cell functionality assays and harmonise procedures to achieve reliable and accurate data, they were enrolled in an ELISpot proficiency scheme as part of external quality assurance. All laboratories performed well in ELISpot proficiency testing with data comparable across laboratories; however, there were a few sporadic outliers in the data. These outliers were expected considering the large number of data points analysed: sporadic outliers would be expected even from experienced and competent laboratories. In this study, we saw comparable ELISpot data from five out of seven laboratories. These five laboratories all supported clinical trials except one: CLS which performed ELISpot regularly. Since CLS was the laboratory responsible for sourcing and qualifying the PBMCs for the ELISpot proficiency scheme, they were expected to perform well in ELISpot testing. In the two laboratories that did not support clinical trials and routine ELISpot testing, we saw significant differences in ELISpot proficiency data compared to other laboratories. It is worth mentioning that these laboratories performed ELISpot testing quarterly and would be less experienced compared to other laboratories. To identify the root cause of this data discrepancy, corrective action was initiated at these two laboratories. The staff were retrained and assessed for competency. Also, improvement measures were put in place such as continuous training and monitoring of their performance in ELISpot testing in subsequent rounds of proficiency testing panels.
As a requirement of the GCLP programme, at any given time, there should be more than one person processing the clinical trial samples to ensure accuracy and reliability of data. Likewise, in ELISpot proficiency testing, different operators at each laboratory fully trained in the required procedures conducted ELISpot proficiency assays on a rotational basis as determined by laboratory management. The influence of the operator on the variability of results is a known factor as shown by Janetzki and colleagues;
Sample integrity is critical for achieving accurate and reliable results in clinical trials. In a multicentre trial, processes for sample processing need to be harmonised to generate comparable data. We assessed the processing of PBMCs in four of seven laboratories, focusing on five areas: sample collection, isolation of PBMC, cryopreservation, thawing of frozen PBMC, and performance in ELISpot testing of clinical trial samples. First, all laboratories were required to process samples to cryopreservation within 6 h of a blood draw. This is to ensure that PBMCs obtained are of good quality as it has been documented by Olson and colleagues that a delay in the processing of PBMC of more than 8 h may reduce cell viability and compromise cell functionality.
In assessments of PBMC isolation, our focus was on cell viability and cell yield. We found that the majority of PBMC samples isolated in all four laboratories supporting clinical trials met the predefined acceptance criteria for viability and cell yield with few outliers seen across the laboratories.
In most cases, samples obtained in a multicentre clinical trial are shipped to a central laboratory either for long-term storage or cellular functionality testing. Therefore, proper cryopreservation after PBMC isolation and shipping is critical in preserving cell integrity and functionality.
To maintain high standards and produce reliable and comparable data, these laboratories were audited regularly for GCLP compliance either by internal or external independent auditors. The audit covers areas such as operating procedure development and documentation, ELISpot and flow cytometry testing proficiency schemes and the data management system. All laboratories were audited annually by an external auditor for GCLP accreditation.
The main limitation of this study was the variation in the operators who performed ELISpot testing at the laboratories. There was a frequent turnover of personnel in most of the laboratories during the study period which made it difficult to assess the inter-operator variability within sites. To assess the operator effect on data variability, data were analysed from only one laboratory which had three operators who consistently performed ELISpot testing over the 4 years on a rotational basis. Though we saw no significant difference in data generated between these three operators, this result does not entirely represent what could be seen across all seven laboratories.
In conclusion, we have demonstrated the capabilities of multiple laboratories in Africa and Europe in processing clinical trial samples to high standards and performing cell functionality assays. Furthermore, we have shown that multiple laboratories can generate reliable, accurate and comparable data by using standardised procedures, having regular training, regular equipment maintenance and using centrally sourced reagents. These efforts and the ELISpot proficiency programme continue across the network of IAVI-sponsored laboratories supporting clinical trials. Therefore, we highly recommend the approach taken by the IAVI GCLP-accredited laboratories to produce such data to any donor, sponsor or research institution who may plan to conduct clinical trials in the region.
We wish to acknowledge the support from the University of California, San Francisco’s International Traineeships in AIDS Prevention Studies, US NIMH R25 MH064712, under which this manuscript was written. We thank University of California San Francisco faculty staff for helpful discussions, Matt Price and Kathy Crisafi of IAVI for manuscript review and laboratory technicians for experimental assistance. We also thank all IAVI clinical research centres’ principal investigators for overseeing the laboratories. The contents of this manuscript are the responsibility of the authors and do not necessarily reflect the views of United States Agency for International Development or the United States government.
The authors have declared that no competing interests exist.
R.K.L. performed the experiments and wrote the manuscript. J.I., S.O., E.T., C.N., M.S. and E.M., performed the laboratory experiments. M.J. isolated and qualified the proficiency PBMCs used in this study and also performed the laboratory experiments. N.H. and L.D. performed the statistical analysis. B.F., J.B., O.A., P.C., G.O.-M. and J.G. designed the study. J.H.C. and P.H. designed the experiments, interpreted results and critically reviewed the manuscript.
This work was funded in part by IAVI and made possible by the support of the United States Agency for International Development and other donors. The full list of IAVI donors is available at
Data sharing is not applicable to this article as no new data were created or analysed in this study.
The views and opinions expressed in this article are those of the authors and do not necessarily reflect the official policy or position of any affiliated agency of the authors.