The Potential of Artificial Intelligence in Pharmacovigilance

Posted on: Thursday 12 November 2020
Author: Manfred Hauben MD, MPH, Pfizer Inc. NYU Langone Health

This article has been prepared by Manfred Hauben MD, MPH, Pfizer Inc. NYU Langone Health.

It is provided for information and does not constitute advice or represent official FPM views or policy. 

 

How to cite:

Hauben, M (2020), ‘The Potential of Artificial Intelligence in Pharmacovigilance’, Journal of the Faculty of Pharmaceutical Medicine, 10 November 2020. Available at: https://www.fpm.org.uk/journals/the-potential-of-artificial-intelligence-in-pharmacovigilance/ (Accessed: <date>).

Pharmacovigilance (PV) is “the science and activities relating to the detection, assessment, understanding, and prevention of adverse effects and other drug-related safety problems”.1  Sometimes equated with spontaneous reporting, contemporary PV is much more expansive, seeking convergent lines of evidence from many sources spanning the drug development continuum. PV’s broad scope, high-throughput processing to achieve regulatory, scientific and clinical objectives in a low signal-noise environment, plus a growing treasure trove of data and technology, makes it a high-grade ore for mining by artificial intelligence (AI).

AI is a broad term, including replicating human cognition by machines (i.e. symbolic logic), and machine learning (ML), for which providing accurate predictions, classifications, clusters, or otherwise  useful patterns, is paramount, regardless of whether it is achieved in the same way as the human neural network.

AI can tackle large, complex, high dimensional (e.g. the number of predictors >> number of samples) data containing complex non-linear relationships, unlike standard statistical analysis which may result in an infinite number of possible, unstable and potentially overfitted solutions. This is amplified when some of the huge number of possible combinations of variables are predictive.

ML includes traditional ML (e.g. logistic regression , decision trees and support vector machines (SVM)) and deep learning, such as artificial neural networks (ANNs). A 2017 review reported that SVM and ANN are the most commonly used forms of AI in medicine.2

Recently available/exploited data span the scale from molecule to human, including chemical structure, pharmacological, genomic, metabolomic, proteomic interactomic, transcriptomic and biobank data sets, plus electronic health records (EHR), claims data, social media, internet search logs and wearable health devices, each utilized in AI with varying potential and pitfalls.

AI has demonstrated/potential utility in multiple PV domains. Lewis et al identified opportunities for AI to augment regulated pharmacovigilance systems subject to good pharmacovigilance standards of practice, including individual case safety report (ICSR) processing, signal management PV quality management systems, product labeling (e.g. quality checking and harmonization) and risk management systems.3 PV practiced in other settings, such as the clinic, present further opportunities (e.g. using machine learning to help identify hydroxychloroquine retinopathy via ocular coherence tomography).4 However, two stand-out use cases are ICSR processing and signal detection and evaluation (SD&E).

With its volume and velocity, ICSR processing can be a massive, resource-intensive endeavor, with regulatory and medical objectives, e.g. compliance with reporting mandates, medical assessment, and signal detection. A non-peer reviewed 2018 industry survey indicated that 62% of pharmaceutical companies are implementing or planning to implement AI in PV, prominently for ICSR case processing.Mockute et al focused on component “cognitive services” for ICSR processing, identifying 51 candidate decision points for AI application.6  Examples of specific ICSR case processing activities with potential or actual AI application include adverse event extraction, validity and seriousness assessment, case causality assessment, MedDRA coding, and quality checking. 3,6-9 Because spontaneous reports typically undergo intake with unstructured text, a form of AI known as natural language processing (NLP) is prominent in AI-augmented ICSR management. Note that NLP does not involve a machine reading raw text. The text is digitized, typically as vectors, after initial pre-processing, to make it machine readable.

For SD&E, ML from bioinformatics-based data can assess structure-activity relationships, and via systems biology/network graph analysis, how drug effects ripple through complex biological networks, including off-target effects, helping toxicity prediction prior to, or during drug development, but also substantiating post-marketing safety signals10-14 via biological plausibility arguments, in line with the Bradford Hill criteria published in 1965 by the English epidemiologist and statistician Sir Austin Bradford Hill. For example, activation of specific subnetworks (“modules”) within complex biological networks may be signatures of drug-induced adverse events, such as myocardial infarction.12 One exercise was even able to tease apart within-drug class safety differences.13 The real-world potential of these types of analyses will depend in part on the development of readily and widely deployable big data analysis environments for seamless workflow.

Much work in applying AI to SD&E aims for pipelining analysis of multiple data sources, such as spontaneous reports and EHRs, to condense the usually serial processes of SD and SE.15 There is keen interest in NLP for SD&E from these data, since an estimated 80% clinical data is unstructured in EHRs.17 Large institutional EHR systems, as well as distributed data networks such as the Sentinel system provide comprehensive environments for ML. 16-18

The vast amount of social media data begs the question of whether AI can mine it for real-time SD? However social media posts are heavily laden with limitations including very low-signal-to-noise ratio, colloquial terminology, emoticons, duplicate reporting (e.g. retweets), and lack of key demographics and identifiers. While NLP is able to extract posts containing product-safety pairs (“proto-Adverse Drug Events”), to date there is no cogent evidence that mining social media discovers signals that are otherwise undetectable by, or detects credible signals earlier than, legacy processes and there are currently no regulatory mandates to screen social media. However, it may add value in selected areas such as addictovigilance.19

Like drugs and devices, there are side effects, warnings and precautions for AI in PV use!  The distribution of research across peer-reviewed, grey (e.g. abstracts, presentations, pre-print servers), and commercial literature is sometimes skewed. Prospective real-world deployment is somewhat sparse. One example of a ML-based SD system routinely used in PV is vigiRank, a predictive model using twelve predictor variables related to quantity, quality and clinical content of individual reports, as well as trends in time and geographic spread.20

Technology “hype cycles” can bias judgement, compromising methodological quality and clinical judgement. A published deconstruction of pharmacovigilance hype cycle effects provides real-world examples of hype cycle effects from the published literature21 which can lead to potentially invalid conclusions e.g. about drug-drug interactions.

Most research is retrospective, and prospective performance often trails due to factors such as-distributional drift, in which training and test data do not represent current data.22-24  Therefore, prospective performance monitoring is important to detect downward performance drifts requiring model retraining, which will also have to be factored into regulation of ML-based medical devices used for diagnosing adverse drug reactions. An illustration of such distributional shifts occurred during development of a device for detection of hyperkalemia from EKGs using a smartwatch. Initially trained on 1.3 million ECGs with corresponding K+ levels from outpatient clinics, initial performance was poor with clinically serious prediction errors. The outpatient data was imbalanced, under-representing sicker patients with higher potassium levels and with ECGs taken closer to the time of serum potassium measurement.24 Using a less filtered data set (and analyzing the entire waveform and not just the T-waves) performance dramatically improved allowing submission for FDA approval. 25

Since AI, especially deep learning, emphasizes predictive accuracy over model interpretability, machines may use unreliable confounders for prediction during development. e.g. using the overrepresentation of rulers in image sets of malignant versus benign images of skin lesions, to accurately predict malignancy. 25

Quality standards for reporting machine learning studies are thus important; the Transparent Reporting of Multivariable Prediction Model for Individual Prognosis or Diagnosis (TRIPOD) statement (2015), and the ensuing TRIPOD-ML should help promote such quality standardization.26

Particularly in a highly regulated environment, concerns exist that machines will miss something important.  In key areas PV professionals achieve nearly 100% accuracy/compliance, a level difficult for a machine to match, underscoring the need to precisely define operational objectives of using AI in PV e.g. reduce the median time for human-in-the-loop during ICSR case processing.

PV professionals may wonder about their professional prospects with increasing AI in PV. A small survey found a generally positive attitude about AI in PV among PV professionals at a pharmaceutical company, e.g., that it would allow shifting human work from high volume, highly routinized work to more meaningful high-value work.27 With the rise of data science and biomedical informatics in drug safety there are also increasing educational opportunities at all levels, even up to distance graduate study programs at world class universities, enabling PV professionals to newly acquire powerful skill sets in drug safety bioinformatics that can massively enrich their career development.

References

  1. Lindquist M. The need for definitions in pharmacovigilance. Drug Saf 2007; 30 (10): 825-830.
  2. Jiang F, Jiang Y, Hui Z, Dong Y, Hao L et al. Artificial intelligence in healthcare: past, present and future. Stroke and Vascular Neurology 2017;2: doi:10.1136/svn-2017-000101.
  3. Lewis DJ, McCallum JF. Utilizing advanced technologies to augment pharmacovigilance systems” Challenges and opportunities. Ther Innov Regul Sci 2020;54: 888-899.
  4. Wright T, Peng Y, Easterbrook M. Machine learning to identify multifocal ERG deficits in patients taking hydroxychloroquine. Investigative Ophthalmology & Visual Science 2019; 60.
  5. Addressing the data challenges of pharmacovigilance. www.oracle.com/us/industries/health-sciences/address-data-challenges-pharma-wp-5018953.pdf. Accessed September 21st, 2020.
  6. Mockute R, Desai S, Perera S, Assuncao B, Danysz K et al. Artificial intelligence within pharmacovigilance: A means to identify cognitive services and the framework for their validation. Pharm Med 2019; 33: 109-120.
  7. Schmider J, Kumar K, LaForest C, Swankowski B, Naim K. Innovation in pharmacovigilance: Use of artificial intelligence in adverse event case processing. Clin Pharmacol Therap 2019; 105 (4): 954-961.
  8. Routray R, Tetarenko N, Abu-Assal C, Mockute R, Assuncao B et al. Application of artificial intelligence for pharmacovigilance case seriousness determination. Drug Saf 2020; 43:57-66.
  9. Rodrigues PP, Ferriera-Santos D, Silva A, Polonia J. Rubeiro-Vaz I. Causality assessment of adverse drug reaction reports using an expert defined Bayesian network. Artificial Intelligence In Medicine 91 (2018); 12–22.
  10. Wu Y, Wang G. Machine learning based toxicity prediction: from chemical structure prediction to transcriptome analysis. Int. J. Mol. Sci. 2018, 19 (8), 2358; doi:10.3390/ijms19082358
  11. Bauer-Mehren A, van Mulligen EM, Avilach P, Carascosa MDC, Garcia-cersa R et al. Automatic Filtering and Substantiation of Drug Safety Signals. PLoS Computational Biology 2012; 8(4): e1002457. doi:10.1371/journal.pcbi.1002457
  12. Ivanov SM, Langunin AA, Pogodin PV, Filiminov DA, Poroikov VV. Identification of drug-induced myocardial infarction-related protein targets through the prediction of drug−target Interactions and analysis of biological processes. Chem. Res. Toxicol 2014;27: 1263-81.
  13. Xie L, Li J, Xie L et al. Drug Discovery Using Chemical Systems Biology: Identification of the Protein-Ligand Binding Network To Explain the Side Effects of CETP Inhibitors. PLoS Comput Biol 5(5): e1000387. doi:10.1371/journal.pcbi.1000387.
  14. Lorberbaum T, Nasir M, Kaiser MJ, Vilar S, Hripcsak G, Tatonetti NP. Systems pharmacology augments drug safety surveillance. Clin Pharmacol Therap 2015; 97(2): 151-57.
  15. Harpaz R, Vilar S, DuMouchel W, Salmasian H, Haerian K et al. Combing signals from spontaneous reports and electronic health records for detection of adverse drug reactions. J Am Med Inform Assoc 2013; 20:413–419. doi:10.1136/amiajnl-2012-000930
  16. Kong HJ. Managing unstructured big data in health care systems. Healthc Inform Res. 2019; 25(1):1-2.
  17. Shah N, LePendu P, Bauer-Mehren A, Ghebremariam Y, Iyer SV, Marcus J et al. Proton Pump Inhibitor Usage and the Risk of Myocardial Infarction in the General Population. PLoS ONE 10(6): e0124653. doi:10.1371/journal.pone.0124653.
  18. FDA’s Sentinel Initiative. www.fda.gov/safety/fdas-sentinel-initiative. Accessed October 1st, 2020.
  19. Rees S, Mian S, Grabowski N. Using social media in safety signal management. Is it reliable? Ther Adv Drug Saf; 2018; 9(10): 591-99.
  20. Caster O, Sandberg L, Bergvall T, Watson S, Noren GN. vigiRank for statistical signal detection in pharmacovigilance: First results from prospective real‐world use. Pharmacoepidemiol Drug Saf. 2017; 26: 1006–1010
  21. Hauben M, Reynolds R, Caubel P. Deconstructing the pharmacovigilance hype cycle. Clin Therap 2018; 40(12): 1981-1990.
  22. Yu K-H, Kohane IS. Framing the challenges. BMJ Qual Saf 2019; 28:238-241.
  23. Challen R, Denny J, Pitt M, Gompels L, Edwards T, Tsaneva-Atanasova K. Artificial intelligence, bias, and safety. BMJ Qual Saf 2019; 28: 231-237.
  24. Kelly C. Karthikesalingam, Suleyman M, Corrado G, King D. Key challenges for delivering clinical impact with artificial intelligence. BMC Medicine (2019) 17:195
  25. The Skinny on Deep Learning Deep Medicine In: Topol E, Deep Medicine. New York: Basic Books, 2019: 60-64.
  26. Colins GS, Moons KGM.www.thelancet.com Reporting of artificial intelligence prediction models Vol 393 April 20, 2019;
  27. Danysz K, CicIrello S, Mingle E, Assuncao B, Tetarenko N et al. Artificial Intelligence and the Future of the Drug Safety Professional. Drug Saf 2019; 42: 491-97

More from the Autumn/Winter 2020 edition of the FPM Journal