Home | Register | Login | Inquiries | Alerts | Sitemap |  


Advanced Search
JKM > Volume 42(4); 2021 > Article
Kim and Han: A survey on the impact of a pharmacovigilance practice training course for future doctors of Korean medicine on their knowledge, attitudes, and perception

Abstract

Objectives

This study aimed to evaluate the impact of a training course on pharmacovigilance for future doctors of Korean medicine (DKM).

Methods

In 2020, a pharmacovigilance training course was conducted for 57 senior students of a Korean medicine (KM) college, and its impact to the students were assessed in terms of the knowledge, attitudes, and perceptions of these students at three-time points: pre-training, post-training, and 4–6 months after the end of the training.

Results

A total of 38 students completed the survey. The average score from the knowledge test increased significantly after training compared to prior to the training (5.47±2.140, 6.61±1.001, respectively, p<0.001) and was maintained until the final survey (6.61±1.220). The rate of correct answers to most of the knowledge test questions increased after the training but decreased in the final survey. In terms of attitudes, self-confidence in causality assessment (2.63±1.025, 4.58±0.826, p<0.001) and spontaneous reporting (2.08±1.050, 4.74±0.446, p<0.001) significantly increased after the training and then slightly decreased (3.92±1.171, 4.40±0.755). The perception level was high prior to the training, and this pattern was maintained throughout the study period. Students responded that pharmacovigilance education was necessary for DKM after training, and for the undergraduates of KM colleges.

Conclusions

This study shows that this pharmacovigilance training course is effective for students majoring in KM but that retraining is required at least 6 months after the initial training. Further follow-up studies are needed to ensure that students actively participate in spontaneous reporting after graduation, and continuous education should be provided to graduates.

Introduction

The thalidomide disaster of the 1960s left a lesson in the urgent need for a constant monitoring system for the safe use of drugs, as unexpected adverse drug reactions (ADRs) that have not been identified in the preclinical and clinical testing phases may appear even in the drugs that have been approved for marketing, and it led to the development of pharmacovigilance systems around the world, starting with the World Health Organization (WHO) Programme for International Drug Monitoring (PIDM).1)
The success or failure of pharmacovigilance depends on spontaneous reporting (SR), which is based on the voluntarily reported adverse events (AEs) from the clinical field, and one of the main types of sources of information for SR is healthcare professionals.2) Since different types of AEs are reported in different healthcare fields3), SR by various types of professionals is very important for obtaining a more complete understanding of ADRs. Therefore, the WHO has emphasized the necessity of pharmacovigilance and SR in all healthcare fields, including herbal and traditional medicine4).
It is difficult to get healthcare professionals, who have already established a routine in clinical practice, to actively participate in SR5). For this reason, an international consensus has been formed that the focus of pharmacovigilance education should also be on the undergraduate curriculum of the university to nurture future healthcare professionals5,6). Therefore, in 2016, the Netherlands Pharmacovigilance Centre Lareb (Lareb) which specializes in education and patient reporting in the WHO PIDM, announced an agreement on the recommendations for a pharmacovigilance curriculum for healthcare undergraduates6). Lareb suggested the key aspects and competencies of the core curriculum for future healthcare professionals and especially focused on the clinical aspects of pharmacovigilance6).
A number of studies have reported the results of conducting pharmacovigilance education for healthcare students79). In Korea, several studies 10,11) that have reviewed the curriculum for pharmacovigilance in medicine, pharmacy, and nursing colleges, and have observed the impact of the educational intervention for pharmacy students have been published. However, in the field of herbal and traditional medicine, there are relatively few reports on the performance of pharmacovigilance education or its impacts. There is one study on the effect of an educational intervention for traditional practitioners in Nigeria12), and another study on the changes in knowledge, attitudes, and self-confidence after pharmacovigilance training courses for students of Korean medicine college13). To the best of our knowledge, the latter 13) is the first study to attempt pharmacovigilance education for undergraduates majoring in herbal and traditional medicine and assess its impact. However, the limitations of this study were that it could not test whether the changes before and after education were statistically significant and that it did not investigate the long-term effects13). The Korean healthcare system is composed of two major axes: Western conventional medicine and Korean traditional medicine. Therefore, the participation and education of experts in the field of Korean medicine and conventional healthcare professionals are essential for full pharmacovigilance. Thus, we designed a study that complemented the limitations of a previous study13) and evaluated the impact of a pharmacovigilance training course for future doctors of Korean medicine in terms of knowledge, attitudes, and perception.

Methods

1. Study design

This study is the result of a survey using self-report questionnaires to investigate the impact of a training course on pharmacovigilance and SR practice for senior students of Korean medicine college. Considering that students are vulnerable research participants, the written consent procedure was omitted and an anonymous survey was conducted. Questionnaires were distributed to students prior to education, and the purpose and methods of this study were explained to them. Only students who voluntarily agreed to participate in this study were asked to fill out the questionnaire. A submission box was placed where the researcher could not see it, and students were guided to submit questionnaires autonomously. The protocol of this study was approved by the Institutional Review Board (SJIRB-Human-20-002).

2. Questionnaire design

The questionnaire was developed with reference to prior studies targeting healthcare professionals and undergraduate students in related fields7,8,11, 1315). The questionnaire was composed of questions to investigate the demographic characteristics (age and gender), experiences in reporting AEs, and the changes before and after the educational intervention of respondents. The changes were evaluated in three areas: knowledge, attitudes, and perceptions about pharmacovigilance and SR. The questions for knowledge test were used as they were in the previous study13). A systematic review by Alwhaibi et al.12) was referred to for classification of the questions on attitudes and perception. The details of questions are presented in Appendix A.
Knowledge levels were tested with 10 questions on the following topics: 1) the definition of pharmacovigilance, 2) the definition of side effects, 3) the definition of ADRs, 4) the concept of SR, 5) the existence and role of Uppsala Monitoring Centre (UMC), a collaborating center in charge of development and management of global individual case safety report database of WHO PIDM 6) the main agent of SR, 7) the subjects of SR, 8) causality assessment, 9) the nondisclosure of patient privacy in SR, and 10) the agency in charge of SR [Appendix A]. These 10 questions were repeated in the same way in three times of surveys. The knowledge test was composed of multiple-choice questions in the form of selecting one fixed answer, except for questions 6 and 7, which allowed for multiple choices.
Attitudes were assessed by three multiple-choice questions asking about confidence in causality assessment and SR to the agency in charge and intention to report AEs. Each question was designed in the form of a 5-point Likert scale (1: do not agree at all, 2: do not agree, 3: no idea, 4: agree, and 5: strongly agree). Students were asked to choose one of five options.
Perceptions were tested by four multiple-choice questions asking about 1) the importance of the role of doctors of Korean medicine in SR, 2) the duty to report AEs as doctors of Korean medicine, 3) the necessity of pharmacovigilance training courses in the curriculum for undergraduates majoring in Korean medicine, and 4) the need for regular pharmacovigilance education programs for graduates. Questions 1–3 were included in all questionnaires, and the fourth question was asked only in the final survey. All questions were designed in the form of a 5-point Likert scale in the same form as the attitude questions, and students were asked to choose only one option.
Finally, a free-response question was included to determine the feelings and thoughts of students after completing the course. The last question of the post-educational questionnaire asked students to freely describe their opinions or ideas about this curriculum or the current domestic pharmacovigilance system.

3. Study participants

The participants of this study were 57 students who took an internal medicine practice class at Sangji University’s College of Korean Medicine in the first semester of 2020. All of them were divided into 10 groups, with 5–6 students per group, and group practice was performed with one group per week from April 20 to June 30, 2020.
The practice training course was conducted in the same way as in the precious study13) as follows. On the first day, self-study guidelines were provided to students. Educational materials were distributed and the UMC’s online pharmacovigilance training course16) was administered to students for self-learning. These online and offline self-learning materials covered introduction to pharmacovigilance, the history and current status of domestic and international pharmacovigilance system, signal detection and causality assessment, statistical reasoning from ADR database analysis. In addition, 5–6 cases of actual AEs were provided to students, and they were given the assignment to self-review the cases, assess the causality, and fill out the AE case report form of the Korea Institute of Drug Safety & Risk Management (KIDS) for healthcare professionals. For this task, the author collected AEs involved in herbal and traditional medicine through academic papers, newspaper articles, and court precedents. On the second day, after checking the assignments and understanding students’ learning level in terms of the basic concept of pharmacovigilance and SR, free discussion was conducted on the assignments. On the third day, students were asked to revise the results of the causality assessment and AE case report form of each case, reflecting the results of the previous day’s discussion. Furthermore, students who had actually experienced or witnessed AEs were asked to report them to the KIDS.

4. Survey

A total of three surveys were conducted prior to education, after education, and at the final assessment. The pre-education survey was performed one week before students began the class, the post-education survey was performed within one week after the end of the class, and the final survey was performed in the last week of October 2020. Since the date of the practice training course for each group was different, the period from the end of the course to the final survey ranged from approximately 4 to 6 months.

5. Data analysis

The responses from the submitted questionnaires were entered by the author into an Excel spreadsheet and analyzed with SPSS (IBM SPSS Statistics 25). The response rate and characteristics of respondents were identified through descriptive statistics. The change in response rate at each time point was statistically tested.
Knowledge level was evaluated with a total of 10 points by assigning 1 point per question, and the distribution of the total score and the change in the average total score for each time point were compared. Considering that the average knowledge score on the post-education survey was 6.7 out of 10 points in the previous study,13) a student who scored 7 or more points was considered to have attained knowledge achievement (knowledge achiever), and the percentage of achievers at each time point was compared. The rate of correct answers for each question was checked. A pattern of change in the number of questions with a correct answer rate of more than 90% or less than 50% was observed. In the case of questions 6–7, which allowed for respondents to choose multiple options, the rate of choice per option was also investigated.
The level of attitudes and perceptions were evaluated by substituting the Likert scale as a score (from 1 to 5 out of 5 points). A case in which a respondent of selected the expected option (option number 4 or 5) for each question was regarded as positive, and the percentage of the positive respondents who chose the expected option was calculated.
Continuous variables were presented as the mean (standard deviation), and repeated-measures analysis of variance (RMANOVA) was performed to test for differences according to the survey time point. Dichotomous variables were presented as a percentage, and Cochran’s Q test was performed to confirm the difference in percentage according to the time point. When RMANOVA or Cochran’s Q test showed significant differences, post hoc comparisons using Bonferroni correction or McNemar’s test were performed. The statistical significance level was set to 0.05. However, in the Bonferroni correction, the significance level was determined according to the number of tests.
The free-response question was analyzed using the thematic analysis method.17) Reading the respondents’ answers repeatedly, we coded them and categorized the codes according to concepts. Through this process, the main themes were derived by incorporating similar answers or common concepts. If necessary, the themes were divided into parent themes and subthemes.

Results

1. Response rate and respondent characteristics

Of the 57 students who received the pharmacovigilance training course, 56, 53, and 38 students submitted the questionnaires prior to education, after education, and in the final survey, respectively (Figure 1). Therefore, the response rate was 98.2% prior to pre-education, 93.0% after education, and 66.7% in the final survey, and there was a significant difference in response rate according to the time point (p<0.001). A post hoc comparison showed a significant difference in response rate between the pre- and post-education surveys and between the pre-education and final surveys (p<0.001, either case) (Appendix A).
The age of the 38 students who completed all three surveys was distributed between 24 and 35 years old, and the average age was 26 (2.1) years old. The numbers of men and women were 25 and 13, respectively, with male-to-female ratio of approximately 7:3.
Seven students answered that they had experienced or witnessed AEs in the past, but none of them had reported these AEs to the agency in charge. The reasons for non-reporting were as follows: not being aware of the existence of an SR system (n=3), not knowing where to report AEs (n=3), not thinking that AEs need to be reported (n=3), thinking that there was a lack of clinical knowledge (n=3), and not being certain about causality (n=1).

2. Knowledge level

The average score of the knowledge test of the 38 students rose from 5.47 (2.140) points to 6.61 (1.001) points after education, and the final score was 6.61 (1.220) points. There was a significant difference in the total score of the knowledge test according to the time point (p<0.001). The post hoc comparison showed that there were significant differences between the pre- and post-education surveys (p=0.003) and the pre-education and final surveys (p=0.003). No significant difference was found between the post-education and final surveys (Table 1).
The knowledge scores of respondents ranged from 0 to 10 in the pre-education survey, and their distribution was narrowed to 4–8 points, and expanded again to 3–9 points in the final tests (Figure 2). The mode increased from 6 points (8 students) prior to education to 7 points (20 students) after education, and then decreased again to 6 points (15 students) in the final survey (Figure 2).
The number of students with knowledge achievement increased more than twofold, from 12 (31.6%) to 25 (65.9%) after education, and then decreased again to 19 (50.0%) in the final survey (Figure 3). There was a significant difference in the percentage of knowledge achievers according to time points (p<0.003). Post hoc comparisons showed that there was a significant difference in the percentage of achievers before and immediately after education (p<0.001) but no difference in terms of other comparisons (Figure 3).
In the analysis of the rate of correct answers for each question, a pattern of increasing after education compared to the baseline and then decreasing in the final test was observed for most questions (Figure 4, Appendix B). In particular, in the case of the question asking about the range of spontaneous reporters (question 6), the rate of correct answers significantly increased after education (50%) compared to the pre-education test (18%) (p=0.001, post hoc p<0.001). In addition, for the question asking about the definition of ADR (question 3), the rate in the final test (55%) significantly decreased compared to that in the post-education test (79%) (p=0.014, post-hoc p=0.012) (Figure 4). A similar trend was also found in regard to the answers to question 5, which asked the existence and role of the UMC (p<0.001), showing that the rate of correct answers increased from 32% to 76% after education and then decreased to 61% in the final survey. The increase between pre- and post-education surveys (p<0.001) and pre-education and final surveys (p=0.003) was significant, but the decrease between post-education and final tests was not statistically significant (Figure 4). In the case of question 4, which asked about the concept of SR, the rate of correct answers increased from 68% to 100% after education, and this effect was maintained until the final survey (Figure 4). This change was statistically significant (p<0.001 either in Cochran’s Q test or post hoc comparison) (Figure 4, Appendix B).
Conversely, the rate of correct answers to the 7th and 10th questions asking about the range of drugs subject to SR and the agency in charge gradually increased over time (5, 8, and 35% for question 7 and 71, 82, and 84% for question 10). However, in these questions, the change according to the time point was not statistically significant (Figure 4, Appendix B).
The number of the questions with a correct answer rate of 90% or more also changed to a similar pattern as that mentioned above. It increased from 0 to 3 questions (questions 2, 4, and 8) and then decreased to 2 questions (questions 4 and 8) in the final survey (Figure 4, Appendix B). In contrast, questions with a correct answer rate of less than 50% showed a different pattern, decreasing from 3 questions (questions 5–7) to 1 question (question 7) and then increasing again to 2 questions (questions 6 and 7) (Figure 4, Appendix B).
Questions 6 and 7, which had the lowest correct answer rates in the post-education and final tests, were multiple-choice questions (Figure 4, Appendix B). Among the answer choices for question 6, asking who can become a reporter of the SR system, doctors (medical doctors, doctors of Korean medicine, and dentists) had a high selection rate of approximately 90% in the pre-education test. After education, 100% of respondents made the selection, and it was maintained at approximately 90%, even at the time of the final test. There was no significant difference in selection rate according to the time point (Table 2). The selection rate for pharmacists, Korean Oriental pharmacists, nurses, and patient caregivers increased significantly after education. It was then decreased in the final test, and the difference between the pre-education and final tests was not statistically significant (Table 2). The changing pattern according to the time point for the option of being reportable by the patient also showed similar patterns to those of other options. However, in this case, the selection rate in the post-education and final tests remained significantly higher than that in the pre-education test (Table 2).
Question 7 was about the category of drugs subject to SR. Among the answer choices for this question, the selection rate of over-the-count drugs, prescription drugs, and raw materials for herbal medicine increased significantly after education. However, it decreased in the final survey, so there was no significant difference compared to the pre-education test. In addition, there was no significant change in the selection rate of vaccines, biological drugs, pharmaceutical herbal medicine, or herbal formulas prepared in Korean medicine clinics, which showed no significant change after education. In particular, the selection rate for herbal formulas that are not included in the SR system of the current domestic pharmacovigilance system of Korea did not show a statistically significant difference, but rather, the selection rate gradually increased over time (Table 2).

3. Attitude level

Table 1 summarizes the average scores for the questions on the 38 respondents’ attitudes toward pharmacovigilance and SR. Confidence in causality assessment was low, with at an average score of 2.63 (1.025) points at first, but increased to 4.58 (0.826) after education. The final score decreased to 3.92 (1.171). This change in the scores according to the time point was statistically significant (p<0.001). As a result of the post hoc test, the differences between pre-education and post-education scores and between pre-education and final scores were significant (p<0.001, either case). Conversely, the difference between the pre-education and final surveys was not significant (Table 1).
The percentage of respondents who chose positive options in regard to their confidence in causality assessment increased from 18% to 92% after education, and then decreased to 66% in the final survey. There was a statistically significant difference (p<0.001). Post hoc tests showed significant differences at every time point, including pre-education and post-education tests (p<0.001), pre-education and final tests (p<0.001), and post-education and final tests (p=0.006) (Figure 3).
The pattern of the change in confidence in SR was similar to that of the changes in confidence in causality assessment, increasing from 2.08 (1.050) points to 4.74 (0.446) points and then decreasing to 4.40 (0.755) points in the final survey. The difference in scores according to the time point was statistically significant (p<0.001). In the post hoc comparison, there was a significant difference in scores between the pre-and post-education tests and between the pre-education and final tests (p<0.001). The difference between the post-education and final tests was not statistically significant (Table 1). The percentage of positive respondents for this question increased from 13% to 100% after education, and then decreased to 84% in the final survey. Positive respondents had statistically significant changes at each time point (p<0.001). As a result of the post hoc test, the changes prior to and after education were significant (p<0.001), while the difference between the post-education and final survey tests was not significant (Figure. 3).
The scores for willingness to report AEs showed a similar change pattern to that of the confidence scores for the first two questions, increasing from 4.58 (0.683) points to 4.90 (0.311) points and then decreasing to 4.63 (0.633) points. The degree of willingness was already very high prior to education, and therefore, the difference in scores after education and in the final survey in comparison to the baseline was not statistically significant (p=0.061, Table 1). The percentage of respondents who had a positive intention to report AEs was as high as 89% prior to education, which increased to 100% and then decreased slightly to 92% in the final survey. There was no significant difference in the percentage of positive respondents at each time point (Figure 3).

4. Perception level

Table 1 summarizes the average scores for the questions on the perceptions of pharmacovigilance and SR of the 38 participants who completed the three surveys. The degree of agreement with the question on the importance of the role of Korean doctors in SR was high at 4.71 (0.565) points in the pre-education survey. After education, this figure rose to 4.97 (0.162) points, and then, it decreased to 4.74 (0.724) points in the final survey (Table 1). The percentage of respondents who agreed with this question also rose from 95% to 100% after education and then slightly decreased to 98% in the final survey (Figure 3).
The degree of agreement with the question on whether SR is the duty of Korean medicine doctors changed slightly from 4.84 (0.594) points prior to education, 4.82 (0.692) points after education, and 4.95 (0.226) points in the final survey (Table 1). The percentage of respondents who agreed with this question rose from 95% to 97% after education and further rose to 100% in the final survey (Figure 3).
The degree of agreement with the necessity of pharmacovigilance training courses within the curriculum of the Korean medicine colleges increased from 4.11 (1.158) points to 4.66 (0.627) points after education and remained at 4.66 (0.627) points until the final survey (Table 1). The percentage of respondents who agreed with this question increased from 82% to 97% and then decreased to 92% in the final survey (Figure 3).
Among the average scores and the percentage of positive respondents to the above three questions, no statistically significant difference among the scores and response rates according to the time points was found (Table 1, Figure 3).
In the case of the question on the necessity of regular and continuous education on pharmacovigilance for doctors of Korean medicine after education, the average score was high at 4.61 (0.595) points. The positive response rate was also high at 94.7% (Table 1).

5. Opinions provided in the free-response question

Twenty-five students submitted answers to the free-response question included in the post-education questionnaire, and the response rate was 43.9% of those who completed all surveys. The answers provided to this question were divided into two major themes, including their impression or suggestions for this training course and their opinions on improving the current domestic pharmacovigilance system for herbal and traditional medicine (Table 3).

1) Opinions on this training course

The most frequent answer was that this course was helpful (n=6). Students responded that this course was useful and impressive (“It was useful to be able to analyze the causal relationship for various real cases”, and “It appears that the work of writing AE report forms will be memorable for a very long time”) and that it was an opportunity to prepare practical countermeasures for safe drug use in clinical practice after graduation (“I learned what to do when faced with ADRs during clinical practice”).
It was also suggested that this course is a necessary curriculum (n=5) (“I think it is something we must know as doctors of Korean medicine”, and “As healthcare professionals... it is something that we should understand well, but education on this theme has not been well done. However, I was very happy to learn through this course, and it was very helpful”).
Regarding the impact of this course, two students expressed their willingness to actively participate in SR (“I felt an urgent need to actively participate in SR”, and “When I see AEs in the future, I will report them”).
They also demonstrated an expectation that this kind of training course will contribute to the safe use of herbal and traditional medicine based on the establishment of a basis for the safety evidence (n=3) (“Many data need to be accumulated for evidence-based analysis or the safety of herbal and traditional medicine”, and “For the safe use and preparation of the basis for the safety of herbal and traditional medicine, we need to learn this content and should be actively involved in SR”).
In addition, there were also suggestions for improving the course. Opinions related to the necessity of expanding AE cases (“More cases need to be covered in the course”) and adding guidance on the steps after the process practiced in this course, that is, the process of verifying the reported cases and how to use the accumulated data (“I wonder how the cases are verified after SR and how to use the accumulated data”).

2) Opinions on the current domestic pharmacovigilance system

Students expressed that the education and promotion of this system need to be increased (n=2) (“I did not know about this system before this course. More promotion is required”, and “It seems that education about this system has not been sufficient. It would be nice for the lower grades and general people to be educated about this system”). They also said that this kind of education and promotion will stimulate more people to participate in SR in the future (n=3).
However, some students answered that the current system needs to be reformed in terms of herbal and traditional medicine. First, opinions that the range of pharmacovigilance should be expanded to include all currently circulated herbal and traditional medicine were raised (n=3) (“More kinds of herbal medicine including decoctions prepared by doctors of Korean medicine should be included in the pharmacovigilance system”, and “I think that it can be a greater help to the health of the people if safety data were provided on more kinds of herbal and traditional medicine widely used in Korea, including drugs prepared by individual doctors of Korean medicine as well as pharmaceuticals”). Second, there were also proposals to make the reporting system more friendly to herbal and traditional medicine (n=2) (“It was more difficult to report herbal medicines than conventional drugs”, and “It will be easier to report AEs if the autonomic ingredient completion function is provided when filling out the herbal prescription”). In addition, opinions on securing a separate department in charge of the pharmacovigilance of herbal and traditional medicine, establishing a toxicity database of herbal and traditional medicine, and providing information in detail on prescribed herbal and traditional medicine were also be proposed.

Discussion

This is a summary of the results from three surveys conducted prior to education, after education, and in the final assessment to evaluate the impact of a practice training course on pharmacovigilance and SR for senior students of a Korean medicine college. The author reported the results from conducting the same curriculum one year prior to this study and evaluating its impact. Rather than traditional lecture-oriented theoretical course, we designed a practical course focusing on the core competencies required of healthcare professionals in the clinical field 6). Students were provided with experiences reviewing actual cases of AEs involved in herbal and traditional medicine, conducting causality assessment, and filling out the AE case report form of the KIDS for healthcare professionals. A previous study13) showed the possibility that this training course can positively change students’ knowledge, attitudes, and self-efficacy. In particular, students answered that the practical experience they gained from this course was impressive and helpful. Similar findings were observed in this current study, which was designed by supplementing some of the limitations found in the previous study13).
The response rates of the pre-education, post-education, and final surveys were 98.2, 93.0, and 66.7%, respectively, showing a pattern of decreasing over time, similar to those of the previous study (97 and 65%)13). Finally, in this study, only 38 students remained in the final analysis group, which is a rather small sample size. However, the general characteristics of this final analysis group did not differ much from those of the previous study,13) including 97% of all trainees. For example, the average age of respondents (26 years old) and gender ratio (7:3) were similar to those of a previous study (26 years old and 6:4, respectively)13). In addition, the fact that no student had AE reporting experience was the same as that in the previous study13). By considering the general characteristics of these study populations, it is estimated that even though the number of the samples in this study was relatively small, the sample group was not heterogeneous with the population of this study (all course trainees).
However, by considering that the response rate decreased from almost 100% in the first survey to less than 70% in the final survey, the possibility that the 38 respondents, the final analysis group, showed biased responses cannot be excluded. Compared to just before or immediately after the course, at the time of the final survey, students’ interest in pharmacovigilance will have naturally declined over time. Additionally, the fact that the subjects of this study were senior students of a Korean medicine college and that the final survey was conducted with less than three months until the national examination of doctors of Korean medicine might also have contributed to reducing the response rate.
In this process, students with much interest in pharmacovigilance may have responded more positively to the survey until the end than did students with little interest so it cannot be ruled out that the results of this study have distorted the educational effect more positively than in reality. In fact, when calculating the baseline knowledge score (data not shown) of all respondents, the average was 5.18, 5.23, and 5.47 points in students who answered once for the pre-education survey; twice, before and right after course; and all three times, respectively. The group with more frequent survey participation tended to have higher knowledge score. Similarly, the average knowledge score from the post-education survey showed a tendency to increase as well, with 6.45 and 6.61 points, for students who answered twice and three times, respectively. Therefore, we cannot rule out the possibility that the 38 students in the complete analysis group were students who were more interested in pharmacovigilance from the beginning and had a high level of knowledge, attitudes, and perceptions compared to all 57 students.
The average score of the knowledge test in this study, like the previous one13), increased after education, This study further confirmed that this increase was statistically significant, and that the increased score was maintained even after several months in the final survey. The percentage of knowledge achievers increased more than two fold after education compared to prior to education, but only half of the respondents remained achievers in the final survey. This trend also appeared in the correct answer rate for each question of the knowledge test, showing a pattern of increasing after education, and then decreasing again in most of the questions. However, exceptionally, for the questions concerning the role of the UMC and the SR system, the percentage of correct answers was clearly maintained until the final test.
In terms of attitudes, similar to the previous study,13) the subjects of this study also showed positive changes after education, but in the final survey, these effects were decreased. The pattern in which the score that was low at baseline increased after education was particularly noticeable for the questions about confidence in causality assessment and SR. The lack of knowledge or confidence in specific skills related to pharmacovigilance and SR is not a unique characteristic of this study but is common in studies among undergraduate students and medical experts9,10). Responses regarding the willingness to report AEs also showed similar patterns, but the differences among the time points was not statistically significant. This is presumed to be due to the remarkable number of positive responses to this question even prior to education.
It is a well-known fact through previous studies 9,15,1820) that even though people have a wealth of general knowledge about pharmacovigilance and have positive attitudes and a high level of perceptions about SR, the rate of SR is significantly low, especially when they lack specific knowledge and confidence in causality assessment or SR practices. Participants in this study also had positive attitudes toward SR from the beginning, but they had a low level of knowledge and confidence in the core tasks of SR, and none of them had ever reported AEs. However, after education, they showed a remarkable improvement in their knowledge level and confidence in SR practices. The causality assessment and SR of AEs are key activities required for healthcare professionals in pharmacovigilance and are the main desired competencies of a pharmacovigilance curriculum for future healthcare professionals5,6,21). Therefore, the training course in this study can be said to have achieved the core objectives for pharmacovigilance practice that are required of healthcare professionals.
Although students’ self-confidence to assess causality and report the ADRs increased, a limitation of this study is that there was no objective evaluation of whether students’ performance was truly good at the practical level and it was one of the limitations of the previous study.13) The ultimate goal of the educational intervention of this study was to make students identify ADRs accurately and voluntarily report them to the agency in charge. One of the best way to objectively evaluate students’ competency for pharmacovigilance practice, it is necessary to put students into the pharmacovigilance practice field, and their actual competency needs to be evaluated by professional staff in charge, such as the approach taken by Schutte et al.7) There was also a report in Korea that pharmacy students who observed the practical work of a preceptor through a pharmacovigilance practice course had significantly higher SR-related knowledge and attitudes than did those who did not, and these students were also more likely to actually act on SR11). In Korea, the KIDS is in charge of pharmacovigilance practice including the management of regional drug safety centers, the receiving and handling of AE reports and related training22). It also operates a next-generation drug safety guidance course for students majoring in healthcare and in-depth practical training courses for pharmacy students23). The KIDS needs to expand its main subjects of education to include students of Korean medicine colleges, so that many experts in pharmacovigilance are produced in the field of traditional and herbal medicine in Korea. In the process, with the cooperation of the KIDS, an objective evaluation of the actual practical competency of students majoring in Korean medicine can be carried out. In addition, regional drug safety centers need to be expanded to include more Korean medical institutions. In Korea, only one Korean medical hospital has been designated as a center specializing in pharmaceutical herbal preparations23). It is expected that the designation of more number of Korean medical institutions as regional centers will promote the education and participation of not only local doctors of Korean medicine but also students in the field.
In this study, the average score for confidence in causality assessment and SR increased significantly after education and then decreased in the final survey, but it was still significantly higher than that prior to education. This confirms the fact that the effect of this course was maintained until approximately 4–6 months after the end of the training. Nevertheless, the absolute value of the average score and the percentage of positive respondents consistently increased but then decreased again. In particular, the percentage of respondents who showed confidence in the causality assessment was significantly decreased in the final survey in comparison to post-education survey. Considering these findings, it is suggested that re-education is required after approximately 6 months of initial training to maintain the core effect of the training.
However, there is room for reconsideration regarding the specific intervals for retraining, considering that the timing of the final assessment after the completion of the training course differed for each group. Students who participated in this study and received this practical training were divided into 10 groups, with each group receiving one week of training in the first half of 2020. Therefore, the timing of the training differed according to their group, so the training completion period was distributed between mid-April for early cases and the end of June for late cases. However, since the final survey was collectively conducted in the last week of October 2020, the follow-up period from the end of the training was distributed at least 4 months and at most 6 months. Therefore, since the maintenance period is judged to be ambiguous between 4 and 6 months, further study is needed to more accurately determine the optimal retraining interval. Ultimately, to make Korean medicine doctors in clinical field keep participating in SR, we suggest that the KIDS and the Association of Korean Medicine cooperate to conduct SR promotion projects and regular education courses using the continuous medical education system for license maintenance.
In terms of perceptions, students had a very positive perception of the importance of the role of doctors of Korean medicine in pharmacovigilance and the obligation of these doctors to perform SR from the beginning. This positive perception tended to maintain a high level continuously, without a significant change at each time point. There were many positive responses from prior to education about the necessity of pharmacovigilance practice courses in the curriculum of Korean medicine colleges, and this tendency was further strengthened even after education. In addition, the majority of respondents expressed a high level of agreement regarding the necessity of regular and continuous pharmacovigilance education for doctors of Korean medicine even after graduation.
Regarding the free-response question, students answered that this course was useful and necessary, especially in the causal assessment of actual AEs and the practice of completing AE report forms. They suggested that more active education and promotion are needed for the current pharmacovigilance system in Korea. In addition, some students believed that it was necessary to expand the scope of pharmacovigilance to cover all ranges of herbal and traditional medicines in circulation, and that there was a need to make the SR system more applicable for Korean medicine. The opinion that the current pharmacovigilance system, which has a relatively small proportion of herbal and traditional medicine, needs to be improved has been commonly pointed out in previous studies of experts in pharmacovigilance24) as well as undergraduates13).
Herbal and traditional medicine has received less interest than have other fields of conventional medicine in the area of pharmacovigilance and its education policy. The respondents of this study, majoring in Korean traditional medicine, showed that they have the potential to participate in pharmacovigilance. The greatest significance of this study is that following a previous study13), a practical training course was administered that satisfied global trends and recommendations, that is, focused on the development of the core competencies for pharmacovigilance required of healthcare professionals in the clinical field5,6,21), and confirmed the improvement of relevant knowledge and attitudes from new candidate participants in pharmacovigilance—undergraduates majoring in herbal and traditional medicine at a university in Korea.
Nevertheless, it is necessary to interpret the results of this study carefully considering the several limitations as follows. The sample size was small and not a pre-calculated optimal size. In addition, the subjects of this study, students from only one university, cannot represent all Korean medicine students. By including only students who participated in all three surveys in analysis, the possibility that the positive effect of this practice course was overestimated cannot be excluded. Only students’ self confidence in core skills was investigated, and no objective evaluation was made on how accurately they performed causal assessment and SR. Finally, this study does not show whether the students actually take an active role in SR in clinical practice. Therefore, although the respondents’ satisfaction with this training course was high, we cannot answer to the question whether this curriculum is really adequate enough to get the students to participate in SR after graduation. To answer this, a follow-up study monitoring the behaviors of the students from this training course after graduation is required.

Conclusions

This study showed that pharmacovigilance education for future doctors of Korean medicine has a positive impact on their knowledge, attitudes, and perceptions. However, a pattern was observed in which the positive change, which was significant immediately after the training, decreased in part in the final survey 4–6 months after the completion of the training. Among these findings, self-confidence in causality assessment, an action directly related to the execution of SR, was included. This suggests that pharmacovigilance education for undergraduates should not be limited to a one-time basis and that re-education is necessary approximately 6 months after the initial training.
The ultimate goal of the training course in this study is to make students active participants in SR in the clinical field when becoming doctors of Korean medicine. However, considering the findings of this study, it is difficult to ensure that all of them will actively participate in SR without re-education after graduation. A more long-term study is required to monitor whether students who have received such education actively report AEs in the clinical field after graduation. In addition, regular and continuous pharmacovigilance education for graduates is urgently required.

Fig. 1
Flow chart of the study
Pre-education survey was performed one week before the class, post-education survey was performed within one week after the end of the class, and the final survey was performed 4–6 months after the class.
jkm-42-4-40f1.gif
Fig. 2
Distribution of the total scores of the knowledge test
Pre-edu, pre-education survey performed one week before the class; Post-edu, post-education survey performed within one week after the end of the class; Final, final survey performed 4–6 months after the class.
jkm-42-4-40f2.gif
Fig. 3
Percentage of positive responders to the questions on knowledge, attitudes, and perceptions
Pre-edu, pre-education survey performed one week before the class; Post-edu, post-education survey performed within one week after the end of the class; Final, final survey performed 4–6 months after the class.
CA, causality assessment; SR, spontaneous reporting; AE, adverse event; DKM, doctor of Korean medicine; PV, pharmacovigilance; KM, Korean medicine
Statistical significance (*P<0.05) was calculated using Cochrane’s Q test with post-hoc McNemar’s test.
jkm-42-4-40f3.gif
Fig. 4
Percentage of correct answers in the knowledge test
Pre-edu, pre-education survey performed one week before the class; Post-edu, post-education survey performed within one week after the end of the class; Final, final survey performed 4–6 months after the class.
jkm-42-4-40f4.gif
Table 1
Survey results on the knowledge, attitudes, and perceptions on pharmacovigilance and spontaneous reporting
Category Pre-edu Post-edu Final P-value
Knowledge 5.47 (2.140)a,b 6.61 (1.001)a 6.61 (1.220)b <0.001

Attitudes Q1 2.63 (1.025)a,b 4.58 (0.826)a 3.92 (1.171)b <0.001
Q2 2.08 (1.050)a,b 4.74 (0.446)a 4.40 (0.755)b <0.001
Q3 4.58 (0.683) 4.90 (0.311) 4.63 (0.633) 0.061

Perceptions Q1 4.71 (0.565) 4.97 (0.162) 4.74 (0.724) 0.092
Q2 4.84 (0.594) 4.82 (0.692) 4.95 (0.226) 0.525
Q3 4.11 (1.158)a 4.66 (0.627)a 4.66 (0.627) 0.011
Q4 4.61 (0.595)

Knowledge is presented as the average of the total score (from 0 to 10 points) with the standard deviation in the parentheses. Attitudes and perceptions are presented as the average of the score for each question (from 1 to 5 points, 1: strongly disagree, 2: disagree, 3: no idea, 4: agree, and 5: strongly agree) with the standard deviation in parentheses. The p-value was calculated by repeated-measures analysis of variance. Superscripts (a, b) refer to groups with significant differences as a result of the post-hoc test using the Bonferroni correction.

Pre-edu, pre-education survey performed one week before the class; Post-edu, post-education survey performed within one week after the end of the class; Final, final survey performed 4-6 months after the class.

Attitude-related questions: Q1. I can evaluate the causal relationship between adverse events and suspected drugs, Q2. I know how to report adverse events to the agency in charge, and Q3. I will report adverse events, if I experience or witness them in the future.

Perception-related questions: Q1. The role of doctors of Korean medicine is important in pharmacovigilance and spontaneous reporting, Q2. Doctors of Korean medicine need to report adverse events to the agency in charge, Q3. Pharmacovigilance practice training courses should be included in the curriculums of Korean medicine colleges, and Q4. Doctors of Korean medicine need to be regularly educated in pharmacovigilance after graduation.

Table 2
Selection rate per each answer choice for questions 6 and 7 of the knowledge test
Question No Option No Pre-edu Post-edu Final. P-value
6 MD 92 100 95 0.174
DKM 89 100 95 0.050
Dentist 89 100 89 0.069
Pharmacist 55a 84a 82 0.006
KO pharmacist 34a 66a 53 0.009
Nurse 42a 71a 50 0.018
Patient 32a,b 71a 55b <0.001
Patient’s caregiver 21a 61a 45 <0.001
No idea 3 0 3 0.607

7 OTC drugs 58a 84a 79 0.015
Prescription drugs 50a 79a 76 0.005
Vaccine 39 55 66 0.054
Biopharmaceuticals 45 61 61 0.223
Pharmaceutical HM 42 61 55 0.128
HF prepared at KM clinics 32 39 47 0.259
Raw materials of HM 42a 68a 71 0.010
No idea 37 16 18 0.035

The values are presented as %, and p-values were calculated by Cochran’s Q test. Superscripts refer to groups with significant differences as a result of the post hoc comparison using McNemar’s test.

Pre-edu, pre-education survey performed one week before the class; Post-edu, post-education survey performed within one week after the end of the class; Final, final survey performed 4–6 months after the class.

MD, medical doctor; DKM, doctor of Korean medicine; KO pharmacist, Korean Oriental pharmacist; OTC, over-the-count; HM, herbal medicine; HF, herbal formula; and KM, Korean medicine

Table 3
Themes derived from the free-response survey
Category Theme Subtheme
Comments on the course Impression Informative course (n=6)
Essential course (n=5)
Attitudes Will actively report AEs (n=2)
Significance of the course Expected to contribute to securing the basis for the safe use of HM (n=3)
Suggestions for course improvement Need to increase the number of clinical cases (n=1)
Need to provide postreporting verification process and identify how to use the accumulated data (n=1)
Comments on the PV system Activation of education and promotion Need to activate the education and promotion of the system (n=2)
Expectation Expected that reports of HM-related AEs and PV will be activated (n=3)
Suggestions for improving the PV system for HM Need to expand the coverage of PV to include the HM in circulation (n=3)
Need to establish a complete charge department for PV for HM (n=1)
Need to reform the reporting system to be more HM friendly (n=2)
Need to establish and HM toxicity data base (n=1)
Need to provide more detailed information on ingredients by supplementing the HM label (n=1)

Appendices

Appendix A

Composition of the questionnaires
Category Item Pre-edu Post-edu Final
Demographic characteristics 1. Age
2. Gender

Experience related to AE reporting 1. Experience or witnessing any AEs
2. Experience reporting AEs
3. Reasons for non-reporting of AEs

Knowledge 1. PV definition
2. Side effect definition
3. ADR definition
4. SR concept
5. WHO-UMC
6. Main agents of SR
7. Subjects of SR
8. Causality assessment
9. Nondisclosure of patient privacy in SR
10. Where to report ADRs

Attitudes 1. Confidence in causality assessment
2. Confidence in SR
3. Willingness to report AEs

Perceptions 1. Importance of the role of DKMs in SR
2. Duty to report AEs as DKMs
3. Necessity of PV training course in the curriculum of KM universities
4. Necessity of regular and continuous PV education after graduation

Free-answer Opinions on this training course and/or the current domestic PV system in view of future DKMs

Pre-edu, pre-education survey performed one week before the class; Post-edu, post-education survey performed within one week after the end of the class; Final, final survey performed 4–6 months after the class.

AE, adverse event; PV, pharmacovigilance; ADR, adverse drug reaction; SR, spontaneous reporting; WHO, World health organization; UMC, Uppsala monitoring centre; DKM, doctor of Korean medicine; KM, Korean medicine

Appendix B

Percentage of the correct answers per each question of the knowledge test
Category Pre-edu Post-edu Final P-value
Percentage of the correct answers per each question Q1 66 82 74 0.241
Q2 84 92 87 0.584
Q3 58 79a 55a 0.014
Q4 68a,b 100a 100b <0.001
Q5 32a,b 76a 61b <0.001
Q6 18a 50a 32 0.001
Q7 5 8 35 0.549
Q8 87 97 95 0.115
Q9 61 76 71 0.211
Q10 71 82 84 0.269

Total score was presented as average (standard deviation), and the p-value was calculated by repeated-measures analysis of dependencies. Superscripts (a,b)refer to the groups with significant differences as a result of the post hoc test using the Bonferroni correction.

Percentage of the correct answers per each question was presented as %, and the p-value was calculated by Cochran’s Q test. Superscripts refer to the groups with significant differences as a result of the post hoc comparison using McNemar test.

Pre-edu, pre-education survey performed one week before the class; Post-edu, post-education survey performed within one week after the end of the class; Final, final survey performed 4–6 months after the class.

Reference

1. Rawlins MD. Pharmacovigilance: paradise lost, regained or postponed? The William Withering Lecture 1994. J R Coll Physicians Lond. 1995; 29:1. 41–9.
pmid pmc

2. World Health Organization. The safety of medicines in public health programmes: pharmacovigilance, an essential tool. 2006.


3. Hornbuckle K, Wu H-h, Fung MC. Evaluation of Spontaneous Adverse Event Reports by Primary Reporter—A 15-Year Review (1983 to 1997). Drug Inf J. 1999; 33:4. 1117–24.
crossref

4. World Health Organization. WHO guidelines on safety monitoring of herbal medicines in pharmacovigilance systems. World Health Organization;2004.


5. Hartman J, Härmark L, van Puijenbroek E. A global view of undergraduate education in pharmacovigilance. Eur J Clin Pharmacol. 2017; 73:7. 891–9.
crossref pmid

6. van Eekeren R, Rolfes L, Koster AS, Magro L, Parthasarathi G, Al Ramimmy H, et al. What Future Healthcare Professionals Need to Know About Pharmacovigilance: Introduction of the WHO PV Core Curriculum for University Teaching with Focus on Clinical Aspects. Drug Saf. 2018; 41:11. 1003–11.
crossref pmid pmc

7. Schutte T, Tichelaar J, Reumerman MO, van Eekeren R, Rolfes L, van Puijenbroek EP, et al. Feasibility and educational value of a student-run pharmacovigilance programme: a prospective cohort study. Drug saf. 2017; 40:5. 409–18.
crossref pmid pmc

8. Schutte T, Tichelaar J, Reumerman MO, van Eekeren R, Rissmann R, Kramers C, et al. Pharmacovigilance skills, knowledge and attitudes in our future doctors–A nationwide study in the Netherlands. Basic Clin Pharmacol Toxicol. 2017; 120:5. 475–81.
crossref pmid

9. Alwhaibi M, Al Aloola NA. Healthcare students’ knowledge, attitude and perception of pharmacovigilance: A systematic review. PloS One. 2020; 15:5. e0233393
crossref pmid pmc

10. Park S-H, Chung KH, Park B-J, Kang DY, Shin J-Y. Educating Healthcare Professionals in Pharmacovigilance: Global Trends and Korea’s Status. Korean Med Edu Rev. 2020; 22:1. 32–45.


11. Yu YM, Kim S, Choi KH, Jeong KH, Lee E. Impact of knowledge, attitude and preceptor behaviour in pharmacovigilance education. Basic Clin Pharmacol Toxicol. 2019; 124:5. 591–9.
crossref pmid

12. Ade-Abolade K, Ogunleye I, Ogunleye O, Osakwe A, Yusuf M, Osikoya K. Educational Intervention to Improve the Practice of Pharmacovigilance among Traditional Medicine Practitioners in Lagos Nigeria. Nigerian Medical Practitioner. 2015; 69:1–2. 9–14.


13. Kim M. Effect of A Pharmacovigilance Practice Training Course for Future Doctors of Korean Medicine on Knowledge, Attitudes and Self Efficacy. JKM. 2020; 41:1. 21–44.
crossref

14. Choi N, Kwon H, Lee A, Park B. A survey on attitudes and awareness of physicians and pharmacists regarding spontaneous reporting system and experience for adverse drug events in Goyang-si. JPERM. 2008; 1:1. 44–52.


15. Bhagavathula AS, Elnour AA, Jamshed SQ, Shehab A. Health professionals’ knowledge, attitudes and practices about pharmacovigilance in India: A systematic review and meta-analysis. PloS One. 2016; 11:3. e0152221
crossref pmid pmc

16. Hegerius A, Caduff-Janosa P, Savage R, Ellenius J. E-Learning in Pharmacovigilance: An Evaluation of Microlearning-Based Modules Developed by Uppsala Monitoring Centre. Drug Saf. 2020; 43:11. 1171–80.
crossref pmid pmc

17. Braun V, Clarke V. Using thematic analysis in psychology. Qual Res Psychol. 2006; 3:2. 77–101.
crossref

18. Gupta SK, Nayak RP, Shivaranjani R, Vidyarthi SK. A questionnaire study on the knowledge, attitude, and the practice of pharmacovigilance among the healthcare professionals in a teaching hospital in South India. Perspect Clin Res. 2015; 6:1. 45–52.
crossref pmid pmc

19. Toklu HZ, an Soyalan M, Gültekin O, Özpolat M, Günay AÇ, Yavuz DÖ, et al. The knowledge and attitude of the healthcare professionals towards pharmacovigilance and adverse drug reaction reporting in Northern Cyprus. J Pharmacovigil. 2016.
crossref

20. Chopra D, Wardhan N, Rehan HS. Knowledge attitude and practices associated with adverse drug reaction reporting amongst doctors in a teaching hospital. Int J Risk Saf Med. 2011; 23:4. 227–32.
crossref pmid

21. Beckmann J, Hagemann U, Bahri P, Bate A, Boyd IW, Dal Pan GJ, et al. Teaching pharmacovigilance: the WHO-ISoP core elements of a comprehensive modular curriculum. Drug Saf. 2014; 37:10. 743–59.
crossref pmid

22. Shin JY, Jung SY, Ahn SH, Lee SH, Kim SJ, Seong JM, et al. New initiatives for pharmacovigilance in South Korea: introducing the Korea Institute of Drug Safety and Risk Management (KIDS). Pharmacoepidemiol Drug Saf. 2014; 23:11. 1115–22.
crossref pmid

23. Korea Institute of Drug Safety & Risk Management. [cited 2021 15 Jan]; Available from: https://www.drugsafe.or.kr/ko/index.do


24. Kim M, Woo Y, Han C-h. Current status of the spontaneous reporting and classification/ coding system for herbal and traditional medicine in pharmacovigilance. Integr Med Res. 2021; 10:1. 100467
crossref pmid

Editorial office contact information
3F, #26-27 Gayang-dong, Gangseo-gu Seoul, 157-200 Seoul, Korea
The Society of Korean Medicine
Tel : +82-2-2658-3627   Fax : +82-2-2658-3631   E-mail : skom1953.journal@gmail.com
About |  Browse Articles |  Current Issue |  For Authors and Reviewers
Developed in M2PI