Usefulness of the Mobile Interactive Pharmacy Education Enhancement Resource (miPEER) Mobile Web-App as a Learning Tool for Electronic Health Records

Corresponding Author:

Kevin Yap
Professor, Department of Pharmacy
National University of Singapore, Republic of Singapore
Tel: +65 6601 3253
Fax: +65 6779 1554
e-mail: kevinyap.ehealth@gmail.com; kevinyap.ehealth@yahoo.com

Abstract

Formal electronic health record (EHR)-related training is lacking in the curricula of many pharmacy schools, thus resulting in a lack of skills base in pharmacy graduates. We developed an in-house mobile web-app – the “Mobile Interactive Pharmacy Education Enhancement Resource” (miPEER) – to train students on how to extract relevant patient health information (PHI) from EHRs to solve clinical cases. miPEER contains 3 core features – “Virtual Patient Records (VPR)”, “Case Questions” and “Druglist”. The VPR consists of mock PHI which simulates those of EHRs, so that students can learn how to extract relevant information to solve counselling scenarios in the “Case Questions”. The “Druglist” provides hyperlinks to drug information of the most frequently-used prescription-only, pharmacy-only and general sales list medications used in Singapore. Through 2 online surveys, we demonstrate how mobile apps, such as miPEER, can potentially train new generations of digitally competent pharmacists to improve the quality of pharmaceutical care.

Keywords

Electronic health records; Mobile apps; Patient health information; User-perception study; Virtual patient records

Abbreviations

CPOE: Computerized prescriber order entry; EHR: Electronic health record; HTML: Hypertext markup language; miPEER: Mobile Interactive Pharmacy Education Enhancement Resource; NUS: National University of Singapore; PharmD: Doctor of Pharmacy; PHI: Patient health information; PIMS: Pharmacy information management systems; SPSS: Statistical Package for the Social Sciences; VPR: Virtual patient records

Introduction

Throughout the years, the changing demands of pharmacy practices due to the onset of advancements in patient care, growing biomedical information and technological innovations have led to multiple reforms in pharmacy and other allied-health education [1-3]. Advancements in technology have also allowed students to learn about the vast amounts of clinical knowledge on their own [3]. However, students are unable to correlate their pharmacological and clinical knowledge into the context of pharmacy practice [2]. Graduates who possess strong clinical knowledge are not competent enough for pharmacy practice training in hospital settings as they lack the required skills associated with patient safety, accurate medication dispensing, assessment of drug-related problems and general communication [2-4]. In response, pharmacy schools have adopted a more integrative approach to bridge the gap between the pharmaceutical sciences and clinical practice knowledge [4-6]..

The use of Electronic Health Records (EHRs) has been suggested as an innovative learner-centric strategy to foster critical-thinking and problemsolving skills, as well as patient-centered care skills in pharmacy students [7]. The advent of EHRs – longitudinal collection of patient health information (PHI) accessed electronically by authorized individuals to support the delivery of healthcare – has enabled pharmacists to enhance the quality of delivery and safety of healthcare through improved accessibility to PHI and enabling better understanding of the health statuses of their patients [4,8-10]. Pharmacy information management systems (PIMS), which includes electronic medications order, automated billing, computer-generated labeling and electronic medication administration systems), also help enhance drug selection, preparation and dispensing workflows [11]. However, there is a lack of formal informatics-related training in the curricula of many pharmacy schools, except for Doctor of Pharmacy (Pharm D) programs [4,7]. Even in schools that offer Pharm D programs, the actual inclusion of informatics as a core competency within the curriculum is much lower than the schools’ desired extent for inclusion [12].

Traditionally, pharmacy students did not need to be trained on health information technologies and how to extract relevant patient information from EHRs. In school, relevant PHI specific to what was needed for case scenarios were usually provided for students during class discussions. However, it is not uncommon for pharmacists to be using computers, electronic resources and other automated technologies in practice settings today. Pharmacists may be unprepared when participating in EHR and computerized prescriber order entry (CPOE) implementation and/or evaluation teams due to their lack of knowledge and exposure to these systems [13,14]. Graduates have to learn about EHR systems and how to extract relevant health-related information to manage patients on-the-job in clinical practices, on top of adapting to a new working environment. Therefore, there is a need to expose pharmacy students to the types of informatics technologies that they will use next time in their curricula.

Although it is acknowledged that health information technologies should be a core pillar for creating an optimal pharmacy practice model [15], much work still needs to be done to incorporate the pharmacy informatics content into the curricula of pharmacy schools [15]. Pharmacy informatics, a subset of clinical informatics, focuses on how data, information and technologies can be integrated with the process of medications use and management, so as to improve health outcomes [15]. It has been suggested that pharmacy informatics education should include topics related to prescribing and prescription reviews (e.g. e-prescribing, CPOEs, labeling, clinical decision support systems, electronic drug information resources), compounding and dispensing (e.g. automated dispensing, robotics, barcode technologies, medication tracking systems), medication administration (e.g. smart infusion pumps, autopatient identification tools), and monitoring of therapies (e.g. EHRs, clinical monitoring and surveillance tools, mobile technologies and telepharmacy, medication databases); so that students can better leverage on these technologies to support medication-related care when they graduate [15]. While this specialization may not seem relevant to majority of pharmacy students, since not all of them will specialize in this field; but the basic needs of this field should be taught to all students whose future practice will rely on such informatics technologies.

In Singapore, the healthcare system is divided into 6 distinct health clusters and many community pharmacies scattered across the country. Each of them deploys their own unique variant of EHRs and PIMS. The lack of a common standardized healthcare record system poses a challenge to the only pharmacy school, which trains majority of the country’s pharmacists. The traditional way of providing specific PHI needed for case scenario discussions in the educational setting leads to a disjoint in the way PHI is accessed and interpreted in clinical settings, resulting in a lack of skills base in pharmacy graduates. This can provide unnecessary stress to these newlypracticing pharmacists. Without any exposure to such systems within the school curriculum, students may encounter difficulties in using EHRs to obtain relevant patient information to manage their patients well, which in turn can predispose patients to other medical errors and drug-related problems. As students may also be exposed to different EHR systems when they graduate and work in various hospital and community settings, there is a need to build a generic platform that contains features common to the different EHRs in the healthcare clusters, so that students can be versatile and adaptable to any system that they are exposed to when they graduate.

Technological innovations, such as the use of smartphones and mobile apps, in pharmacy education can potentially bridge this gap. Smartphones and mobile apps are increasingly becoming commonplace among the healthcare community [16-18]. With the growing adoption of smartphone technologies among both healthcare professionals and healthcare students, mobile apps are a potential platform that can be used to train pharmacy students to be familiar with EHRs and improve on their skills in handling PHI [18].

As part of a technology-enhanced educational initiative in the pharmacy practice curriculum at our university, a mobile web-app called the “Mobile Interactive Pharmacy Education Enhancement Resource” (miPEER) was developed as an educational tool to familiarize pharmacy students with the types of PHI in EHRs and use them appropriately when providing clinical care. This app was designed with the goal of providing students with a generic platform that consisted of common elements from EHR systems populated with mock patient data, enhanced with features that would be relevant for the training of students for clinical practices. Our hypothesis was that the app would be useful to pharmacy students as a learning tool to familiarize themselves with the types of PHI in EHR systems. Thus, the objectives of this study were to determine the usefulness of miPEER as a learning tool for students, and its usability as a mobile app on smartphone devices.

Methods

▪ Development of miPEER

miPEER was first conceptualized on storyboards based on existing EHR systems. It was then developed on an Android operating system (OS) as a Virtual Patient Record (VPR) [15], before switching to a mobile web-app platform. The latter was developed using hypertext markup language (HTML5) and jQuery Mobile on Adobe Dreamweaver CS5.5 (Version 11.5 Build 5315) (Adobe Systems Software, Ireland). It could be accessed online from web browsers on computers or smartphone devices (e.g. Safari app in the Apple iPhone) (Figure 1). miPEER contained a total of 7 features organized into 3 core and 4 supporting features (Table 1). The VPR feature consisted of mock patient health information which simulated those of EHRs, so that students could learn how to extract and use the relevant information to solve counselling and dispensing scenarios in the “Case Questions” feature. The “Druglist” feature provided drug information of the most frequently-used prescription-only, pharmacy-only and general sales list medications used in Singapore, which were linked to their respective monographs from the UpToDate^® intranet database [19,20]. The supporting features included “About Us”, “Statutes”, “Terms of Use” and “Useful Links”. The “About Us” section contained contact information of the miPEER development team, while relevant laws and regulations related to pharmacy practice in Singapore were contained in the “Statutes” feature. The “Terms of Use” feature contained a short disclaimer on the use of miPEER with information about its purpose, advertising and privacy policies and user risks. Lastly, hyperlinks to drug information and journal resources, as well as professional pharmacy organization websites could be accessed from “Useful Links”. The end-product was then uploaded onto a Dreamhost server for online access by pharmacy students.

Table 1: Descriptions on the content contained within the 7 features of miPEER.

▪ Design of User Perception Study

A user-perception study was conducted within a pharmacy practice module in an undergraduate class between August and November 2013. The study was approved by the university’s Institutional Review Board. Classes were held once every week where students had to participate in counseling and dispensing sessions, and use miPEER to solve clinical case scenarios. Final-year undergraduate pharmacy students enrolled in the module had access to the mobile web-app.

Two online surveys were conducted to obtain student feedback about miPEER. Both surveys were created on the online student intranet portal and administered at different times during the module (Figure 2). The first survey (Survey #1) consisted of 10 questions and was conducted over 3 weeks (#1a, #1b and #1c) at the end of each pharmacy practice class. The questions aimed to identify the usage patterns of miPEER by the students and whether they were more adept its usage over time. The results of this first survey were then collated one day before the start of the next class. The second survey (Survey #2, post-course survey) consisted of 13 questions and was conducted to determine if the implementation of miPEER had enhanced the learning experiences of students. Additionally, qualitative feedback was sought on how miPEER could be improved.

▪ Statistical Analysis

Results from both surveys were collated in IBM Statistical Package for the Social Sciences (SPSS) Statistics (v20) (IBM Corporation, Armonk, NY) and analyzed using descriptive statistics. Pearson’s Chi-square test and Fisher’s exact tests were used to evaluate the relationships among the student demographic variables, weekly response rates, as well as the perceptions of miPEER between students with and without prior EHR experience. The Wilcoxon signed ranks test was used to evaluate the mean rating scores among the different weeks. P-values below 0.05 were considered to be statistically significant.

Results

▪ miPEER Survey 1 Results

Weekly participant response rates of the first survey were 85.3% (123 out of 150), 42.7% (64 out of 150) and 50.0% (75 out of 150) respectively. The most and least popular devices used by students to access miPEER were laptops/MacBooks (65 out of 87, 75%) and tablets (11 out of 87, 13%). There was an increasing trend of students using desktops/ mac computers and decreasing trend of students who used smartphones to access miPEER as the weeks passed (Table 2). Although there was a significantly higher proportion of students who accessed a hardcopy version of miPEER in week 2 compared to week 1 (46.9% versus 30.9%, p=0.047), the percentage decreased to again to a similar percentage in week 3 (29.3%, p=0.405). Students were more familiar with the usage of the app over the 3 weeks, shown by an increasing trend in the mean familiarity scores from week one to week three (3.39 versus 4.12, p<0.0001). The most useful VPR parameter was “Medical History” (75 out of 87, 86%), as agreed by more than twice the number of students compared to “Lab Data” (36 out of 87, 42%), the least useful parameter. Nearly three-quarters (62 out of 87, 71%) of the respondents reported that the “Case Questions” feature was their most frequently used feature. In fact, an increasing proportion of students used the “Case Questions” feature as the weeks passed (67.5% in week one versus 77.3% in week three, p=0.845). In general, students gave miPEER better rating scores over the three weeks in terms of providing an insight to the workings of an EHR system and its application in clinical practices (3.52 in week one versus 3.67 in week three, p=0.438); enabling them to understand the types of PHI available in an EHR (3.57 in week one versus 3.78 in week three, p=0.082); and making it easier for them to prepare for their pharmacy practice classes (3.30 in week one versus 3.35 in week three, p=0.758). Over half of the respondents (46 out of 87, 53%) used miPEER about 2-3 times per week outside class to prepare for their cases. The proportion of respondents using miPEER in a study environment decreased (87.8% in week one to 78.7% in week three, p=0.676) over the weeks, while the proportion using it when travelling increased (9.8% in week one to 20.0% in week three, p=0.622). Nevertheless, at the end of the three weeks, majority of the respondents still used miPEER in a study environment (59 out of 75, 78.7%), but only 15 out of 75 (20.0%) used it while traveling (e.g. in buses, trains, cars).

Table 2: Results of Survey #1.

▪ miPEER Survey 2 Results

Thirty-eight (25%) students answered the postcourse survey (Table 3). Thirty respondents (79%) had prior experience with EHRs, of which 27 (90.0%) of them encountered EHRs during their preceptorship attachments at local hospital pharmacies. Respondents preferring laptops/MacBooks (22 out of 38, 58%) were significantly higher when compared to smartphones (7 out of 38, 18%), tablets (5 out of 38, 13%), desktop/Mac computers (3 out of 38, 8%) and hardcopy versions (1 out of 38, 3%) (p<0.001 each). Respondents who used miPEER in a study environment (34 out of 38, 90%) were also significantly higher than those who used it while traveling (1 out of 38, 3%) and during class time (3 out of 38, 8%) (p<0.001 each).

In general, about two-thirds of the respondents agreed that students who had not encountered EHRs before would have a better idea of what EHRs would be like after using miPEER (24 out of 36, 67%). More than half (21 out of 36, 58%) also agreed that students would know how to identify and extract relevant PHI from EHRs after using miPEER. Overall navigation experience of miPEER was well-received, with 86% of the respondents (31 out of 36) agreeing that its graphical user interface was easy to understand and that they had no trouble navigating through the app. Three-quarters (27 out of 36, 75%) found miPEER easier to use and navigate over time as they became more familiar with it.

Nearly two-thirds of the respondents (23 out of 36, 64%) felt that the PHI in the VPR feature was organized clearly and easy to understand. However, only 39% (14 out of 36) felt that the PHI provided was detailed enough to solve the case studies. The VPR features that were better representatives of real-life EHR parameters were “Patient Biodata” (20 out of 36, 56% agreed) and “Prescription History” (19 out of 36, 53% agreed). In fact, a significantly lower proportion of students with prior EHR experience agreed that the “Medical History” (8 out of 29, 28% versus 6 out of 7, 86%; p=0.008) and “Lab Data” parameters (8 out of 29, 28% versus 6 out of 6, 100%; p=0.002) were good representations of real-life EHRs, compared to those without EHR experience.

Large proportions of respondents felt that the content within the Case Questions (23 out of 36, 64%) and Druglist features (22 out of 36, 61%) were organized clearly and easy to understand. However, lower proportions felt that the content in these features reflected clinical practices (20 out of 36, 56% for Case Questions; 19 out of 36, 53% for Druglist). The features that students felt could be better improved were the VPR (28 out of 38, 74%), Druglist (20 out of 38, 53%) and Case Questions (19 out of 38, 50%).

Three-quarters of students (28 out of 37, 76%) saw the usefulness of miPEER as a teaching and learning tool for pharmacy practice modules. More than half (19 out of 36, 53%) wanted miPEER to be used as an e-learning platform for future cohorts of pharmacy students. Among all the clinical and pharmaceutical science modules, future pharmacy practice and dispensing classes garnered the most votes (27 out of 38, 71%).

Discussion

miPEER was a mobile web-app developed as a training aid for students to know the workings of an EHR system, and also how to extract relevant PHI to manage patient cases. Local pharmacy students in Singapore are generally not trained on pharmacy informatics and EHR systems in their undergraduate years. This is the first major effort to integrate such training for new batches of pharmacy students, as the practice faculty deemed these skills necessary for students’ clinical practices after graduation. Overall, students felt that the use of miPEER was a constructive experience. Many of them reported being able to draw connections from their clinical pharmacy and pharmacy practice knowledge via miPEER, and were interested to see how miPEER could be further developed to suit their learning needs.

Prior encounters with EHRs were a major factor affecting the user perceptions of miPEER. The responses of students with prior EHR experiences seemed to differ from those who did not have experiences with EHRs. The former group generally felt that miPEER was less representative of real-life EHRs, compared to the latter group. Their perceptions could be due to the structure and presentation of miPEER. During the storyboarding process, only the basic elements of EHRs were taken and used as a reference for building the features of miPEER. In clinical settings, the data in EHRs are more complex in structure and more detailed in terms of health-related information. The relatively more simplistic structure of miPEER could have affected the students’ perceptions of its realism in simulation, since undergraduates who had already encountered EHRs in their community and hospital preceptorship attachments might have had higher expectations with regards to its complexity. Furthermore, features that were usually not found in EHRs (e.g. “About Us” and “Statutes”) could have distorted the students’ perceptions of miPEER being an accurate representation of EHRs. The different EHR designs used in various hospitals, and the fact that patient records in hospitals also varied with community pharmacy settings could have further aggravated this distortion. A followup study could be done to ascertain which key aspects of EHRs were lacking in miPEER, and which of the existing features would be less useful in clinical settings. Pharmacists in practice could also be engaged in future studies to obtain their feedback since they would probably be more familiar with EHRs from their daily work. Their feedback could then serve as useful guidelines for future development updates in miPEER.

miPEER was developed as a mobile web-app to overcome the problem of developing multiple versions of native apps on the Apple iOS, Android and Windows 8 platforms. Additionally, the development team believed that the prevalence of smartphone usage in the everyday lives of students could potentially facilitate the adoption of miPEER in the educational setting. Access to miPEER on mobile web browsers (e.g. Internet Explorer, Mozilla Firefox and Apple Safari) in most handheld devices (e.g. Android and Windows smartphones, iPhones, iPads and tablet computers) would grant this web-app the advantage of a greater penetrance into its target population of the tech-savvy undergraduate students [21,22]. Developing miPEER on the mobile web platform would also allow for easy updating of the app. However, the decreasing trend of smartphone usage over the weeks and students’ preference to use miPEER on laptops/MacBooks instead was unexpected. Feedback from the respondents attributed this preference to the relatively smaller screen size of smartphone or tablets, or the lack of mobile internet subscriptions by the students. The smaller screen size of smartphones could have made it uncomfortable for students to sift through the vast amounts of PHI on miPEER, while limited data usage imposed by the students’ telecommunications provider as a result of their cheaper subscription plans could have also deterred them from using it on their smartphones; thus limiting its potential as an educational tool for students. Future work could possibly look into merging parts of miPEER into native apps (e.g. Druglist, About Us, Terms of Use), while tapping on the mobile internet for data that needed to be updated real-time (e.g. Case Questions, VPR). Additionally, miPEER could also be developed for use on “phablets” – devices which are a cross between smartphones and tablets – so as to overcome the limitation of small screen sizes, but yet retain the portability of smartphones and tablets, so that it might be more useful for students to access while on the go.

In general, the students felt that miPEER was suitable to be implemented in their clinical and pharmacy practice modules, in particular for their dispensing, law and pharmacovigilance, and pharmacology and pharmacotherapy. This trend was similar among students with and without prior EHR experiences, even though the former group perceived the app as not being realistic enough. Their perceptions echo the views of pharmacy faculty in the USA and the Accreditation Council for Pharmacy Education Standards and Guidelines to include a core set of pharmacy informatics competencies into pharmacy programs [11,14,23]. We believe that both groups of students could see the usefulness and direct relevance of using miPEER in their modules to train them on extracting PHI on EHRs because majority would be going out for their pre-registration attachments in hospitals and community pharmacies. Besides applying their knowledge and skills to prescription reviews and medication-related decision making, it would also be essential that they are able to survey and identify prescribing and medication use trends, such as potential drug interactions or adverse drug reactions, through the use of EHRs [14]. Although the VPR feature of miPEER would still need to be improved in order to reflect the complexity of EHR systems in clinical practices, this app was not meant to replicate EHRs, but used as a learning tool for the education of pharmacy students. Feedback from a respondent suggested that miPEER would be more useful for a lower year undergraduate module (i.e. years 1-2) to train students before they went out on their community and hospital preceptorship attachments. This suggestion has been brought to the attention of our department heads and will be considered for future batches of pharmacy students.

Limitations

The main limitation of this study was the onset of “survey-fatigue” among the students. Students could have become less inclined to complete the survey objectively after multiple rounds of surveys. Furthermore, the responses of individual students were anonymous throughout the study, thus their weekly responses could not be tracked. The responses obtained from the same individuals could have provided a more accurate view of their user-perceptions. A focus group consisting of students who would be willing to provide feedback on a regular basis could be carried out in future studies, so that further enhancements could be made to cater miPEER as a mainstream learning tool for pharmacy students.

Another limitation was that this study only assessed the usefulness of miPEER based on the perceptions of students. Its effectiveness as a learning tool could not be evaluated as there were no assessments that were designed to correlate their perceptions with their grades. Studies on subsequent batches of students could potentially target the effectiveness of miPEER through analysis of students’ grades in assessments specially designed to test students’ knowledge and skills on identifying and extraction of PHI from EHRs to solve clinical cases. The results could potentially determine if miPEER would indeed be an effective learning tool that has a positive effect on the students’ learning experiences.

Conclusions

Technological advances and innovations in the educational arena are changing the landscape of pharmacy education. The use of miPEER has benefitted students by exposing them to the workings of EHRs in clinical settings, and also provided opportunities for students to hone their critical thinking and analytical skills by training them to extract relevant PHI from a large database to solve their clinical case scenarios. With the growing adoption of electronic platforms to manage PHI in hospital and community pharmacies, it is imperative that new pharmacy graduates are able to work with these systems with ease without compromising on patient care and safety. To achieve this, existing pharmacy curricula need to better integrate pharmacy informatics content, so that students can better leverage on these technologies to support medication-related care when they graduate. The use of pharmacy-specific apps has the potential to train new generations of pharmacists who are digitally competent in practice settings, so that the quality of pharmaceutical care can ultimately be improved. The use of miPEER in our institution’s pharmacy practice course definitely presents a good opportunity to train such relevant skills in our students.

Conflicts of Interest

All authors have no conflict of interests that are directly relevant to the study.

Disclosure of Funding

This article was written as part of a bigger study funded by the Teaching Enhancement Grant (C-148-000-036-001) awarded by the Centre for Development of Teaching and Learning, National University of Singapore (NUS).

Acknowledgements

The authors would like to thank A/Prof. Wai Keung Chui for his administration support, A/Profs. Joyce Lee and Christine Teng, and Asst. Prof. Joanne Chang from the NUS Pharmacy Department for providing the content for the mobile app.

References

1. City of Edinburgh Scottish Development Agency. The Edinburgh Declaration. Med. Educ 22(5), 481 (1988).
2. Dandannavar VS. Effect of integrated teaching versus conventional lecturing on MBBS phase I students. Rec. Res. Sci. Technol 2(11), 40-48 (2010).
3. Ratka A. Integration as a paramount educational strategy in academic pharmacy. Am. J. Pharm. Educ 76(2),19 (2012).
4. Kirwin JL, DiVall MV, Guerra C, et al. A simulated hospital pharmacy module using an electronic medical record in a pharmaceutical care skills laboratory course. Am. J. Pharm Educ 77(3), 12-62 (2013).
5. Pearson ML, Hubball HT. Curricular integration in pharmacy education. Am. J. Pharm. Educ 76(10), 204 (2012).
6. Custers EJ, Cate OT. Medical students attitudes towards and perception of the basic sciences: a comparison between students in the old and the new curriculum at the University Medical Center Utrecht, The Netherlands. Med. Educ 36(12),1142-1150 (2002).
7. Frenzel JE. Using electronic medical records to teach patient-centered care. Am. J. Pharm. Educ 74(4),71 (2010).
8. Yap KY, Chan A, Chui WK. Improving pharmaceutical care in oncology by pharmacoinformatics: The evolving role of informatics and the internet for drug therapy. Lancet. Oncol 10(10), 1011-1019 (2009).
9. Angaran DM, Angst W, Barthell EN, et al. Connecting Americans to their healthcare: Executive summary. Connecting for Health. New York: Markle Foundation 1-12 (2004).
10. Carter JH. What is the electronic health record? (2nd edtn), Electronic Health Records: A Guide for Clinicians and Administrators. USA: ACP Press1-20 (2008).
11. Fox BI, Flynn AJ, Fortier CR, et al. Knowledge, skills, and resources for pharmacy informatics education. Am. J. Pharm. Educ 75(5), 93 (2011).
12. Zeind CS, Blagg JD Jr., Amato MG, et al. Incorporation of Institute of Medicine competency recommendations within Doctor of Pharmacy curricula. Am. J. Pharm. Educ 76(5), 83. (2012).
13. Melton BL. Informatics: Catching the next wave in pharmacy education. Hosp. Pharm 48(11):890-891 (2013).
14. Fox BI. Health information technology: Are we aware and engaged? Am. J. Pharm. Educ 77(6), 113 (2013).
15. Fox BI. Information technology and pharmacy education. Am. J. Pharm. Educ 75(5), 86 (2011).
16. Williams J. The value of mobile apps in health care. Healthc. Financ. Manage 66(6), 96-101 (2012).
17. Payne KB, Wharrad H, Watts K. Smartphone and medical related app use among medical students and junior doctors in the United Kingdom (UK): A regional survey. BMC. Med. Inform. Decis. Mak 12, 121 (2012).
18. Boulos M, Wheeler S, Tavares C, et al. How smartphones are changing the face of mobile and participatory healthcare: an overview, with example from eCAALYX. Biomed. Eng. Online 10, 24 (2011).
19. Toh TW, Chui WK, Yap KL. Development of a virtual patient record mobile app for pharmacy practice education. Arch. Pharma. Pract 5(2), 66-71 (2014).
20. Wolters Kluwer Health. UpToDate®. Wolters Kluwer Health 2014.
21. Friedrich R, Merle ML, Peterson M, et al. The rise of Generation C: Implications for the world of 2020: Booz & Company (2010).
22. Fox Z. Forget Generation Y: 18- to 34-year-olds are now 'Generation C'. Mashable Website (2012).
23. The consensus of the Pharmacy Practice Model Summit. American Society of Health-System Pharmacists. Am. J. Health. Syst. Pharm 68(12),1148-1152 (2011).