The Appropriate Framework for Monitoring, Measuring and Evaluating the Performance of Quality in Healthcare: Systematic Review

Corresponding Author:

Zaadoud Brahim
Clinical Neurosciences Laboratory, University of Fez, Morocco
Telephone: +21652430099
E-mail:Zaadoud@gmail.com

Received: 20-Dec-2021, Manuscript No.ijocs-22-50274; Editor assigned: 24-Dec-2021, PreQC No. ijocs-22-50274(PQ); Reviewed: 06-Jan-2022, QC No. ijocs-22-50274(Q); Revised: 11-Jan-2022, Manuscript No. ijocs-22-50274(R); Published: 22-Jan-2022, DOI: 10.37532/1753-0431.2022.16(1).221

Abstract

Objective: Aims to review the literature in the field of performance measurement in health care, and to contribute to a broader discussion of performance measurement.

Methods: We conducted a thematic analysis of the international literature, identifying themes around the terms “Performance Assessment, Performance Measurement, Health Indicators, Health System Performance”. Studies concerning performance measurement in health in all settings were included. Studies before 2018 were identified from PubMed, Scopus, Web of Science, and Google Scholar.

Result: 37 articles are reviewed and a set of conceptual frameworks are analyzed, and results are interpreted following the seven areas of the conceptual framework: Fundamental questions in performance evaluation, Aims and objectives, Role and goals, Performance, Conceptual frameworks, Dysfunction of the health system, and Performance assessment. All areas of care are involved, health promotion, preventive and curative care.

Conclusion: Performance measurement is an essential component of quality improvement. However, certain dysfunctions have been identified: the lack of systematic evaluation of results, insufficient documentation, and the lack of evaluation of resources related to quality.

Keywords

Quality improvement; Health care; Performance measurement

Introduction

Improving population health outcomes depends on improving the quality of care. A wide range of QI interventions is applied around the world. On QI approaches, such as accreditation, Certification, and Total Quality Management (TQM), there is some evidence that these process measures of quality of care. However, the evidence is less convincing for improving patients’ health outcomes [1]. Quality of care measurement identifies areas for improvement, tracks improvements, and provides comparative information on health system performance for stakeholders and Performance measurement has been described as ‘the process of quantifying the efficiency and effectiveness of action’; whose system makes it possible to measure the efficiency and effectiveness of actions [2,3]. In these years, several frameworks for measuring health system performance have been proposed. The definitions of Health System Performance Assessment ‘HSPA’ are controversial. According to the World Health Organization ‘WHO’, health system performance assessment is a process of monitoring, evaluating, reporting, and reviewing the achievement of health system objectives. Unresolved issues include goal-setting processes, methods, choice of performance indicators, and treatment of health system effectiveness in the HSPA, the treatment of health system efficiency within ‘HSPA’, the treatment of equity, data gaps and inconsistencies [4].

The purpose of this paper is to undertake a systematic review of qualitative literature to identify conceptual performance measurement frameworks for health care and practical strategies for improvement and to synthesize the available literature on how performance is defined and measured in Healthcare Quality and contribute to broader discussions of performance measurement. To synthesize evidence of effects and conclude frameworks for measuring health care performance. Despite the efforts made, some questions remain unanswered, which the leaders have to give responses to propose a very good guide for designing the conceptual framework for performance monitoring and evaluation. What are the objectives of the conceptual framework? How will these conceptual frameworks be used? Do quality approaches improve health care performance, and what is the appropriate method for measuring the quality-of-care performance?

Methods

We seek to identify appropriate conceptual frameworks for the measurement of qualityof- care performance, to assess the feasibility and comparability of different models. Several strategies were used to ensure study rigour and quality of the results, including the use of an approved protocol; triangulation of data acquired from multiple sources over a period.

The PubMed, Scopus, Web of Science, Google Scholar searches were conducted using keywords such as performance assessment, performance measurement, health system performance, monitoring and evaluation. We performed our literature review following a five-step process [5]. First, we conducted a scoping study. Second, we identified relevant studies for our review in the literature. Third, we selected studies that met our specific selection criteria. Fourth, we read the selected articles and developed a dataset that included key variables and characteristics of each study. Fifth, we synthesized the extracted ideas. The searches covered the period from 2000 to 2018. The searches were carried out with great sensitivity to extract all related and attainable studies. After the search, all obtained scientific resources were reviewed by researchers, and the scientific literature related to the fields of study was extracted [6]. We also conducted a thematic analysis of the international literature, identifying themes around; The fundamental questions in performance evaluation, aims, and objectives, role, and goals of primary care, the performance of primary care, the conceptual framework, dysfunctions of the health system, performance assessment [7]. According to Braun, et al. Thematic Analysis (TA) is a method for identifying and analyzing patterns of significance in a data set [8]. Abstracts and article titles were analyzed using an exploration tool. The result of this analysis is a set of concept maps, where the frequency of concepts, the hierarchical order of appearance, and the proximity between concepts are visualized. Each thematic part is formed based on the connectedness of the concepts and is highlighted by the most relevant concept in terms of frequency and connections (relational analysis).

Data organization began with the management of coded files. Coded files were created to represent information related to the residential aged care facility and the research participant at that site. We were Examining the full data set as a precursor to developing a coding frame to create a conceptual tool with which to classify, understand and examine the data, then we were checking the reliability of the coding frame, Once a coding frame has been devised and reliability checked, the entire data set must be coded a new, and we were analyzing the data using a data analysis package [9].

Criteria for considering studies in the review. All peer-reviewed publication types including literature reviews, quantitative studies, qualitative studies, mixed-methods studies, and discussion papers were included in the review (Table 1).

Table 1: Provides a summary of the inclusion and exclusion criteria.

Search strategy

We have considered in the review the publications focused on health performance assessment. The electronic databases were used to search for relevant articles: Medline, Cochrane. The following search terms were used: performance measurement, performance evaluation, performance assessment and health care quality (Table 2).

Table 2: Shows the search strategy in the databases.

Limits

Were used to include only articles written in the English language and published between 2000- 2018.

Statement of Ethics

The paper is exempt from ethical committee approval because it did not involve people or animals.

Discussion

Literature base. The database search yielded 1231 articles and 14 Other items are identified by other sources, of which 628 were excluded due to duplicates and selection criteria. 617 articles screened by abstracts and titles. 324 articles removed after review of abstracts and titles for relevance were further excluded; 256 articles were removed after scrutiny based on the inclusion criteria, leaving 37 articles for inclusion in the review, the literature search chart is shown in Figure 1.

Hierarchy of evidence

The included studies comprised discussion papers, literature reviews, mixed-methods studies, case studies, observational studies, audits, and qualitative studies. Shows the coding assigned to each of the individual articles included in the systematic review, summarizing the evidence sources for each of the identified categories (Table 3).

Table 3: Characteristics of studies included in the systematic review.

We reviewed most of the frameworks used at the international level. 37 articles are reviewed, a set of conceptual frameworks are analysed, and results are interpreted following the six areas of the conceptual framework: fundamental questions in performance evaluation, aims and objectives, role and goals, performance assessment, conceptual frameworks, and dysfunctions of the health system.

Fundamental Questions in Performance Evaluation

Several important questions raised by various researchers are a very good guide for designing the conceptual framework for performance monitoring and evaluation in health care [10]. Three research questions have been derived from the purpose; How and why are performance measurement systems being developed in health care services? What are the issues that hinder the development of health performance measurement systems? What enabling factors can be identified in the development of performance measurement systems in health care services [11]?

Aims and Objectives

For Christopher and Murray, the aim of ‘HSPA’ is to promote strategic accountability for health system actions. Various frameworks have included goals related to health, quality, consumer satisfaction, allocative efficiency, technical efficiency, cost containment, political acceptability, health inequalities, and financial sustainability [12]. According to Smith, et al. objectives might include: setting out the goals and priorities for a health system; acting as a focus for policymaking and coordinating actions within the health system; measuring progress towards achievement of goals; informing public debate on the health system amongst stakeholders and citizens; and promoting transparency and accountability to citizens and other legitimate stakeholders for the way that money has been spent [4]. Three strategic objectives should be met by the health system performance assessment process: to inform health system performance and support policy planning, and to provide a transparent and accountable vision; and to monitor the health system performance over time [13]. For Urbanoski, five healthcare system strengthening performance areas are identified: implementation of person-centered integrated service delivery models; patient and family engagement in healthcare; addressing inequality in access to quality healthcare for vulnerable populations; better outcomes through continuous quality improvement; and reduction of ineffective health spending and waste in healthcare [7].

Role and Goals

To pursue its goals, primary care should guarantee the provision of services that are [14]:

• Universally accessible

• Integrated

• Person-centered

• Comprehensive and community-oriented

• Meeting different health needs and these services must be provided

• Sustainable partnership

• Oordination

• Continuity of care

In the Organization for Economic Co-operation Development ‘OECD’s work to measure and improve health care system performance, health care quality is understood to comprise three dimensions: effectiveness, safety, and patient-centeredness (or responsiveness). These dimensions are applied to key stages of the care process: preventive care, acute care, chronic care, and palliative care [15]. According to Jenna Evans, the Prospects for future investigation has to: Understanding the links between system performance and its contributing elements; Evidence-based metrics which map service gaps to system outcomes; and, Process and outcomes metrics at the system level which evaluate the quality of “patient journey” [16].

Performance Assessment

Performance is defined as a multidimensional concept that integrates the dimensions of safety, efficiency, quality of services provided in a relevant and timely manner, responsiveness, and equity [17,18]. Performance measurement is about assessing and reporting on how well the health system is achieving its goals [19]. The process of quality improvement is very beneficial for the health care system. Appropriate evaluation remains the fundamental element for eventual improvement as indicated by Starfield, helps to better understand the mechanisms of operation as well as the potential benefits and risks of health care systems, to measure their impact and their level of adequacy. Performance measurement in the healthcare sector is an important tool for improving service quality [20,21]. However, it is limited by the lack of understanding of how these measures should be conducted [21]. In general, the information system allows the monitoring of the performance of the care systems [20]. It also shows the difficulty deriving from implementing a top-down performance management system enforced by law [22]. In addition, the research confirms the persistence of two traditional problems of the Italian public sector. One refers to the limited attention given to the role and importance of performance management [23]. The second is the false conviction that changes can be introduced by law [24].

Conceptual frameworks

Performance measurement frameworks seek to determine the activities and success of a program’s strategy and provide insights for future improvements [25]. Multiple performance frameworks have been designed to assess health systems, health service delivery organizations[26-28] and health quality [29,30]. The most Health System Performance Assessment ‘HSPA’ in the word is shown in Table 4. The three first frameworks represent the theoretical foundations of the fourth conceptual framework that follow.

Table 4: Comparative table of quality of care measurement frameworks based on performance criteria.

The performance domain is divided into two main components: structural and performance. The structural domain describes the health care system, practise context and organization of the practice in which any primary care organization operates. The performance domain includes features of health care service delivery and technical quality of clinical care [31]. Reasons given for developing such a framework are as follows: (a) to define the scope and dimensions of measurement; [32,33]; (b) to help align the performance measurement system with other policy priorities and financial exigencies [34]; (c) to provide a clearly defined vision to encourage buy-in by clinicians and consumers; and (d) by involving potential end-users of the data in the design of the framework, to ensure its future usefulness [35]. A conceptual framework encompassing multiple domains and with balanced representation across the structure, process and outcome indicators are considered to be a key element of health reform over time [36,37]. The ‘Euro HOPE’ project has been developed using two different approaches. The first approach relies on developing a coherent conceptual framework for information collection, analysis and dissemination. Another approach assembles readily accessible data, often the by-products of existing national data collection, such as hospital discharge registers, as well as work that has been done for other purposes [38]. Three interconnected tiers were determined, including health status, non-medical determinants of health and a tier representing health promotion and the health system. The performance of the health system is grouped into four main dimensions: quality, accessibility, efficiency, and sustainability. Quality of care is further subdivided into five sub-dimensions (effectiveness, appropriateness, safety, patient-centeredness, continuity). Equity, the fifth dimension, is a transversal dimension, which is presented across all tiers [13].

Dysfunctions of health system performance assessment

Health systems have dysfunctions and multiple challenges that can be categorized under five broad themes: governance and leadership, organization, health information and information system, financing, and human and material resources. Mannion and Braithwaite [39] attempted to classify dysfunctional consequences of national performance measurement systems into four groups: poor measurement itself, misplaced incentives and sanctions, breach of trust as intentional bullying and gaming, and politicization of performance systems, and politicization of performance systems, and provide four categories of adverse consequences: poor measurement, misplaced incentives and sanctions, breach of trust, and politicization of performance systems. Performance indicators are employed for four basic functions: facilitating accountability; monitoring health care systems and services as a regulatory responsibility; modifying the behavior of professionals and organizations at both a macro (population) and micro (patient) level; and forming policy initiatives [40]. According to Mainz [41] some dysfunctions were revealed: a lack of systematic outcome assessment; a lack of documentation; a lack of resource evaluation related to quality for specific diseases; persisting variations among providers in care for similar patients. Seven dysfunctional consequences were identified: misrepresentation of data by hospitals; increased anxiety and stress among hospital employees; tunnel vision; financial pressures on poorly graded hospitals; incentives to purchase unnecessary equipment; erosion of public trust; and restricting access to hospital services by patients. These were caused by the way the grading system was implemented: poor standards of audit; how the audit process was conducted; and the timing of audits. The pay-forperformance element of the grading system and the focus on structural aspects in the standards improved in grading particularly difficult for those hospitals that had been assessed as underperforming [42].

Limits

The limitations of our study are

• The difficulty of being exhaustive given the large number of conceptual frameworks that deal with quality of care

• The difficulty of assessing the reliability and validity of qualitative studies

• The comparability of results between countries remains difficult given the difference in performance levels between countries, especially for developed countries

The implications of this study

We hope that our findings will be useful to a wide range of stakeholders, increase the likelihood of effective and efficient use of resources, Resources and organization could be improved through the implementation of quality improvement guidelines, Evaluation of the achievement of results could benefit from rapid feedback. Promote interventions of proven usefulness and outlaw ineffective practices. Improved quality of care has been reported by accreditation and certification bodies around the world and hospital performance has been increased through the use of performance measurement frameworks.

Conclusion

Performance measurement is the key element in attempts to improve the quality of health care. Performance measurement must include the use of outcome and process measures. However, some shortcomings were identified: lack of systematic outcome evaluation; lack of documentation; lack of evaluation of quality-related resources for specific diseases; and persistent variation among providers for similar patients.

Acknowledgement

The authors would like to thank the two reviewers whose commitment supported the outcomes achieved.

Conflicts of Interest

There is no conflict of interest.

Funding Sources

There is no sources of funding

Author Contributions

All authors are contributing to writing paper.

References

1. Zeng W, Gheorghe A, Nair D. Health System Level Approaches to Addressing Quality of Care in Low and Middle Income Countries, RBF Health (2016).
   [Google Scholar]
2. Sorian R. Measuring, reporting, and rewarding performance in health care. Commonwealth Fund, Commission on a High Performance Health System, (2006).
   [Google Scholar]
3. Neely A, Gregory M, Platts K. Performance measurement system design: A literature review and research agenda. Int J Oper Prod Manag 25, 1228-1263 (2005).
   [CrossRef]             [Google Scholar]
4. Peter S, Health Systems Performance Assessment. WHO, (2014).
   [Google Scholar]
5. Franco Santos M, Lucianetti L, Bourne M. Contemporary performance measurement systems: A review of their consequences and a framework for research. Manag Account Res 23, 79-119 (2012). [CrossRef]            [Google Scholar]
6. Jahanmehr N, Rashidian A, Khosravi A, et al. A conceptual framework for evaluation of public health and primary care system performance in Iran. Glob J Health Sci 7, 341 (2015).
   [CrossRef]            [Google Scholar]
7. Urbanoski K. Strengthening performance measurement for mental health and addiction in Ontario (2017).
   [Google Scholar]
8. Braun V, Clarke V. Using thematic analysis in psychology. Qual Res Psychol 3, 77-101 (2006).
   [Google Scholar]
9. Oyebode J. Qualitative Research Methods in Mental Health and Psychotherapy: A Guide for Students and Practitioners Edited by David Harper & Andrew R. Thompson. Wiley-Blackwell pp.272 (2011). [CrossRef]           [Google Scholar]
10. Veillard JE, Garcia-Armesto S, Kadandale S, Klazinga N. International health system comparisons: from measurement challenge to management tool. Performance measurement for health system improvement: experiences, challenges and prospects 641-672 (2009).
    [CrossRef]            [Google Scholar]
11. Kollberg B. Performance measurement systems in Swedish health care services (Doctoral dissertation, Institutionen för ekonomisk och industriell utveckling) (2007).
    [CrossRef]             [Google Scholar]
12. Murray CJL, Frenk J. A framework for assessing the performance of health systems. Bull World Health Org 78, 717‑731 (2000).
    [Google Scholar]
13. Vrijens F, Renard F, Jonckheer P, et al. The Belgian Health System Performance Report 2012: Snapshot of results and recommendations to policy makers. Health Policy 112, 133-140 (2013).
    [CrossRef]         [Google Scholar]
14. De Maeseneer J. European expert panel on effective ways of investing in health: Opinion on primary care. Prim Healthc Res Devlop. 16, 109-110 (2015).
    [CrossRef]             [Google Scholar]
15. Health System Level Approaches to Addressing Quality of Care in Low and Middle Income Countries
16. Evans J, Wodchis W, Quigley L, et al. Where is the ‘system’ in health system performance? Curr efforts Future Prospects (2011).
17. Carroll JG. Crossing the Quality Chasm: A New Health System for the 21st Century. Qual Manag Healthc 10, 60‑61 (2002).
    [CrossRef]             [Google Scholar]
18. Veillard J, Champagne F, Klazinga N, et al. A performance assessment framework for hospitals: The WHO regional office for Europe PATH project. Int J Qual Healthc 17, 487‑496 (2005). [CrossRef] [Google Scholar]
19. Smith PC, Mossialos E, Papanicolas I, et al.  Principles of performance measurement in Performance Measurement for Health System Improvement Experiences, Challenges and Prospects, (2009).
    [Google Scholar]
20. De Vos M, Graafmans W, Kooistra M, Meijboom B, Van Der Voort P, Westert G. Using quality indicators to improve hospital care: A review of the literature. Int J Qual Healthc 21, 119-129 (2009). [CrossRef]             [Google Scholar]
21. Elg M, Broryd KP, Kollberg B. Performance measurement to drive improvements in healthcare practice. Int J Oper Prod Manag. (2013).
    [CrossRef]            [Google Scholar]
22. Micheli P, Neely A. Performance measurement in the public sector in England: Searching for the golden thread. Public Adm Rev 70, 591-600 (2010).
    [CrossRef]             [Google Scholar]
23. Martin J, Spano A. From performance management to strategic local government leadership: Lessons from different cultural settings. Public Money Manag 35, 303-310 (2015).
    [CrossRef]             [Google Scholar]
24. Spano A, Aroni A. Organizational Performance in the Italian Health care Sector. InOutcome-Based performance management in the public sector, Springer, Cham,  pp. 25-43, (2018).
25. Kalinichenko O, Amado CAF, Santos SP. Performance Assessment in Primary Health Care: A Systematic Literature Review, University of Evora, CEFAGE-UE (Portugal), (2013).
    [CrossRef]             [Google Scholar]
26. World Health Organization. Monitoring the building blocks of health systems: A handbook of indicators and their measurement strategies. World Health Organ, (2010).
    [Google Scholar]
27. Bradley EH, Pallas S, Bashyal C, et al. Developing strategies for improving health care delivery: Guide to Concepts, Determinants, Measurement, and Intervention Design, (2010).
    [Google Scholar]
28. Geue C, Lewsey J, Lorgelly P, et al. Spoilt for choice: Implications of using alternative methods of costing hospital episode statistics. Health Econ 21, 1201-1216 (2012).
    [CrossRef]            [Google Scholar]
29. Kelley ET, Arispe I, Holmes J. Beyond the initial indicators: Lessons from the OECD health care quality indicators project and the US national healthcare quality report. Int J Qual Healthc 18, 45-51 (2006).
    [CrossRef]             [Google Scholar]
30. Bhattacharyya O, Mossman K, Ginther J, et al. Assessing Health Program Performance in Low-and Middle-Income Countries: Building a Feasible, Credible, and Comprehensive Framework. InPrivate Sector Entrepreneurship in Global Health, University of Toronto Press, 129-163 (2019).
    [CrossRef]            [Google Scholar]
31. Hogg W, Rowan M, Russell G, et al. Framework for primary care organizations: the importance of a structural domain. Int J Qual Healthc 20, 308-313 (2008).
    [CrossRef]             [Google Scholar]
32. Ten Asbroek AH, Arah OA, Geelhoed J, et al. Developing a national performance indicator framework for the Dutch health system. Int J Qual Healthc 216, 65-71 (2004).
    [CrossRef]            [Google Scholar]
33. Arah OA, Westert GP, Hurst J, et al. A conceptual framework for the OECD health care quality indicators project. Int J Qual Healthc 18, 5-13 (2006).
    [CrossRef]             [Google Scholar]
34. Berwick DM, James B, Coye MJ. Connections between quality measurement and improvement. Medicalc 30-38 (2003).
    [Google Scholar]
35. Collopy BT. Clinical indicators in accreditation: An effective stimulus to improve patient care. Int J Qual Healthc 12, 211-216 (2000).
    [CrossRef]            [Google Scholar]
36. Adair CE, Simpson E, Casebeer AL, Birdsell JM, Hayden KA, Lewis S. Performance measurement in healthcare: part I–concepts and trends from a state of the science review. Healthc Policy 1, 85 (2006). [Google Scholar]
37. Braithwaite J, Hibbert P, Blakely B, et al. Health system frameworks and performance indicators in eight countries: A comparative international analysis. SAGE Open Med 5:2050312116686516 (2017). [CrossRef]    [Google Scholar]
38. Hakkinen U, Iversen T, Peltola M, et al. Health care performance comparison using a disease-based approach: the EuroHOPE project. Health Policy 112, 100-109 (2013).
    [CrossRef]            [Google Scholar]
39. Mannion R, Braithwaite J. Unintended consequences of performance measurement in healthcare: 20 salutary lessons from the English National Health Service. Int Med J 42, 569-574 (2012).
    [CrossRef]             [Google Scholar]
40. Bankauskaite V, Dargent G. Health systems performance indicators: methodological issues. Presupuesto y gasto publico 49, 125-137 (2007).
    [Google Scholar]
41. Mainz JA. Developing evidence-based clinical indicators: a state of the art methods primer. Int J Qual Healthc 15, 5-11 (2003).
    [CrossRef]             [Google Scholar]
42. Aryankhesal A, Sheldon TA, Mannion R, et al. The dysfunctional consequences of a performance measurement system: the case of the Iranian national hospital grading programme. J Health Serv Res Policy 20, 138-145 (2015).
    [CrossRef]              [Google Scholar]