Step 2. Analysis of actual practices
This step in the implementation of change model aimed to assess current
care in targeted settings by identifying and using measurement
indicators.30 Indicators can be divided by structure,
process or outcomes and are measurable elements that help evaluate
quality of care and identify where changes are most
needed.31
Measuring outcomes for mental health problems is a worldwide
challenge32 and Canada’s health care system is no
exception.33 Compared to other fields of care, many
mental health quality measures are not necessarily available in
electronic medical records or manageable to use for research and quality
improvement initiatives.32 Consequently, we conducted
an analysis of actual practices with qualitative data sources instead of
analyzing actual performance using quality indicators.
Data sources for this step of the process were collected from recruited
participants in each FMG (Table 2) through face-to-face interviews (≈ 45
to 60 minutes) and non-participatory observation of a nurse-patient
encounter. We also retrieved and examined relevant documents on
professional activities and collaborative care: documents on specific
programs for people with LTCs or CMDs, educational materials, clinical
nursing mental health assessment canvases, and health care professionals
referral forms.
The data collection period lasted from December 2018 to April 2019.
Interviews with primary care providers and nurses were the first data to
be collected (n=25), followed by interviews with patients (n=7) and
observations (n=7). Field notes were written after each interview and
observation to describe first impressions and links to previous
interviews/observations. The data from each FMG were separately analyzed
and interviews and observations were audio-recorded and transcribed by a
professional. Finally, two authors (EE, PR) also listened two to three
recordings to validate whether interviews with primary care providers
and nurses were capturing all elements of the practice. Analysis of
actual practices was done in two main stages.
First, we documented current collaborative practices in each FMG,
including nursing activities and challenges in the delivery of
collaborative care to patients with CMDs and LTCs. We conducted (AG,
JBH) a thematic analysis of interview
transcriptions,34 with NVivo 12.0, and then
schematized results to illustrate the collaborative care process and
relationships between professional activities (see Additional File 1 for
an example).
Second, we assessed the level of achievement and quality of professional
activities involved in the CCM using two analysis tables built from the
previously mentioned AIMS Center tool.26,27 These two
tables respectively summarize the main activities of the CCM and the
care manager’s activities (see additionnal files 2-3). Throughout this
stage, tables were updated.
For each activity, the first author identified the professional who
performed the activity and how it was done, the level of achievement,
the resources and competencies required as well as any barriers to
carrying out the activity. Triangulation of patient interviews,
observations of nurse-patient encounter and documents from each FMG were
integrated to enrich the assessment of each activity. The analysis was
done in an iterative way and results obtained during the quality
assessment were validated with the author who conducted a thematic
analysis. Finally, preliminary results were presented to all members of
the research team to validate results and to make decisions about how to
deal with activities that were more difficult to assess. The team
decided to ask for the opinion of local working groups regarding the
assessment of these activities (step 4).
Discussion on lessons learned. Analysis of actual
practices using qualitative data (verbatim from observations and
interviews, schemas, and documents) provides a thorough understanding of
current collaborative care and how professional activities are
delivered, helping to identify potential activities that could be
improved (targets for change). A limitation of this approach was the
amount of data to collect and to manage in a short period of time. In
addition, in the context of our study, we didn’t use quality indicators
measuring collaborative mental health care to assess the actual
performance as recommended in the Grol & Wensing model. Indeed, using
quantitative data from the registry would have helped to better
prioritize targets for change.
Assessing all professional activities involved in the CCM can be a time
consuming challenge. The CCM includes many activities that are sometimes
harder to evaluate because of their subjective nature or require
specific and valid measurement tools adapted for such an analysis. For
instance, patient’s engagement in their care was more difficult to
assess. Consequently, to measure the level of engagement of patients it
may be relevant to combine qualitative data sources with a valid
questionnaire. For future research, an example of a valid tool that
might help to better assess patients’ engagement in their care, is thePatient Assessment of Chronic Illness Care
(PACIC). 35 This is a validated self-reported tool to
measure the extent to which patients with LTCs receive care that is
congruent with Wagner’s Chronic Care Model.35 However,
to be more specific in assessing satisfaction with and engagement in
both mental health and physical care, it might be useful to adapt the
PACIC.
Furthermore, the Assessment of Chronic Illness Care (ACIC) might
also be a useful tool to assess the overall level of achievement and
quality of professional activities.36 The ACIC was
built to help identify areas for improvement in the care of people with
LTCs.36 The ACIC covers many principles and
professional activities of the “Patient-Centered Integrated Behavioral
Health Care Principles & Task Checklist” developed by the AIMS Center,
but lacks specificity for mental health treatment and services. For
further study, it might be interesting to adapt the ACIC by combining
elements of this previous version that are specific to mental health
care (e.g. psychiatric consultation or psychotherapy).