A total of 46 psychotherapy providers participated in the electronic survey, including psychologists (n = 28), licensed clinical social workers (n = 17), and a non-licensed social worker (n = 1). Of the participants who shared information on their age, 37% (n = 17) were aged 25–39, 54% (n = 25) were aged 40–59, and 2% (n = 1) were 60+. Survey participants were asked, “For what percentage of your patients do you employ measurement‑based care?” Among the 40 providers who indicated that they use measurement‑based care with Veterans in their clinical practice, reported usage with patients ranged from 9% to 100%, with a mean of 79.45% and a median of 82%.
Results presented here are organized by perceived benefits, drawbacks, and educational needs related to MBC. The identified sub-themes provide insight into the nuances of implementing MBC and integration of the practice into clinical workflows. Quotes are reported verbatim (written).
Benefits
Many benefits of MBC were reported by participating providers and are organized across three subthemes: symptom awareness and monitoring, guiding clinical decision making, and facilitation of care delivery.
Symptom awareness and monitoring
Twenty-six participants (56.5% of the total respondents) reported the benefit of symptom awareness and monitoring, including identification of symptoms, illumination of severity of symptoms, and the ability to easily track symptoms over time. Eleven participants (23.9% of total respondents) reported that the assessments help identify symptoms that were not revealed during clinical interactions, with one provider sharing, “[MBC] Helps to guide the treatment and open a forum for discussing specific symptoms that may otherwise go unnoticed” [ID007]. Another provider explained, “It helps determine what symptoms may be more prevalent in a patient’s situation. The Veterans are usually not aware of the degree of their depression for instance until they complete the PHQ-9” [ID0036]. Another noted, “[MBC] helps you to be thorough and not miss symptoms in your assessment of current concerns” [ID0040].
Insight into symptom severity can be helpful in ongoing treatment planning, and as noted in the survey responses, Veterans may not be aware of the severity of their symptoms. One provider spoke to the benefit of quantifying symptom severity, expressing that MBC “Puts symptom severity into an easier to understand number” [ID0028].
One provider summed up the benefits of symptom awareness and the subsequent opportunity for guidance in treatment decision-making, stating, “Significant benefits are diagnostic clarity, current snap shot of ‘mental coordinates’, opportunity to discuss current factors, see improvement, make adjustments, understand their thought process and why they answer the way they did, provides feedback and other data such as habit and automatic thought process in responses if there are not any changes” [ID0035].
Guiding clinical decision making
MBC can guide clinical decision-making through objective tracking of symptom change (or lack thereof), identifying progress toward care goals, and informing treatment and, ultimately, discharge planning.
Providers noted benefits of collecting assessment data, indicating that when first administered the assessment results give a baseline from which the response to treatment can be monitored. One provider shared, “It allows you to better understand the patient and gives you a baseline at which to measure the efficacy of treatment” [ID0041]. As data collection continues over the course of treatment, progress can be measured and viewed in a more objective manner. According to one provider, “It helps provide additional data to patient and clinician’s subjective experience and judgement to determine the effectiveness in treatment” [ID0033]. This objective understanding of the symptom change was noted as important by participating providers.
Providers spoke about how MBC’s provision of insights into nuanced symptom changes is valuable for treatment planning by helping to refocus goals and priorities and support patients’ ability to identify and act on areas of concern. According to one provider, MBC “provides a structure for involving the patient in a discussion regarding progress (or lack of)” [ID0002]. Other providers spoke about how MBC is good for identifying unmet needs, noting MBC helps with “tracking progress relative to goals [and to] identify sources of support needed that may not be being addressed by the current therapy” [ID0009], and “For me and the patients to track their progress over time and see when therapy is making a difference and if not, where they may be stuck” [ID0010].
Respondents reported how the psychoeducation component of MBC is a valuable tool in helping patients to better understand their diagnoses and symptom severity. “Given the need for episodic care and the frequency with which Veterans present with complex, chronic problems, I think use of MBC can help both Veteran and provider feel clear and satisfied about progress in treatment. I work almost exclusively with patients with chronic issues because of my specialty and use of measures helps educate patients that fluctuations in symptoms do occur and help them think about quality of life beyond symptom focus” [ID0039].
Outside of the psychotherapy realm, one provider spoke about MBC’s value in psychotropic medication management. “For Veterans that are taking medications, if working, then I want to see their symptoms to be moderate to mild vs. moderate to severe. Then it can indicate a need to review medications to either increase dosage, maintain the dosage or even lower dosages at times” [ID0036].
Facilitation of care delivery
Providers conveyed that MBC facilitates engaging Veterans in treatment, supports rapport building, saves time and helps Veterans move toward acceptance of a clinical diagnosis. Helping Veterans feel more engaged in their own treatment was noted to be a benefit among many providers. Providers articulated important insights into their perceived value of MBC saying, “completing the assessments during session strengthens partnership” [ID043], that MBC “… helps patients feel more involved in their own care” [ID0004], “validates a Veteran’s experience” [ID0011], “[provides] better education about mental health conditions” [ID0012], and “can help people improve their self-monitoring and self-awareness of different symptoms and make links between symptoms and an overall diagnosis.” [ID0038].
Time was discussed in the report of benefits of MBC. MBC was reported as being a time-saver by some, but not all providers. One provider shared “The biggest benefits of MBC are its use during intake assessments (almost as a ’short cut’ during these long assessments to help with time management)” [ID0006]. Time was also discussed in the context of being able to identify problem areas and make change right away, “Being able to track progress and act on areas of concern right away is invaluable. This is one of the other major pros of MBC” [ID0019].
In addition to the clinical benefit of MBC, some providers described adjunctive benefits, specifically how MBC processes can act as a facilitator of care. One beneficial factor associated with successful uptake of clinical interventions is the ease with which individual technological systems (i.e. BHL Touch and VHA’s electronic health record (EHR)) work cohesively in support of the intervention. Some providers discussed the ease with which MBC can be integrated into video telehealth visits with one provider indicating that BHL touch provides an advantage in MBC, stating “Since I started using BHL touch, I no longer feel I have a disadvantage to face-to-face providers (who can hand a form to a client, have them fill it out, then look it over later)” [ID0006]. Providers also pointed out the added benefit of the text message sent to collect symptom data acting as a secondary appointment reminder.
Drawbacks
Providers’ reports on the drawbacks of MBC spanned three categories: assessment completion and self-report bias, structural barriers to provider engagement, and obstacles to evidence-based practice model adherence.
Assessment completion and self-report bias
Providers reported a drawback of MBC is that Veterans do not always complete the assessments, noting the circumstances in which this happens and a variety of possible reasons. Although reasons for non-participation are unique to each Veteran, providers cited a variety of perspectives as to why some Veterans may not complete assessments, including MBC not aligning with a Veteran’s goals, provider concerns about MBC having a negative impact on a Veteran, the possibility of self-report bias or response distortion, discouragement from stagnant scores or negative trends, and fear of impact on VA disability benefits. Incomplete assessments were reported to be expressly difficult in digital settings, with one provider describing how, “when using during a telehealth visit- it can be hard to get compliance when asking them to do it prior to the session” [ID0026].
Providers noted that perhaps Veterans are inconsistent in completing assessments due to the Veteran’s perspective on the value of MBC. Some providers reported that the number of instruments used and the frequency of assessment places a burden on some Veterans, leading to MBC feeling “clunky” and “less personalized for patient care”. One provider shared “Patients won’t always respond or get frustrated with doing them.” [ID0046]. The value of MBC is not always realized immediately among Veteran patients, making it difficult to get “Veteran buy in” to see the benefits.
Concerns about negative psychological impacts on Veteran patients were communicated.
The psychological impacts described included frustration, demoralization, discouragement, and anxiety provocation. One provider shared their experience that “For some Veterans, it reinforces how poorly they are doing to fill the questions out week to week…and it raises anxiety to fill out the questionnaire about how their anxiety is. There are cases where the measurement process really bothers Veterans. In those cases, I make adjustments rather than disqualify them from an evidence-based practice (assuming there are not other issues such as non-compliance from homework or high resistance to the structured session format)” [ID0006]. Further speculation on how MBC may impact the Veteran was conveyed by one respondent saying “Some people communicate that seeing the numbers can elicit shame or discouragement that they are not improving, even though people in therapy tend to get worse before they get better” [ID0038].
Providers conveyed apprehension in accuracy of reported symptom data, due to self-reporting bias and reporting distortion. In some cases, inaccurate symptom data may be due to questions not being specific enough or fatigue in filling out the assessments. One provider expressed how MBC can create difficulties in the session, noting that issues arise “When Veterans want to add to the questions because they are not specific enough or when Veterans elaborate too much on each question or get frustrated by questions (especially in longer assessments, like BASIS 24 and IMRS” [ID0022].
Some providers posited that response distortion may be influenced by a desire to seem better than they are or by a fear of reduction in benefits due to improvement. In these cases, providers indicated reticence to use MBC. Regarding reporting that may minimize appearance of symptom burden, one provider stated, “Some Veterans ‘fake good’ and engage in more denial and minimization of symptom, so the numbers are reflective of what they report in sessions.” [ID0038].
Further concern about self-report bias and response distortion was raised around the issue of suicidality. “Patients learning how to respond to the measure to avoid having difficult conversations or hospitalization. For example, a Veteran might learn that if they endorse having suicidal thoughts, they will get follow up questions that might lead to being hospitalized. Rather than saying, ‘I need to have this conversation,’ they choose to deny having any suicidal thoughts. Alternately, some Veterans tell us what they think we want to hear rather than how they actually feel because they want to be perceived as doing well in therapy or on a test (yes, scholastic assessment carries over for a long time!)” [ID0041].
Alternative ideas were offered to explain response distortion, with one provider sharing, “Some patients who are therapy dependent react negatively to the positive outcomes. Some patients learn to answer negatively because they think that will keep them in therapy longer or look better for disability claims” [ID0043]. Leaning into the concerns about disability benefits, one provider responded, “Possibly some Veterans will over-report out of concern that their service connection could be impacted (though I think this is a minority in actuality)” [ID0007.] Another stated, “some Veterans are hesitant to answer honestly for fear of reduction in disability/benefits time” [ID0037].
Structural barriers to provider engagement
Providers cited structural barriers to using tMBC, including concerns over asynchronous collection of time sensitive data (i.e., suicidality screening), functional and technical constraints of software used to support tMBC, and overall time constraints.
Three instances were offered as examples of problems that arise due to asynchronous MBC. The first concerned timing and coordination with the EHR scheduling software. Providers noted that confounds arise when a Veteran completes their assessments ahead of time but then cancels or misses the appointment, as their assessment data then lacks an associated clinical visit and cannot be uploaded. The next concern about asynchronous assessment also related to cancellation but referenced the burden on the provider to keep track of what forms were sent out if a Veteran cancels and reschedules an appointment. Finally, results from an asynchronous assessment of symptoms may trigger the need for emergent follow up, which is problematic when follow up contact with the patient is unsuccessful. Clinician responses to a positive suicide screen gathered during asynchronous assessment may need to be dynamic, as the symptom measures are not always sensitive to acute versus chronic suicidal ideation, both of which can occur in Veterans accessing care. Providers conveyed the need to navigate legal and ethical clinical practice standards when working with individuals who report suicidality.
Various functional and technical constraints were identified by participating providers. The library of assessments offered within BHL Touch was noted by several providers to lack assessments that are needed for some patients. Some providers still utilized assessments that were not in BHL Touch, with one respondent stating, “I have been finding other ways to readily integrate other, more relevant measures into my practice that are not a part of the VA catalogue. But this does create issues with time in session, especially for Veterans who are being seen virtually and cannot fill out the measure in session” [ID0033].
One provider described workflow inefficiencies when standardized assessments are not available through BHL Touch, noting that alternate methods (e.g., email or patient portal) often reduce response rates and require using valuable session time to complete measures that could otherwise be done independently.
There are times when something is not on BHL touch, and in those cases, I have to e-mail or MHV [My HealthEVet, patient portal] the info (which then I typically don’t get anything back) or I have to complete the assessment verbally (which takes session time I prefer to spend doing something non-standardized). For example, I am a provider in an OCD study which asks for the Y-BOCS to be filled out 1x/2 sessions. I have to e-mail rather than have a text sent, and the Veteran tends to fill it out in session with me rather than getting it done ahead of time. When I use BHL touch for the same Veteran to complete a PHQ and PCL, they typically get it done outside of session [ID0006].
Technical issues reported were related to software malfunctions, accessibility (if a Veteran’s device does not allow them to see the provider or screen during a video visit), and the impact of technical failures on the provider and patient relationship. One provider expressed, “When it doesn’t work, it can be very frustrating (this can be especially important during an intake assessment when there may be high suspicion or nervousness coming into the appt)” [ID0006].
Concern about technological proficiency among Veterans was reported as a barrier to MBC. Some providers reported a lack of confidence in their Veteran-patients’ ability to access and navigate the technology used by VHA in MBC (i.e., BHL Touch). On the other hand, one provider spoke about how providers need to be able to respond to Veteran’s preferences, “in this day and age the patients dictate the level of technology they are comfortable with yet as providers we need to be familiar with all options” [ID0036].
Time as a barrier
Issues around time as a barrier to uptake of MBC were reported by 28% (n = 13) of respondents. When queried about the potential drawbacks of MBC, some providers simply responded with one word, “time”. Others shared more nuanced perspectives, noting that the time required to complete surveys during sessions and to review the data for discussion and treatment planning impacted adoption. One provider explained, “the biggest problem I have is time required if the assessments are completed in session. BHL is a great tool, but Veteran compliance varies” [ID0002]. Time as a barrier crossed over with reported Veteran engagement issues, with one provider noting, “it does take session time and often patients aren’t as interested in that as other things, so it feels like a struggle to make them do MBC stuff” [ID0039].
Another provider suggested that time constraints are related to reviewing and planning, stating a barrier to adoption is “needing the time to review the assessments. Needing the time to adjust the course of overall treatment when assessments suggest so” [ID0024].
The concern for time was not universal, with one provider stating “I don’t know of any drawbacks. Time is really not an issue when you get used to it” [ID0019]. Despite that provider’s enthusiasm, the sentiment was not universally shared and participating providers offered important insights into the drawbacks of MBC that impact its usability in clinical care.
Measure validity and therapeutic value
Provider reports on barriers to fidelity to MBC as an evidence-based practice included not always understanding the therapeutic value of MBC, the predictive validity of specific measures and their sensitivity to change over the course of treatment, and how to respond clinically when Veteran’s scores did not indicate clinical improvement. For example, providers relayed their perception that scores are not necessarily always accurate, that “measures don’t always accurately capture change” [ID0020]. Some providers noted that the since the assessments are symptom-focused, they do not assess treatment goals that are non-symptom related, such as interpersonal or functional goals, with one provider sharing, “[MBC] does not always capture important qualitative elements of change (e.g., sense of confidence, improvement in relationships, sense of social connectedness, movement toward personal life goals like applying for jobs or school). Some therapy goals may be more interpersonal and less symptom focused, or some symptoms may be expected to demonstrate less overt change that can be tracked easily, such as personality disorders or SMI [serious mental illness]” [ID0041].
One provider indicated that sometimes assessment scores do not match the lived experiences of Veterans, stating “although the scores can be helpful on eliciting more discussion about certain symptoms I don’t think they are necessarily very accurate for various reasons; i.e. the Veteran’s perception of their score upon further discussion does not always match what one would assume if just looking at numbers” [ID0026].
Beyond concerns about item or total score validity, providers shared uncertainty regarding the optimal frequency of assessment administration (e.g., how frequently the assessments are sent out) so as to not unduly burden patients and to detect meaningful symptom severity changes and inform treatment planning.
Stagnation or decline
Providers expressed concerns about the potential for challenges navigating clinical care, for the clinician or the Veteran, when a Veteran’s symptom scores lack evidence of clinical improvement. One provider stated that when Veterans’ assessment scores do not improve over the course of treatment, this can have a demoralizing effect for Veterans and can negatively impact clinicians, sharing “Demoralization if their scores don’t change. If there are structural reasons the treatment will need to end (i.e., come to end of protocol, etc.) and the scores are not improved, it makes it hard for the clinician to handle that clinically” [ID0039]. In addition, this provider noted that it is challenging to utilize assessment results on a systems level due to other clinicians’ lack of utilization and this can decrease their motivation to use MBC, stating “My interest in it at a program/team/clinic level is hard because most people don’t use it. So it’s hard to get the data I want from it. And then over time, it impacts my own individual motivation to use it” [ID0039].
Some providers shared concern that MBC scores are viewed as the main indicator of patient outcomes and that their clinical performance would be evaluated based on their patients’ MBC results. One provider reported, “I have heard providers express concern that their clinical performance/efficacy will be judged according to the Veterans’ MBC results” [ID0007].
Another shared, “They are not the end all be all. There are a variety of reasons why someone may score a certain way and that needs to be discussed and taken into account. At times it feels like VA places too much emphasis on MBC being the main driving indicator of therapy success” [ID0011].
Educational needs
Providers requested training in psychometric properties of assessments to inform selection and interpretation, how to talk to Veterans about the Collect, Share, and Act components of MBC, and workflow and integration into their own practice. They asked for small-group or step-by-step training, workflow guidelines, and a full assessment directory. Respondents spoke about the need for comprehensive training in MBC, with one provider sharing her belief that when MBC is used by providers who are poorly trained, it leads to poor patient experience.
Assessments selection and interpretation
Two overarching educational needs around assessment selection and interpretation were identified: (1) awareness and understanding of assessments and psychometric properties to inform selection and interpretation and (2) knowing which assessments best capture symptom burden related to specific diagnoses.
Providers broadly indicated a need for a better understanding of what is available, with one provider suggesting the desire for a “directory of measures”. One provider asked for a “resource including what other providers use when treating a specific condition and why” [ID0001]. Another provider spoke to the gap in available assessments, reporting, “I am having problems finding a good assessment tool to use for anger management. Most published measures regarding anger are ‘trait’ measures rather than a ‘state’ measure and not that useful for repeat assessment” [ID0002].
Beyond awareness of what is available, providers shared that to feel more confident in using MBC, they need to better understand psychometric properties of the assessments available on BHL Touch and what constitutes clinically meaningful change in scores. Several providers emphasized the need for additional guidance on how to interpret assessment results and apply them meaningfully in treatment, noting that further training would help them use these tools more effectively. Providers specifically expressed interest in learning how they can translate scores into treatment planning. One provider noted that a barrier to adherence was “Not fully understanding the meaning gained from some assessments” [ID0024]. Others described what would help them, sharing they would like “Education on the specific meaning/interpretation of the assessments and how they could inform the treatment and give meaning to Veterans for the major assessments” [ID0024], and “Training to better help the providers to understand the results of the tools and how to utilize them effectively in treatment” [ID0042]. One provider reported a detailed and insightful educational need related to understanding the minimal important difference for the measures, stating that education would be helpful on
What are statistically significant changes in total score for some of the more common measures (PHQ9, GAD, PCL, CESD). I.e. I think I’ve heard 5 pts for phq9 but I am not sure so this is just a fake example: Say you are working with a pt with a depression goal and although your pt’s total score improvement did not trend at/below minimal depression, but say they had a (greater than) or = 5 pt reduction in total score from 18 to 12 and this is deemed statistically significant for this measure meaning that they likely saw a meaningful change and this is still considered a successful episode of care…[ID0045].
A need for nuanced education around selection and interpretation of assessments that best measure symptoms associated with particular mental health conditions or patient circumstances was reported. Personality disorders were identified as challenging diagnoses for utilizing MBC, as some providers indicated it is difficult to find brief assessments that sensitively assess personality disorder symptoms.
Provider-delivered psychoeducation
Providers consistently asked for education on how to discuss MBC with Veterans. Specifically, providers want to know how to explain the meanings and interpretations of the assessments, how to inform Veterans of how their results will impact their treatment”, how to discuss symptom changes (or lack thereof), and how to make discussing MBC more relatable to the Veteran patient. One provider responded, “How to show results of assessments over time over VVC, just some discussion about helpful ways to talk about use of measurements with Veterans, and education about how to best use some of the measures (info about the purpose, data about reliability/validity)” [ID0032].
Some providers sought nuanced education on specific topics, such as how to discuss scores that do not align with patients’ narrative reports or how to address changes over time, effective implementation when caring for patients presenting with therapy blocking behaviors, or who may be experiencing higher levels of distress in an outpatient setting. One provider noted the need for education related to caring for patients with particular diagnoses, sharing “MBC troubleshooting for certain presenting concerns — for example in the DBT [Dialectical Behavioral Therapy] team, where treatment can last for a year, we don’t have great options for MBC for patients with BPD [Borderline Personality Disorder]- we have tried a range of them like the DERS [Difficulty in Emotion Regulation Scale], BSL-23 [Borderline Symptom List 23]- none of them capture what we want to track for individual patients and other measures are not available on MHA [Mental Health Assistant] /BHL/VHA system. Other personality disorders as well – not a lot of brief options for tracking change in the first place (generally extremely lengthy!), and PDs [Personality Disorders] may not be expected to show significant changes in discrete symptom count” [ID0040].
Workflow and integration into clinical practice
Providers reported broad educational needs related to workflow, varying from broad requests of “anything to streamline the workflow” [ID0007] to a “step-by-step process” [ID0028]. Education needs around technology varied. Some providers reported a need for fundamental training on the use of BHL Touch software, such as how to log in, send the surveys to the Veterans, and integrate the assessments into the EHR. One provider requested education on how to work with BHL to add assessments to the library, stating “Getting more measures built into the programs. I have a niche measure for my ADHD patients that others might find very insightful, but it’s not on any of the MBC programs. I have to deliver it via WebEx Poll any time I want to use it, which is very cumbersome and time consuming, but worth it” [ID0041].
Another issue providers reported needing help navigating is more than one licensed independent practitioner (LIP) may be using MBC with the same patient, with one provider asking for information on “which LIPs should be administering them” [ID0036], speaking to the nuances of engaging in MBC in a multidisciplinary setting.
