Date:
Tuesday, July 21, 2026
Duration: approximately
75
minutes
Featured Speaker
Michael Fallahkhair, Deputy
Associate Administrator for Rural Health, Health
Resources and Services Administration
Sara J. Couture, Social Science
Analyst, Division of Healthcare Quality and
Outcomes, Office of Health Policy at the Office of
the Assistant Secretary for Planning and Evaluation
Mark Holmes, Director, North
Carolina Rural Health Research Center
Join us for an overview of two innovative resources
designed to support a better understanding of rural
hospital financial performance. This webinar will feature
subject matter experts from the Office of the Assistant
Secretary for Planning and Evaluation (ASPE), showcasing
its
Rural Hospital Financial Performance Dashboard,
followed by researchers from the North Carolina Rural
Health Research Center presenting the Rural Hospital
Financial Distress Index. Together these objective,
data-driven tools provide valuable insights into the
financial health of rural hospitals and can help inform
planning, policy, research, and decision-making.
Attendees will learn about each resource, see live
demonstrations, and explore how these complementary tools
can be used to better understand financial challenges
facing rural hospitals.
From This Webinar
Transcript
Kristine Sande: Hello everyone and
welcome to today’s webinar. I’m Kristine Sande and I’m
the program director for the Rural Health Information
Hub. And we are delighted to be collaborating with the
Federal Office of Rural Health Policy to host today’s
webinar, Rural Hospital Financial Performance: Tools for
Data-Driven Decision Making.
It’s my pleasure to introduce Michael Fallahkhair from
the Federal Office of Rural Health Policy where he is
deputy associate administrator. FORHP is located in the
Health Resources and Services Administration of the US
Department of Health and Human Services. In his role, Mr.
Fallahkhair helps to lead the work of FORHP, which is
charged with advising the secretary of HHS on rural
health issues and improving the delivery of rural
healthcare. Mr. Fallahkhair previously served as the
Chief of Staff for the Office of the Assistant Secretary
for Health, Executive Officer in HHS Immediate Office of
the Secretary, and as principal advisor in FORHP. He has
his past work experience in the Office of Budget of the
Assistant Secretary for Financial Resources at HHS as
well as the Office of Management and Budget. Welcome,
Michael. And with that, I’ll turn it over to you to
introduce our speakers.
Michael Fallahkhair: Thank you,
Kristine. I really appreciate that. And good afternoon
everyone. Thank you for joining today’s webinar on rural
hospital financial performance and data -driven decision
making. So as Kristine said, as you may know, FORHP
supports the rural communities, the hospitals, the
clinics in the broader rural healthcare delivery system
through grants, technical assistance, research, policy
analysis, and the greater coordination across the Health
Resources and Services Administration, but also the
Department of Health and Human Services. And so in that
with that in mind, we’re holding this webinar because
rural hospital financial performance still remains one of
the most important issues facing rural healthcare. Rural
hospitals provide care across the entire continuum. So if
you think about it’s from emergency, inpatient,
outpatient, primary care, post-acute care, and even in
some communities, long-term care. And when these
hospitals are financially vulnerable, the effects reach
patients, they affect the workers, the local economies,
and access to care close to home.
These hospitals often face structural pressures. They’re
different from other many hospitals, especially urban
hospitals. They’re often lower patient volumes, workforce
shortages, there are reimbursement pressures,
uncompensated care, changes in service demand all affect
how rural hospitals operate and whether they can even
maintain the key services they need to for their
communities. Now, the Assistant Secretary for Planning
and Evaluation or ASPE, their recent work here helps
provide national view of those pressures. Their report
found that rural hospitals are more vulnerable to closure
or conversion to outpatient-only facilities than urban
hospitals. That’s the significant differences ASPE will
definitely get into more. Now, at the Federal Office of
Rural Health Policy, this work, our work, directly
informs how we then think about supporting rural
hospitals. We have a hospital state division that
administers all of our programs like the Flex, the SHIP,
and other hospital-related technical assistance programs.
Our policy research division reviews federal policy
regulations and rulemaking, including CMS policies for
potential rural impact.
Our grantees and partners work directly with rural
hospitals and community. So today’s webinar also reflects
importance of this collaboration that we have across our
office, but also at HHS. Rural hospital sustainability is
not just a FORHP issue or an ASPE issue. It requires
shared data, analysis and coordination across the
department, including, like I said, ASPE, HRSA, CMS, but
also FORHP-funded research partners like the University
of North Carolina where Mark Holmes comes from.
Dashboard we’ll discuss today includes data through 2023.
Provides a tool for examining hospital level financials
and structural information. And it’s really important to
remember that it does not replace local context or answer
every question, but it helps to give policymakers, state
researchers, rural stakeholders like you all, a common
evidence base to better understand the trends and
identify areas of concern. This is especially timely as
HHS and CMS continue to focus on rural hospital
sustainability and implementation through programs like
the Rural Health Transformation Program. So better data
and analysis can really help states and ourselves and
communities understand where these hospitals are
financially vulnerable, where services may be at risk,
where TA technical assistance or policy attention may be
most useful.
I also want to highlight and thank for funding the Rural
Health Information Hub. Thank you guys so much for
hosting this webinar, being the key statutory
clearinghouse that we have in our office that can help
inform decision makers and rural stakeholders at all
levels, national, state, community about rural health.
It’s especially meaningful that the secretary’s office
through Assistant Secretary for Planning and Evaluation,
ASPE and Sara took this important topic very seriously
and their desire to work with RHIhub together on this is
very important as well. We really thank you all for that.
So with that context, I’m very pleased to introduce
today’s presenters. So first we have Sara Couture.
She’s a Social Science Analyst in ASPE’s Office of Health
Policy, and her work focuses on using healthcare data to
inform policy and developing accessible data-driven
resources. It’s amazing. Mark Holmes is the PhD. He’s the
director of the North Carolina Rural Health Research
Center. His research examines why rural hospitals
experience this financial distress, how payment policy
shapes their viability and where and what that evidence
suggests about preserving access in rural America. So
thank you all for joining us. Thank you to our speakers
for being here. We look forward to a really practical
discussion on how these tools can help inform planning,
policy, research, and decision-making. Handing over to
you guys. Thank you.
Sara J. Couture: Awesome. Thank you very
much, Michael. As Michael said, my name is Sara Couture
and I am part of the work that ASPE’s been doing on
looking at the state of rural hospitals in the United
States. The usual disclaimer that the findings and
conclusions are those of the authors do not necessarily
represent the position of the Department of Health and
Human Services. And I just want to take a moment to thank
the team that’s really been working on a variety of
different projects around rural health. The dashboard was
developed with myself and my colleague, Fredo Louis, and
with a lot of the data work in helping us identify which
graphics was really work with Kaushik, Eden, Ge, and
Scott. We want to thank everybody for their hard work on
all of these important projects.
So ASPE had kind of a suite of rural hospital projects.
The goal of these really was to provide internal
policymakers hospital level data to help understand the
financial and environmental risks that influence the
financial state enclosures within rural hospitals. There
was a variety of different projects. The main one was
focusing on providing a model to predict risk of
closures. And when we were putting those information
together, we were able to get hospital level information
and we want to be able to provide in a way that was
accessible. And we also want to improve how to understand
where residents get their care and the impact that
hospital closures have on those particular patterns.
To do this for both the initial issue brief as well as a
dashboard, we’ve combined multiple data sets, including
the Medicare cost reports data, the American Hospital
Association’s annual survey, the area health resource
files, and county level information. We were able to make
this data more accessible using a dashboard, which I’ll
do a demo of here soon.
But we also used this data to develop a Cox model to
predict risk closures that was published as a separate
issue brief. The Cox model is not in the dashboard, but a
lot of the same variables from the model are what
showcased in the dashboard. We also did a few other
projects, including a Medicare and Medicaid claims level
analysis of care patterns in rural areas as well as a
literature review.
So the goal of the dashboard itself was to provide the
public with comprehensive information on rural hospitals
that are both open and closed. And we were able to do it
from 2012 to 2023, although we are looking on updating
the 2024 data. The dashboard includes three main
components. The first one is a national level map that
shows open and closed rural hospitals by year. And it
allows users to filter on multiple things, including
identifying specific hospitals, filtering by closure
risks, the occupancy rates, quartiles—so
highest occupancy versus lowest
occupancies—ownership type, and system
affiliations.
We’re also then able to select a specific hospital and
drill in to see a longitudinal view of some of their
data, including their occupancy rates, number of beds,
Medicare and Medicaid share of discharges, liability
asset ratio, and annual profit margins. And then we also
want to be able to provide a bigger context of what was
happening in the county that that hospital was located.
So it includes things of the amount of access to the
healthcare workforce, other healthcare facilities, as
well as some demographic information on the county level.
So some key findings is the dashboard itself, like I
said, aligns with the model, although the model itself is
not in the dashboard, that helps identify key
determinants of rural hospital closures or conversions in
outpatient-only facilities. The analysis, that you can
see the issue brief is linked down here below, did show
that low occupancy rates, for-profit ownerships, and
proximity to an urban counties are key predictors of
closure or conversion.
I’m going to quickly share my screen and show a couple
examples of how to use the dashboard. So when you go to
the ASPE Rural Health Dashboard, you’re going to first
get this page, which provides an overview as well as
going over some of the variables and characteristics that
I did in the presentation. From here, you can proceed to
the dashboard and it’ll have this map that shows the
number of rural hospitals and in this case in 2023, so
that’s a default right now, but you can use this number
line to see it over time and it’ll showcase the number of
hospitals, how many closed that year, and how many total
open hospitals there were. You can also see the number of
hospitals by state down here, and you can do it from 2012
to 2023.
Over here, you’re also able to use those filtering
mechanisms that I mentioned earlier. I’m going to go back
and show you the hospital name and ID in a few minutes,
but first I’ll go through some of the other filters. So
you can filter on looking at just the closed hospitals.
So here’s an example. These are in red and you can also
see the hospitals and where they were located down in the
bar graph below.
You can filter by specific states. So you can see the
total number of hospitals by the state either using the
filter here. You can also click on the state within the
bar chart here so you can see Texas is being highlighted.
You can filter by occupancy rates. So you can see in 2023
which hospitals had the highest occupancy and you can
also filter by which ones had the lowest occupancy rates
as well as being a critical access hospital. And this
one, as I stated, was kind of important is which
hospitals are in a county that are adjacent to an urban
county.
So to be able to identify a specific hospital, you can
use one or two ways. The way we recommend is if you have
the hospital ID is to enter it here. You click there,
push enter, and then you’ll be able to find that
particular hospital on the map. Please note that if this
hospital say closed in 2016, it would not be on the 2023
map. So if you’re having a hard time finding a hospital
that closed, make sure you go to a year that it was
opened or the year that it closed.
So to be able to drill through, there’s a few ways of
doing it. One is to hover over this dot and you’ll see
this drill through and then you click on hospital
characteristics. Another way is you can just click on the
dots and use this button here.
This provides an overview of the hospital itself. So we
provide the information. So this one is Indiana
University’s Health Bedford Hospital that’s in Hartford
City, Indiana. You can see it’s a nonprofit hospital
that’s system affiliated and critical access, and it
closed in 2023. You can also see which county it’s in up
here. And we tried to provide some longitudinal data that
we found in the ASPE brief were important for risk of
closure. So occupancy rate’s a major one. You can see in
this hospital that the occupancy rate was decreasing over
the last 10 years before it closed. And the actual bed
numbers itself was 15 and that was pretty consistent
across. You can see the Medicaid and Medicare share of
discharges per year. It’s pretty common, and you see here
that it’s a high percentage, especially of Medicare
patients that was part of their discharge. You can also
see their profit margin by year as well as their
liability asset ratio. In order to see a little bit more
information about Bedford County itself and get a bit
more perspective on what’s going on in this particular
hospital, you can go up here to click on county profiles
and here provides a bit more information about Bedford
County, including its population density from 2010 to
2020. Its root code as well as the average population
over the past 10 years and if it’s adjacent to an urban
county or not. And as I stated earlier, this does have a
significant increase in closure risk.
From here, we have a couple different ways of being able
to explore a little bit more about the county itself. So
the first one is looking at access to healthcare
providers. The default will be looking at primary care
providers and we look at a ratio of population per
provider over time. So over the past 10 years, about a
little less than 2,000 people per primary care provider
within Blackford County. If you click this dropdown menu,
you can explore other types of providers. So say nurse
practitioners, you’ll see it over time. If there are no
nurse practitioners or if the data is missing, it’ll look
blank like you could see it over here. So that can mean a
couple things. It’s not necessarily that there’s no
providers. It could be that we don’t have the data for
that particular year.
We can also look at access to other healthcare
facilities. So this one’s for rural health clinics, so
either they didn’t have one until 2021 or we just didn’t
have the data for it, but they have about one rural
healthcare clinic per 12,000 people. You can also explore
other types of clinics. So there’s no federally qualified
health center here or they don’t provide the data for it,
but there’s many different types that would be available
depending on the county.
We also provide some information on the length of life in
this county compared to other counties within the state.
So we looked at this at quartile risk. So if it’s a one,
it means that it’s in the top 25% of counties in that
particular state, in this case Indiana. If it’s four,
it’s at the bottom 25% of counties in that state. So here
the length of life was in the bottom 25% of counties in
Indiana.
And then finally, we’re able to explore some of the
county demographic information, including the household
income as well as some others such as the population
total. So this one has been decreasing over time. And you
can also see the age groups. So if you look here, there’s
a pretty high proportion of people under the age of 18
and over the age of 65, but not a whole lot of people of
working age, for example.
I’m going to do one more example to just see a comparison
of a hospital that is open, at least in 2023, and show
how you can also find a hospital by name. So for this
one, we are looking for Helena Hospital in Arkansas. If
it is a common name, we’ll get multiple ones in there. So
when you do that, you’ll have to click on each hospital
to see which one you’re interested in. There are many
common hospital names. It’s also pretty common if there’s
a larger hospital system, each of the little clinics will
have a separate name based on the town. So the hospital
ID really is the easiest way of finding it, but you have
this ability too.
All right, so we’re going to look at this hospital in
Arkansas. So this is Helena Regional Medical Center in
Helena, Arkansas. It’s a for-profit system-affiliated
hospital and a non-critical access hospital. And you’re
seeing some interesting things here as well. So one, you
see the occupancy rate is also overall decreasing and the
number of hospital beds had a pretty sharp decrease and
then a bit of an increase in 2023. You can see its share
of Medicare and Medicaid data, which still is pretty
significant, although less than the previous hospital, as
well as its profit margins and liability to asset ratios.
So you can see financially this hospital does seem to be
having some potential problems.
If we go to the county profile, we’ll get a similar view
as we saw before. You’ll see the density from 2010 to
2020 and as well as be able to explore the different
context of what’s going on within the hospital system,
including access to providers, which this one has been
improving over time. Access to other types of healthcare
facilities. Federally Qualified Health Centers here. It’s
standing, which was also in the bottom 25% for length of
life and its demographic information, including household
incomes and population, which has been showing a decline.
So that’s overall the demonstration of the Rural Health
Dashboard. We have seen that it has been useful for
policymakers internally and we hope will be helpful for
other researchers as well.
Mark Holmes: Thank you, Sara. I think
I’m just going to jump right in without a Kristine and
Michael buffer, so appreciate that. I want to thank
RHIhub and the federal office for inviting us here today
and in particular to share the webinar with ASPE. It’s a
really impressive tool. And if you haven’t checked it
out, I encourage you to spend some time poking around
like it because it’s really impressive.
So my role today is to talk about the Financial Distress
Index and I’m going to spend a fair amount of time.
That’s what I’m going to really be focusing on. And I
have a colleague that many of you may know, Erin Fraher
at the Sheps Center who likes to do the talk in one
slide. And this way you can read this and then zone out
and update your MySpace account or whatever. My students
like to do it this way as well, but really five bullets
here.
And we’re talking a lot about rural hospital closures,
and I think the smarter question is upstream and how
healthy is it before it closes? And I think that’s
consistent with the ASPE presentation you just saw and
really thinking about it not as a closure as the final
outcome, but what can we do upstream and prevent those
from, or at least prepare for those better?
The Sheps Center built the Financial Distress Index or
FDI. We’ll talk about that. Again, that’s the bulk of
what I’ll be speaking about today, but it’s a simple
plain language measure of a rural hospital’s financial
health and is really trying to give you an easy thing to
conceptualize around. Built for rural hospitals, open
source, forward-looking and based on public data with an
annual update.
It’s going to be a double-edged sword here in
some sense, but it works. We’re going to present some
evidence of its predictive power and talking about how
well it’s forecasting outcomes two years down the road.
But I also want to spend a fair amount of time talking
about it as a screen, not a diagnosis. It’s a prediction.
It’s not as good as weather predictions to be honest. And
so if you look at it’s something that you want to keep an
eye on, but it’s going to involve a human in the loop and
that’s going to be a major portion of the secondhand. If
you look at your weather app on your phone and it says
it’s going to rain today, you’re going to grab an
umbrella, but you may or may not need it, but it’s
something that you can look at in order to prepare.
First we’re going to start with the why and give the
background for what we’re trying to do. You saw the ASPE
website that talks about hospitals closures and what’s
open. We’ve been doing a closure website for a while as
well and the closures are getting the headlines. This is
our map and you could see we could talk about this
particular map for a while in terms of trends and where
these are located and not. But how can we assess the
financial health of rural hospitals before they close?
And by doing so, it allows us, interpreted broadly, to be
better prepared to prevent closures or prepare for more
orderly transition. So I’m going to use both languages
throughout the next 30 minutes in talking about
preventing them or preparing a community for the
transition to a post-hospital healthcare ecosystem.
Since we’re on here, the other thing I’ll say that I
jotted down during Sara’s talk is that people sometimes
get surprised, but counting closures is not as
straightforward as you might think. I think it’s an easy
thing for us to conceptualize in that there’s a hospital
that’s open today and it’s not tomorrow, but there’s a
lot of gray area or grayness into what constitutes a
hospital closure. The standard that we’re taking is, does
a community lose access to inpatient facility? Which may
be different from other definitions. And so it may be the
case that you pull up the ASPE website and you pull up
our website and say, “There’s a discrepancy here.” I
haven’t spent time looking at all 136 or whatever we’re
at today to compare them, but it is a difference in terms
of how to think about it. So for example, we count a
closure if a hospital were to move 25 miles away from the
town, that community lost its access to its inpatient
even if it built a new facility 25 miles away and
different places are going to count that as a closure. We
also have discussions about, for example, a hospital that
closes its inpatient wing, we define it as a closure even
if it continues to provide outpatient services. So, the
notion of what closure is is also a little nuanced.
So financial analysis is hard and here we have the
wizard, George Pink trying to teach the frustrated
student here how to think about financial analysis. But
when I started working with George about more than two
decades ago, I came in as an economist and was like, “All
right, so let’s do some easy things like, higher profit
good.” Well, not necessarily. It depends. Maybe you’re
over investing and maybe it’s a one-time thing. And what
I quickly learned is that to really have a sense of what
you’re doing of what the hospital looks like requires a
comprehensive overview. And this is why people get
degrees in finance and hire consultants because it’s not
as simple as a quick measure.
I would rather have a profit margin of eight than a loss
of four, but there’s a wide variety of circumstances that
fit behind that. And so what we tried to do and what we
have done is to build this comprehensive, simple measure
to more quickly assess a hospital’s financial position,
even if it is only a rough estimate.
So why we built this? And there’s a long history of
financial indices to capture financial distress. And some
of the early examples are, I think the father of this is
the Altman Z score or Z score depending on whether you’re
from Canada or the United States, but it was built for
publicly traded companies and really calibrated on
manufacturing firms. That’s not going to translate well
to hospitals for a couple of reasons. One, healthcare’s
pretty different for manufacturing, but second, very few
hospitals are publicly traded and almost no rural
hospitals are publicly traded. So it’s not going to work
as well for that.
Bill Cleverley and other associates have designed a
couple different indexes, the Financial Distress Index
among others. And that was designed for hospitals, but
not as calibrated as much for rural. And we felt like we
could do better than that. So that’s what we set to do in
the mid-2010s to really find something that works better
for rural hospitals. And the concept here is that there’s
a measure of financial distress that’s unobserved. So we
don’t know that a hospital is in distress. We can see
markers of it. If it closes, that’s the ultimate sign of
financial distress or a bankruptcy is also a sign of
financial distress, but negative profit is also a sign of
financial distress. Having negative equity or a negative
fund balance, depending on how you think about accounting
procedures. These are all signs of distress.
What we’re trying to do is really measure that unobserved
latent variable kind of concept of, what is it that’s
behind the scenes that is really leading to these signs
of things that are developing?
So these are the principles that we sat down as we were
thinking about this, and we started with a
non-proprietary open source formula where the formula
instructions are available to the public. And I’m going
to take a moment here to commend ASPE because they’ve
done exactly this as well. If you look, Sara presented
the dashboard, but there’s a findings brief that’s
associated with that, with these Cox regression model
that they use in order to predict it. And you can see the
coefficients right there. They’re very clear about where
the data come from. So that is open and anyone can use it
to reconstruct their own or to reconstruct the data and
say, “Well, here’s how to do the data. Here’s the
formulas that go in. We add them together and we get
these predictions that come out of it.” And we felt that
was really important to be transparent in that respect.
It’s designed for rural hospitals. We calibrate it among
rural hospitals and rural hospitals has quotes around it
because we’re defining that semi-broadly, not just
hospitals that are located in rural areas, but critical
access hospitals, for example. So it’s designed for rural
hospitals, which look pretty different than urban
hospitals in many respects. They’re smaller, they
generally have lower profit margin, all the things that
we know about for those who are following rural
hospitals. And so it was important not to build this on
hospitals in Boston and New York City and Chicago and LA,
but to build it on hospitals in the thumb of Michigan and
the panhandle of Oklahoma, because that’s going to be
more relevant for rural hospitals.
We wanted it to be forward-looking and predicting future
outcomes. So thinking about today based on what we
observed, what do we think is going to happen in the
future? Because that’s going to be important if we’re
going to think about how do we intervene? How do we
prepare? How do we prevent? It’s got to have something
that looks forward.
Consistent with that first bullet, we wanted it to use
public data. So it’s largely built on HCRIS [Healthcare
Provider Cost Reporting Information System] or Medicare
cost reports, but it has other elements as well that go
into it. But all the data that are behind it are
available to the public.
We wanted it to be useful for simulation. And so for
example, we wanted it to be able to say what happens if
revenue falls by 20%? What happens if this policy is
enacted? What happens if a hospital stops reinvesting?
What happens…variety of thought experiments in order to
predict what would happen? For example, we did this about
10 years ago using this model to look at what would
happen if critical access hospitals were to lose their
cost-based reimbursement policy, go to PPS payment using
estimates that were developed by others of a 20 to 30%
decline in Medicare revenue. We ran that through. We were
able to see what would happen in terms of the financial
distress.
We want to face validity so that when we listed the
predictors, people look at it and go, “Yeah, that seems
right. I think that’s something that belongs in there.”
Again, very similar to what ASPE’s model as well. I’ll
also take another moment here. An advantage, I think, and
a strength of having a couple different models and
multiple teams looking at this is that people are
thinking of new elements that go in there that are a
little creative. For example, we have not included a
contiguous proximate, I forget the exact noun, but next
to an urban county. And I think they showed that that was
important. That’s something that we should be thinking
about as we update our model as well.
Using that as a kickoff, update annually, but revising
regularly, this is version 3.0 of the FDI. The first one
was built in 2017. We updated it with new data, so with
new cost report data, with new other elements that go
into it, but then revise it for a different model, a
post-COVID world, for example, regularly, roughly every
three to five years.
And then finally, the results should be easy to
understand. And I have a nota bene on the results there.
The results, the implication should be easy to understand
even if the Greek letters that are behind it are a little
harder to follow. Again, consistent with ASPE, many
people on this call don’t know what a Cox model is, but
they can look at the output of it and go, “Okay, higher
is bad. I get it.”
Okay, so that’s the “why.” So now
we’re going to get into the “what.”
What is the FDI? So we’re going to tell a two-part story
here and on the left here we have GPT’s representation of
George Pink, Tyler Malone, and myself who are behind this
version of the FDI. And on the right we have Tom Morris.
And here we have this really fancy automobile and what
we’re going to hear about first is how awesome it is. And
then we’re going to get into the second part of the talk
where we talk about some of its limitations.
So rather than start with Greek letters, I think it’s
easiest to think about this in terms of a boxes and
arrows kind of concept. And so I’m going to walk through
this, but let me orient you the big idea here. The
current information are all the variables that are going
into the model.
So we use all of these data, profitability, looking at
this year, last year and two years ago, outpatient
revenue, uncompensated care, benchmark performance, et
cetera, et cetera. You can read the whole box. We put
that into our model to predict distress and out of that
come the following four categories of results: highest
risk, mid-highest risk, mid-lowest, and lowest. We spent
three months coming up with what to call those four
levels and academics are not known for being creative,
and so that’s what we came up with. The brief where we
talk about this is on the bottom right there and you can
blow it up and look at it.
So let’s walk through each of these components. So the
first box here is the finances, and this is playing a
major role, and this is the heavy lifting of the model. A
lot of the predictive power comes out of these
indicators, both direct financial performance
profitability is probably unsurprisingly a huge predictor
of future distress. Think of it’s the opposite of the
mutual fund. Past performance does not predict future
results. Here we’re saying it does, and in fact it does.
But we also include government reimbursement policy
because that’s going to be real important. ASPE’s results
had CAH as an important indicator. They included Medicare
and Medicaid payer mixes as predictors. We have measures
of the fee index, which is a measure at the state level,
I’ll use the word generosity, but the percent of Medicare
fees that Medicaid program in the state offers. And those
are going to be important drivers of the underlying
finances. And so think of that as a structural thing. And
so this is exactly the kind of thing where we can look at
and say, what happens if we were to change the fee index?
So we could look at it and model, “Okay, what happens if
a state goes from 90% of Medicare to 100%?” And see what
that does to the distress level.
Hospital characteristics, ownership size, are you a
member of a system? And then market characteristics or
community characteristics depending on your perspective.
But the service area, the community that you’re serving
is going to be really important. And think about the
economic condition. It’s going to be a lot easier to have
strong finances if you’re in a place with a high per
capita income than you’re in a community with a low per
capita income and all the things that go behind that.
Again, market size. We know that smaller hospitals and
smaller markets generally lead to higher distress levels.
And then so we’re looking at the risk in two years hence.
We’ll talk about that in a little bit. But again, this is
the forward-looking part of the model. And then finally,
next slide. It’s a straightforward categorization, but it
has a gradient. And so rather than saying at distress or
not, it’s really recognizing that there’s a continuum
here. And we could break it into 10 groups. We figured
four was a pretty nice round number. We might go crazy in
the next iteration, have five groups. Stay tuned to see
how wild and crazy we feel at that year.
And then what we’ve done here is you can see the new
variables that are outlined in red here to show what’s
changed since the previous. And so again, this reinforces
this notion of we’re revising it regularly and
recognizing that the conditions that lead to distress in
the mid-2020s are probably different than they were in
the early 2010s. And so continuing to refine the model
based on new updates.
All right, so how well does it work? Would love to nerd
out and talk about Greek letters here, but this is a
representation of the prediction. And so that purple
line, what you can think of, for example, is the
horizontal access is the probability of distress, but
think of it as a probability of a certain signal. And so
anything on that purple line means that we’ve completely
nailed it. And so anything in the zero to 10% bucket has
an average mark of distress between zero and 10%.
The way we did this in the second bullet there is the
training sample and test sample. So we took the list or
the collection of rural hospitals in multiple years and
we looked at a subset of them and built our model and
then said, “All right, how does that predict in the set
that weren’t included in the model?” So what we’re doing
in this and this is good predictive practice, because
we’re not predicting what the model has already seen.
We’re using the model to predict things it hasn’t seen
yet and seeing how well that works.
In terms of the lower side, that green circle there, look
how well those dots are tracking with that perfect
prediction line. We got a fist bump there. We’re pretty
excited about that how well we were doing. The size of
the bubble represents the number of hospitals in that.
And so you can see that the biggest bubble is those at
the really low risk. And you can see that as probability,
the predicted probability goes up. The observed
probability of bad outcomes also increasing right on that
purple line. So that was great news.
You can see that at the higher levels of prediction,
we’re under predicting distress a little bit,
particularly those what, four, well, three points at the
bottom left of the circle. So for those guys, we’re
predicting pretty high risk. Sorry, we’re
under-predicting or over-predicting distress. I said
under-predicting there. We’re predicting a higher risk
than we actually are. So that’s something that we’re
going to look at refining as we go forward.
Okay, so that was the overall prediction. Let’s look at
what this means from the signals. And so distress is an
assessment, but what we really see are the signals of
that. And so for this model, we’re using closure, which
is again, the ultimate signal, the hospital negative
equity. And so the hospital owes more than it’s worth and
a negative cashflow margin that it is not meeting its
bills effectively. And you can see here that we have the
bucket’s lowest, mid-lowest, mid-highest, highest. And
for all four signals, the higher your risk category, the
more likely you are to have that signal. So among the
lowest risk categories, 0% closed. Among the highest
risk, 3% closed.
Looking at the negative cashflow margin among the lowest
risk, 5% or almost 6% had a negative cashflow margin
compared to 62% in the highest risk. And so this is I
think evidence that shows that this model’s working
pretty well. It does a pretty nice job of predicting
who’s going to have signals of distress two years later.
So limitations, how to use responsibly. So I’ve spent the
first 15 minutes or so talking about how great this is.
I’m selling you on this some kind of sports car, I’m not
really sure what. Now we’re going to talk about how to
use this responsibility responsibly or the limitations of
the FDI. So here’s the same deal. And George, Tyler, and
Mark are saying that, well, the FDI was, or whatever it
was, efficient and fast and fun, but it does a terrible
job carrying 2x4s. And so the way to think about this is
that the FDI is a nice to great tool. I would say it’s
great. Other people might say it’s fine, but it’s not
going to work in all circumstances. And you can see here
that it’s a terrible tool for certain applications such
as this.
I’m going to start with this one, and these are numbered
in terms of the limitations to think about with FDI. It
is not a perfect predictor and we like to consider it as
a screen or as a diagnosis. It is a unidimensional.
Again, just we have the four buckets based on cost
reports and other public data. It has no private
information. It works pretty well in populations. That
was the bar chart, but it’s going to be less accurate.
And again, less accurate, not inaccurate, but less
accurate for individual cases like hospitals. And we’ve
tried a number of analogies over the years. The one I’m
trying today is a body mass index. Most people here are
familiar with the notion of a body mass index. And
generally if you have a lower BMI, you’re going to have
lower risk for CVD and other outcomes. And so we know
that US Preventative Services Task Force and there are
other best practices that recommend screening for CVD and
other risk based on your BMI.
But it’s omitting important data, your age, your smoking
status, your cholesterol level. I remember when I met
with my PCP about 15 years ago, he said, “All right,
here’s your BMI. Here’s your risk of cardiac outcome in
the next five years.” That’s nothing. I can live with
that. He says, “Now, if you come back 10 years from now
and look like the same thing, here’s your risk of a
cardiac event.” I said, “All right, I guess I better do
something.” So that’s a great example right there. BMI is
a unidimensional. With age, all of a sudden we had a very
different assessment of what my risk is in the next five
years. Saquon Barkley is a NFL running back. He and I
have roughly similar BMI. One of us is in much better
shape. One of us is going to be at much higher risk for
CVD and other outcomes. Again, another example of that.
And if I found, I’m sure there’s a smoker out there with
a BMI that’s lower than mine, but has high cholesterol,
has high blood pressure, probably not healthier than me,
almost certainly going to have higher risk for CVD as
outcomes. So FDI is one measure that can be useful for
prediction, but only think of as a screen concept.
Second, we say at risk. We have these four categories,
highest, mid-highest, mid-lowest, and lowest. But what
does at risk for distress mean? And I don’t know what it
means to be at risk for something. I define it as
non-zero. I am at risk for a meteor to come crashing
through this ceiling, especially how weak this building
is in the next two seconds. Didn’t happen. It was a
non-zero risk. It was highly unlikely, but it was
non-zero.
But other people have a different concept of what at risk
means. Is it 5%, more than 50%? And there are a variety
of other rural hospital distress indexes that are using
different thresholds. And I’ll have a slide with examples
of that, but none of us are right and none of us are
wrong. And the picture on the right here is a pretty
clever thing that was done a few years ago. And I don’t
remember exactly what they did, but it was something like
these are words, would these be adverbs I think? I think
these are adverbs that are used in Dutch news stories.
Always certain, almost certain, almost always. And they
took these adverbs and they asked people, lay people, I
think it’s 650 people or something like that, what does
almost mean? What does always mean? What percent would
you ascribe to that? And there’s a couple takeaways here,
but first of all, I would say that looking at this in
always, the average probability if something is always
happening as assessed by people is 96%. So as a
statistician, I’m going to start there and I have some
questions.
But look at the wide variety on some of these words and
how wide the spread is and what people think about when
it comes to that. I blow a section of this up and here
are some things that probably kind of mean at risk,
maybe, I don’t know. Probable, possible, maybe,
uncertain, chance, liable to happen. All of these are
sort of measuring something that means could happen. But
I think, no, I know that when we talk about at risk for
distress, there’s going to be a wide variety of what
people think. And look at all this. Basically the most
common answer for all of these between possible, maybe
uncertain chance liable to happen is right there at 50.
And the opportunity I didn’t have here that I’m kicking
myself, but I’ll use in the next version is that classic
Stepbrothers episode with, “You’re saying there’s a
chance.”
And in that meme, “You’re saying there’s a chance.” That
was really like a 0.1%, but people hear chance and think
50%. So at risk for distress means a lot of different
things to different people. It definitely does not mean
is going to close, but it means there’s some chance
between zero and between 100.
So here’s an example. You can see the wide variety of
what at risk means. Vulnerable to closure, at risk of
closure, a third of rural hospitals, the highest, here’s
our percentage of the map where we show our assessment of
the risk of closure. And I did not cite any particular
indices here because that’s not the point. The point
isn’t to say that these guys say there’s more, these guys
say there’s less. What the point is, is that I don’t know
what the correct metric is here. Generally, I probably
would say that every hospital’s at risk for closure, but
I think in some respect, if we think about those
probabilities that we saw in the previous slide, anything
that looks like 2,100 hospitals are at risk of closure is
just ridiculous. So we need to better understand when
we’re talking about at risk for distress, what does that
mean? And it’s really hard to compare apples when you
have different indices using different cut points.
Third comment, it’s not a statement about management
quality or the system’s commitment to the hospital or,
or, or. Hospitals at high risk may survive for years and
there are plenty of examples that since we started this
in 2017 of rural hospitals that have been highest risk
for 10 straight years and they’re still here. There may
be an atypical financial model and there’s at least one
hospital that’s basically financed by an extremely large
endowment of a certain private equity. And so their
operating margin is like negative 10, negative 15% each
year, but they slide that endowment over income over and
they’re doing just fine.
It may be that a high distress hospital is supported by a
system for strategic purposes. Maybe for example, to have
a beachhead and say, “Well, we’re going to lose money at
this particular rural location, but with that spot there,
it helps us with referrals for hips and cardiac at the
flagship facility.” Maybe supported by local government,
cost report or other data may be inaccurate. When we say
that we use cost report data, you can tell the people who
are familiar with cost report data and they go, “Oh.” I
mean, cost report data does have challenges. It’s two
years lagged often, but it’s not bad usually depending on
which parts you’re looking at. And so the degree to which
those data are inaccurate are going to influence the
predictive power of the model. And generally, we’re more
concerned with hospitals on a trend of increasing risk
than those that are at perpetually high risk.
You need a human in the loop. And if we think about as
we’re all building our AI agents, we probably don’t want
to hit go and let them run amuck and start changing
things on your server of your institution. It’s great for
a great first draft, but you need to go behind and verify
it. It’s the same idea with FDI. If you are someone who’s
trying to look at hospitals in your state and figure out,
where are my best resources best spent? It’s a good first
start, but then you go from there and work there and
think about other sources of information. Well, these
guys are high risk, but I know that they’re highly
supported by the system. They’re never going to close.
These guys are reported as mid-low, but these data are
two years ago and the plant just closed six months ago
and I know that they’ve been texting us to say they’re
having trouble meeting payroll. That’s human in the loop
that needs to be used to be updated, that FDI account.
And I think it’s really important to use these
responsibly for all the reasons that we’ve said before.
These are not a definitive diagnosis. And so anytime
you’re using the FDI or any index like this, you need to
recognize that anything released to the community could
have a self-fulfilling cycle. And saying, “Elm Valley
Hospital is in financial distress.” Is really hard
because the community says, “Well, geez, I guess we
shouldn’t go there anymore.” And it may lead to that. So
these FDIs are really important for policymakers and
practitioners, particularly on the government side,
internal planning purposes, anything that’s designed for
those interventions and preparing. And they are extremely
sensitive and those who are using it need to be
recognizing the power and what can happen with
irresponsible use.
All right, conclusion. Three bullets. So here’s the comic
all put together. FDI has some very fancy things and we
think it is really nice, but you need to recognize the
job that you’re using it for and recognize its
limitations.
Okay, so key takeaways. I think we’ve covered all these
rather well. I’m not going to read the list, but just
really talking here about using them as a point of
information. It is an easy assessment. It’s like a
traffic light, but you need to understand, well, traffic
light’s probably not a good one, but it’s a good screen
to let you look at it and get a quick sense of at a high
level, how do we think this hospital’s doing? But really
you need to go behind it and think about, get that human
in the loop. What other information do we have? What else
can we use behind it? And thinking about what this
“at-risk” notion means.
Here’s my contact information, and all the things that
you have, if you want to pull out your QR code. The
standard disclaimer. And I also wanted to recognize how
many people have contributed to this over the years.
There’s a long list there ranged alphabetical by last,
but a number of people have updated it or revised it or
published on it or refined it. And it really is a
brainchild of multiple people and some of whom have left
our center and some of whom are still around.
For rural health research like this, the Rural Health
Research Gateway is a great resource for you to go and
check out. If you’re not familiar with that, give it a
look up. And there’s tons of information about rural
health research and policy. And I think that concludes
today’s presentation. Thank you very much.
Kristine Sande: We do have a few
questions that have been entered while you all were
speaking. So we’ll start with those. There’s a few here
for Sara to get started. The first question is, “Can you
look at hospital closure status by percent of hospitals
within a state?”
Sara J. Couture: Right now, the
dashboard is not designed to do that, but I think that’s
a very interesting enhancement to put in our backlog as
we’re updating the dashboard.
Kristine Sande: All right. And the next
question is, “This is great facility data. How do you see
policymakers using the information?”
Sara J. Couture: Yeah, so we’ve already
been in conversations with CMS. The Rural Hospital Fund
as part of the One Big Beautiful Bill has already been
discussed as this is a potential tool as they’re
considering ways of targeting funding as part of that
initiative, as well as a few other initiatives by the
White House.
Kristine Sande: All right. Next question
is, “This is a great tool. Why are there a few states
with no data on the dashboard such as New Jersey and
Connecticut?”
Sara J. Couture: Yeah, so the definition
of rural hospital was using USDA’s urban rural county
continuum codes, and they defined it as four to six on
those code scales. So for New Jersey, there’s two
counties in New Jersey that are in there, but they either
didn’t have a hospital that we would consider a rural
hospital in there. And then Connecticut’s an interesting
case because the way that their counties are defined also
is not super aligned with those codes. So I was going to
flag that for our team as well as like, we should make
sure we’re not missing any counties in Connecticut, but
the counties that are in there are urban counties.
Mark Holmes: And Kristine, I’ll just add
that we use a slightly different definition. And so this
I think is important to keep in mind as you’re looking at
these different tools. We’re using the HRSA or federal
office definition plus CAHs, as I mentioned. And so
there’s just different ways to approach it and it’s just
useful to look at it. And so that would mean that we have
more hospitals if I’m doing that math quickly in my head
than ASPE.
Kristine Sande: Yeah. And that’s a great
reminder. Anytime you’re looking at a rural tool or a
rural stat, it’s important to ask what definition is
being used here. So now there’s a couple of questions for
Mark. “So is the FDI a tool that can give specific
hospitals at risk in each state?”
Mark Holmes: So great question. And this
comes back to point number five, I think, in that we
generally like to hold that data as close as possible.
Hospitals can see their own and they can look at trends.
Critical access hospitals can look at it using a tool
called CAHMPAS, C-A-H-M-P-A-S, that hopefully critical
access hospitals are aware of. That’s a portal that’s
funded by the federal office through the Flex monitoring
team. We provide the hospital level specific data to the
State Offices of Rural Health in each state. So
Connecticut SORH can see the distress for the rural
hospitals in Connecticut, but not anywhere else. So
that’s the degree to which those data are shared.
Kristine Sande: So Mark, “Does the FDI
find the same predictors such as system affiliation,
continuous urban hospitals, et cetera, as the dashboard
research?”
Mark Holmes: No, it does not. So again,
that’s part of the fun of doing things like this. Is
there an example of, again, Sara, help me out. It wasn’t
contiguous. It was next to, proximate.
Sara J. Couture: Adjacent to an urban
county.
Mark Holmes: Adjacent. Yes, thank you.
Adjacent to an urban county is the one that I remembered
that they have that we don’t. And what? We have Medicare
Advantage, I think is an example of something we have
that they don’t. So again, just different perspectives. I
wonder, certainly we know that profitability is carrying
and other financials are carrying a lot of the heavy
lift. Sara, do you know, certainly there’s lots of P
values there, most of those were significant. Are you
able to speculate or remember in terms of the predictive
power? Are your measures of financial performance really
carrying the bulk of that prediction as well?
Sara J. Couture: Occupancy rate I think
was our top one, profit margin, and liability assets.
We’re also pretty significant, but occupancy rate was a
pretty significant one for us.
Kristine Sande: This person says, “I was
trying out the ASPE tool when Sara was doing the
demonstration. How are the county data located?”
Sara J. Couture: So for the county
information, we use it based off the hospital cost
reports and then we link it with a variety of different
types of county data to include stuff from the American
Hospital Association data as well as some of the county
rank files. I can provide a link of the full data sources
in the description, if I’m understanding that correctly.
We also did geocode all of our hospitals.
Kristine Sande: All right. And another
question is, “What role does the number of outpatient
service lines play in impacting financial status or
risk?”
Sara J. Couture: I’d have to turn back
on my team on that particular one. I was less involved in
the development of the Cox model myself. So if anyone on
my team wanted to answer that, feel free to.
Mark Holmes: But I think the other thing
I’ll say is Brian Whitacre out of Oklahoma State has done
modeling as well as some teams from ETSU, East Tennessee
State, looking at specifically this history. There’s
another example, this was presented at National Rural
Health Association in May, I think, that really
underscores another example of variables that other
people are considering that we look at and go, “Wow,
that’s pretty clever. We should think about that as
well.” So we don’t have outpatient service lines in our
model today, but do think that that’s something we want
to think about for the future.
Kristine Sande: So on behalf of RHIhub,
I’d like to thank our speakers for the valuable
information and insights you’ve shared with us today. And
thanks also to our participants for joining us.
The slides that are used in today’s webinar are currently
available at www.ruralhealthinfo.org/webinars. So thanks
again for joining us and have a great day, everyone.
