Homelessness is a concern across the United States, both in large cities and smaller communities (U.S. Department of Housing and Urban Development, 2016). Because tensions between community members and the homeless are prevalent in many areas, the first aim of this study was to identify the impact a new permanent supported housing facility had on emergency service use by homeless individuals enrolled in the program. As initial program data was shared with the community, attitudes and responses from leadership began to change, which led to a secondary aim of the study: To describe the interaction between a community and program evaluation data as a means for changing attitudes towards a contentious social issue.
Communities, both small and large, are implementing innovative models to address the crisis of homelessness. Housing First (HF) is a model designed to provide permanent, supportive housing to individuals who have experienced homelessness for a pronounced period; people who have most often experienced co-occurring alcohol or other substance dependencies. HF allows the most vulnerable and chronically homeless individuals to achieve housing with no barriers or contingencies to being housed. This model has shown to be cost-effective by decreasing service utilization, such as emergency room care, decreasing criminal justice activity that brings people into contact with police and court systems, and increased quality of life in areas such as safety (Aubry et al., 2015; Brown et al., 2016; Srebnik et al., 2013). In the fall of 2017, one small community in Alaska (population approximately 30,000) opened its first and only HF modeled facility. Prior to this, there were only six HF programs in Alaska, all located in the larger urban areas. This facility was funded with the enthusiastic support of local social services agencies and some skepticism on the part of the city council and members of the public at large.
Tensions around how this community should address homelessness had developed long before the Housing First (HF) facility opened its doors to tenants. In this small town, as in many communities, a long-standing tension existed in the downtown area, which serves as both a business and tourism hub as well as a centralized, relatively safe locale for people who are homeless to congregate. At the end of 2016, an ad hoc workgroup was formed by the city mayor and was comprised of downtown business owners, city employees, and a local police department official, but excluded agencies and programs working to address homelessness from the discussion. The ad hoc group drafted an ordinance that would ban homeless individuals from sleeping in the entryways of private businesses (DeGrave, 2016).
Before this proposed ordinance, camping had already been prohibited on sidewalks, but entryways of businesses were viewed as private property, which limited the police department’s ability to respond. In effect, this required business owners to file ‘no trespass’ orders on any individual who they didn’t want to sleep in the doorway of their business. In February of 2017, following hours of contentious testimony, the city narrowly passed the ordinance, which allowed the police to remove people from sleeping in entryways and issue a citation and arrest the individual if they did not move (Brooks, 2017).
Conflict flared in the community between business owners wanting to avoid vandalism and trash in the entryways and other community members who viewed this step as criminalizing homelessness (Gullufsen, 2017). After the ordinance passed, it appeared some of the homeless population moved from entryways of business to land close by, which was owned by a mental health-focused foundation. This produced yet more tension within the community and added to the controversy concerning makeshift housing, the rights of people who are homeless, and the rights of business owners to exercise control over commerce (McCarthy, 2017). The foundation gave the homeless campers a two-week notice to vacate and encouraged them to access the local shelter or a campground a mile from the downtown area (McCarthy, 2017).
Amidst this tension and conflict, the first and only HF facility opened in the community in the fall of 2017, lead by a board of directors that were appointed to the Housing First non-profit that had formed in approximately 2015. This non-profit was somewhat unique in that it was led by a partnership between the local homeless shelter, the local mental health center, and the local St. Vincent De Paul Society. It had evolved out of ongoing, cross-agency and community conversations about how to address the challenge of homelessness. Executive directors from all three agencies served on this board, along with members from other social service agencies and local business leaders. After several years of fundraising, program development planning, and community effort, this board oversaw the opening of the new facility. Funding for the HF project came from a mixture of local, state, tribal and federal funds, along with a resource investment by existing social service agencies (staff time, etc).
Some residents viewed the opening of HF as a panacea to the homelessness problem. At the same time, others felt the cost of the facility was far too much money to spend on a population they believed to be making poor choices. The level of high community interest, along with significant tension, established a need for rapid feedback on the outcomes the newly opened facility generated in the community. This demand for accurate outcome data in a short period of time led the HF board to reach out to the local university to design a program evaluation that met these needs.
In the beginning of the process, the research team met with several members of the Board on multiple occasions to understand their goals for an evaluation. Board members requested that the study focus in two directions. First, they were interested in learning if HF resulted in a decreased use of emergency services like the ambulance and the emergency room, as documenting any reduction in service use was viewed as essential to gaining more community support. Second, they were interested in any changes tenants may experience in their quality of life as a result of housing, believing this would be more helpful information to inform program staff on the well being of tenants and support the program as it endeavored to make program improvements. Finally, the Board of directors requested that the design be implemented where data could be collected and reported early, rather than having to wait a number of years before they had outcome data to share. A plan to use existing data from the hospital, the police department, and the local fire and rescue agency was developed, and a data consent process was identified before the study implementation to ensure that the data would be readily accessible and delays would not occur.
In alignment with the goals of the HF board of directors, this study sought to answer the following research question: “Did emergency service use and quality of life change for the residents after moving into HF?” As it is important to the story of community engagement in research, this article also documents the impact program evaluation results had on the community’s evolving response to the HF facility.
Housing First (HF) is a model that allows people who have experienced prolonged homelessness to achieve housing without having typical contingencies, such as stable income or sobriety. The HF model began in 1992 in New York City under a program called Pathways to Housing, and has expanded across the U.S., Canada, and to other parts of the world (Padgett, Henwood & Tsemberis, 2016). HF models provide services such as recovery support, case management, and health care, but housing is not contingent on engagement in these services (Padgett, Henwood & Tsemberis, 2016). The individuals typically served by HF programs face serious issues, including substance dependencies, mental health diagnoses, and high medical needs. This model allows “low-barrier” housing, meaning there is not an expectation of sobriety, medication compliance, or treatment (Stahl et al., 2016; Srebnik et al., 2013).
Decreased Service Use
Chronically homeless individuals have been shown to have increased service utilization in all areas due to the variety of serious health concerns they face (Larimer et al., 2009; Srebnik et al., 2013). In some studies, HF has been shown to decrease some areas of high utilization, especially those that are very costly, such as psychiatric treatment admission and emergency room use (Aubry et al., 2015; Brown et al., 2016; Srebnik et al., 2013). Psychiatric and hospital stays are expensive, and chronically homeless individuals have a higher frequency of repeat admissions than the general population (Aubry et al., 2015; Folsom et al., 2005). Emergency room services are one of the most expensive and most highly used services by homeless individuals (Kushel et al., 2002; Larimer et al., 2009; Srebnik et al., 2013). For some, it is about surviving while living on the street, whether medically, physically, mentally, or a combination of these, it can create a challenge to maintain and sustain primary care, resulting in frequent emergency room use for even minor problems (Kushel et al., 2002). Other HF programs have demonstrated increased stability and decreased emergency room usage by residents (Russolillo et al., 2014; Srebnik et al., 2013). In the HF model, there are support services built in to help individuals with access to services and coordination of different levels of care, which adds to the decrease in emergency room services (Russolillo et al., 2014).
Health and Wellbeing
Housing First residents have been shown to access more primary care and other preventive health care services, which in turn reduce emergency room utilization, hospital admissions, and sobering centers (Gilmer et al., 2010; Srebnik et al., 2013). Some studies have demonstrated that reducing the costs for these more expensive services outweighs the costs to operate a HF residence and therefore prove to be more efficient and effective for the community (Gilmer et al., 2010; Srebnik et al., 2013). When HF models are used, individuals appear to remain housed for long periods, even with ongoing co-occurring disorders or severe medical conditions (Collins et al., 2013; Larimer et al., 2009).
Quality of Life
Some research suggests that HF programs can create a supportive community for individuals, along with a connection to other residents and staff, providing an overall sense of connection to the city, which can lead to positive relationships and increasing self-confidence (Stahl et al., 2016). Having housing available that is “low-barrier” helps foster client choice and creates a sense of control over their life (Rog et al., 2014; Stahl et al., 2016). Research has demonstrated housing can create a sense of control in one’s life, which can also help to increase confidence and autonomy (Stahl et al., 2016). Housing First programs aim to foster individual choice and keep people in permanent housing (Rog et al., 2014) and can decrease the chances of returning to homelessness, even if a resident chooses to leave (Aubry et al., 2015; Collins et al., 2013; Whittaker et al., 2016). When a person has increased self-confidence and choice, they can have more optimism towards the future and a sense of hope (Polvere et al., 2013; Rog et al., 2014). Relationships can also improve with housing, and several studies note that over a more extended period of time, HF residents’ sense of security, stability, and satisfaction with relationships improved (Aubry et al., 2015; Henwood et al., 2014; Whittaker et al., 2016).
Challenges Remain after Housing First
While generally positive outcomes have been consistently found in HF modeled programs, not all study findings point to successful results. Specifically, some studies have found that individuals who become housed express feelings of shame and hopelessness regarding their continued drinking and separation from friends who remain homeless (Polvere et al., 2013; Rog et al., 2014). Levels of ongoing alcohol consumption have also been the focus of several studies, with some finding no significant decrease in alcohol and drug use. In contrast, others have found a reduction over time (Rog et al., 2014).
Although HF has shown a reduction in costly services, food bank usage has seen to increase (Aubry et al., 2015). This correlation suggests HF may reduce homelessness but not necessarily associated issues such as food security. A systematic review of HF outcomes found limitations regarding sample sizes, the use of control groups, and evidence of fidelity (Rog et al., 2014).
Recruitment and Data Collection
Participants in the study were recruited at the point of move-in to the facility beginning in the Fall of 2017. Researchers set up a recruitment table in the shared community area of the new facility and distributed recruitment letters to all residents about the purpose of the study. As this was a new program with newly hired staff, the decision was made by the program to move the new tenants into the facility in “waves” with a small number of residents moving in each week over four weeks.
Additionally, in the first six months of the program, several tenants either passed away from health conditions, moved to different programs to better suit their needs, or were evicted for violence towards other residents. When this happened, new tenants were admitted, and the result was staggered start dates for participants. At each point, a new resident was admitted, program staff notified project researchers who met with new tenants before or within a few days of moving in. At this time, researchers collected informed consent, two self-report measures of well-being (The AST and CSR, described later in this section), and tenants signed consent for research staff to request use of data from local providers (such as the emergency room, the local sobering center, etc.). Participants were compensated with a $20 gift card to a local grocery store that did not sell alcohol, a decision made based on the philosophy of neutrality around alcohol consumption, that the program neither provides nor limits access to alcohol. Each tenant’s move-in date was recorded, and once each study participant reached six months of residency, researchers followed up and re-administered the AST and CSR and again collected use data from the four emergency services providers (ER, police, sleep-off, and ambulance).
Instruments and Data Collection
Emergency services data were collected by giving each provider a signed release of information (ROI) specific to their agency signed by each study participant. The agencies reported participant-specific data that included the date a participant utilized a service and how many times the service was used in the study time frame. In the case of the police department, it was requested that they report each contact (for any reason) they had with participants during the time frame. Ambulance data was counted only if the participant was transported.
The Alaska Screening Tool (AST) and the Client Status Review (CSR) are both tools developed and used by the State of Alaska Department of Health and Social Services and are required to be used by any behavioral health grantee receiving funding from the State. These tools are used to screen and to further lead to the assessment and treatment planning process. They were selected because they address the quality of life content areas identified in the research questions, and they may make the data comparable across broader date sets within the State. Both the AST and CSR were administered at baseline (either a few days before move-in or a few days following move-in dependent on participant availability) and again at six-months post move in.
The AST (SOA-DBH, 2011) is a 37-question screening tool covering topics on symptoms of mental health, substance use, co-occurring disorders, adverse life experiences, fetal alcohol spectrum disorder (FASD), traumatic brain injury (TBI), major life changes, and partner violence. At the beginning of the tool, there are three questions for descriptive purposes, which are birth year, sex, and race. This tool is a self-report measure that asks participants about previous days of symptoms regarding the categories and is intended to be a possible indicator for diagnosis of those problem areas.
The CSR (SOA-DBH, 2014) is a 25-question screening tool that is categorized to focus on health and quality of life, substance use, legal involvement, health behavior, and descriptive information, such as housing arrangements in the last 30 days, schooling, birth year, sex, and race. The first 17 questions are self-reported days of health and safety. Question 18 specifically uses a Likert scale where the participants used a seven-point scale to rate nine domains: housing, ability to support basic needs, safety in your home, safety outside your home, how much people in your life support you, friendships, family situation, spirituality, and life in general. This question is further used for analysis in this study.
Description of Service Use Data Collected
To determine the emergency service utilization, data were acquired from the one local hospital emergency room (ER), the city police department (PD) encounters, nights spent at the local the sobering or “sleep off” center (SO), and emergency medical services transport by ambulance (EMS). Service use data were collected from six months before entering HF, and six months post moving into HF. Emergency room visits were counted by requesting visit data for each enrolled participant from the one community hospital in the community. Police department data was collected from the one community police department, and contacts were counted each time a participant encountered the police, as a victim or perpetrator, not necessarily a citation or arrest. The SO center is also the only sleep off option in the community. They provide a sobering center where emergency medical technicians patrol the street, assisting intoxicated individuals to a safe place for them to sober up for 12–24 hours. The city-run EMS service was contacted for transport data for each participant before and after moving into HF. For the purpose study, data were included only when a participant was transported by the ambulance. Emergency service utilization data were reviewed for how many times participants accessed the ER, the SO, had contacts with PD, and were transported by EMS.
A Wilcoxon Signed-Rank Test was used to compare pre and post service usage data as well as self-reported data on the CSR and AST. A non-parametric method, the Wilcoxon Signed-Rank Test was selected due to the small sample size of 27 participants for service use data and 25 for quality of life data. SPSS was used for statistical analysis as well as to generate demographic data on the population.
This study was approved by the Institution Review Board at the University of Alaska Fairbanks. Participants signed an initial informed consent, and they were asked again for verbal informed consent at each subsequent data collection point. All collected data was de-identified before entry into databases for analysis, and all identifiable data was kept in a locked area approved for confidential storage. Further, it was essential to researchers that participants were honored and recognized in the data analysis process; to this end, researchers met with participants to share the study findings before making them available to the general public. Study participants were interested in the data findings and were pleased by what the data showed. Collectively, the group expressed they were eager to have this data shared with the community and a broader audience.
Twenty-seven participants provided service use and demographic data; however, only 25 participants provided data from the CSR and AST. This difference is due to two participants being unable to complete the re-administration of the AST and CSR due to health-related absences. The average age of study participants was 50 years old, with a range of 31–64 years in age. Seventy-four percent were male, 26% female. Eighty-five percent of participants were Alaska Native/American Indian, 11% were Caucasian, and 4% indicated mixed-race. It is worth noting that this program did not specifically recruit Alaska Native/American Indian participants. Invitations to the program were based on a local Point in Time count, where individuals experiencing homelessness were surveyed and ranked by risk factors. This over-representation of indigenous Alaskans in the program is similar to what is experienced in other systems in the State, such as the child protection and prison systems.
Sixty-three percent report their highest level of education to be either graduating from high school or completing their GED. Fifteen-percent indicates they have a college degree or some college, 11% report having vocational training, with 4% having attended or complete graduate school, and an additional 7% report they dropped out of school before completion of high school or GED.
When asked about employment at the time of move-in, 48% of participants indicated they were disabled and not looking for work. Twenty-six percent reported they were unemployed and looking for work; 19% said they had some type of employment, and an additional 7% indicated they were unemployed and not looking for work. Participants reported the median number of months they had been homeless to be 125, approximately ten years (see Table 1).
|Tenants (N = 27)|
|Average Age: 50 years Old|
|Average Length of Homelessness: 125 months|
|Alaska Native/American Indian||23||85%|
|High School Diploma or GED||17||63%|
|Less than High School||2||7%|
|Unemployed Looking for Work||7||26%|
|Unemployed Not Looking for Work||2||7%|
Changes in Emergency Service Use
As stated, four areas of community emergency services were identified as frequently used by study participants: The local hospital emergency room (ER), the local “sleep off” sobering center (SO), the community police department (PD), and ambulance services (EMS).
Data from the six months prior to participant move-in identified 383 individual visits to the ER by the 27 study participants. This number dropped to 153 emergency room visits in the six months after move-in. A Wilcoxon Signed-Rank Test revealed a statistically significant reduction in the use of the ER. Following participant move into HF (z = –2.388, p = .017) with a medium effect size (r = .46). The mean number of ER uses dropped from pre-HF 14.9 visits to a post-HF mean of 5.67 visits.
The SO center reported providing 357 nights of sleep off to HF residents in the six months before moving in and just five nights of service in the six-months post. A Wilcoxon Signed Rank Test identified a statistically significant difference in use post-move in for this service as well (z = –3.924 p < .0001) with a large effect size (r = .76). The mean number of sleep-off nights dropped from pre-HF 13.22 nights to .19 nights post-HF move in.
Local law enforcement identified 711 individual contacts with study participants in the six months before participants move in and 216 contacts in the six-months post. A Wilcoxon Signed Rank Test documented a statistically significant difference in use pre and post move-in (z = –2.032 p = .042) with a medium effect size (r = .39). The mean number of PD contacts dropped from 17.4 in the six months before move-in to an average of 8 contacts post move in.
Regarding the use of ambulance services, it was reported that participants had 157 individual transports in the six months before HF move-in. In the six months following, this number dropped to 76 transports. Similar to other service categories, a Wilcoxon Sign Rank Test revealed a statistically significant drop in service use (z = –2.308 p = .021) with a medium effect size (r = .44). Use in ambulance transports dropped from a mean of 5.81 transports prior to HF residency to 2.81 transports post-move in (see Table 2).
|Total use number pre||Total use number post||Probability value (b)||Effect size (r)|
To help illustrate the magnitude of this change, researchers tabulated participants’ use data by months (e.g. how many visits, etc.) occurred in each participant’s 6th, 5th, 4th, etc. month before move-in) to visually demonstrate the data trend (see Figure 1). Reconfiguring the data by month allowed community members, funders, and service providers an opportunity to see the magnitude of the decrease and note smaller month by month trends.
Indicators of Wellbeing
In addition to substantial decreases in service use, several indicators of wellbeing also showed statistically significant changes during the six-month time period.
Predictably, HF residents reported that they felt safer “where they sleep” after moving into their permanent residence. Before move-in, participants reported a mean score of 3.31 on a 7-point Likert scale, equal to dissatisfied to mixed ranking. Six-months post-move-in, residents report a mean Likert scale of 5.84, or a satisfied to pleased ranking, with a statistically significant difference, (z = –3.91 p < .0001) and a large effect size (r = .78). Additionally, a significant difference was reported by residents about their safety outside their home. Before move-in, residents reported a mean Likert score of 3.12 (dissatisfied to mixed), and a mean Likert score of 4.68 (mixed or satisfied) after moving in (z = –2.71 p = .007) also with a large effect size (r = .54).
Physical and Mental Wellbeing
Participant self-report of several indicators of health also showed statistically significant post-move-in differences. Participants reported before moving in, they felt their physical health was poor on an average of 19.6 of the last 30 days. Six months after moving in, residents reported an average of only 14.8 days where they felt their health was poor (z = –2.135 p = .033) with a medium effect size (r = .42). Residents also reported improvement in their overall sense of wellbeing, with scores on the question “How do you feel about your life in general?” moving from a mean score of 4.04 (mixed) prior to move into a score of 5.24 (satisfied to pleased) six months post move in (z = –2.978 p < .003) with a large effect size (r = .6).
Residents also reported a significant difference in how they felt about themselves. When asked how many days they “felt bad about themselves, were a failure, or let their family down” participant scores went from an average of 6.58 days (out of 14 days) prior to moving in to an average of 3.2 days six months after move-in (z = –2.170, p = .03) with a medium effect size (r = .43). Finally, it is important to note that residents reported a statistically significant increase in how cut off they felt from others after moving in. Prior to move in, residents report they felt “cut off or distant from people” an average of 2.7 days (out of 14 days). Post move-in, residents report feeling cut off or distanced a mean of 5.44 days z = –2.276, p = .02 with a medium effect size (r = .46).
Participants were asked both the number of days they had one or more drinks as well as the number of days they had four or more drinks in the last 30 days. While numbers of mean days decreased in both categories, only the question of four drinks or more was statistically significant, with 17.38 days prior to move in and 12.84 days post, z = –2.176 p = .03, with a medium effect size (r = .44) (see Table 3).
|Mean Score Pre||Mean Score Post||Probability Value (p)||Effect size (r)|
|Safety where you sleep||3.31||5.84||<.0001||.78|
|Safety outside your home||3.12||4.68||.007||.54|
|Days your physical health was poor||19.6||14.8||.033||.42|
|Feelings about life in general||4.04||5.24||.003||.6|
|Felt bad about themselves, etc.||6.58||3.2||.03||.43|
|Felt cut off or distant||2.7||5.44||.02||.46|
|Drank 4 or more drinks in a day||17.38||12.84||.03||.44|
Changes in the Community
While not reflected in the formal data, during the process of the study, the authors became aware of the intense interest the community had in the project. After the first public sharing of the data in the summer of 2018, the authors of this study were invited to speak at the chamber of commerce, at a local city assembly meeting, and were interviewed by the local newspaper and radio station. Through this public and media attention, the study authors began to observe a shift in attitude from some community members who had been skeptical of the program. This change was primarily observed through more open-minded or favorable newspaper articles, comments posted on a community Facebook page in response to these articles, and emails voicing interest and support sent from community members to the study team as well as to members of the board. After the first round of data sharing, the lead author was invited to sit down with the community’s mayor and discuss homelessness and the city response.
In the late summer of 2018, the HF board was presented with an opportunity to apply for funding to expand the program to include an additional 32 units. However, applying for the grant would require substantial match funding. In September of 2018, less the one year after passing the controversial camping ban, the same city assembly invited researchers to speak on and share the study results. Following the presentation of the data, assembly members unanimously voted to provide an additional $1.8 million to fund the needed match, a decision met by a standing ovation by both the hearing audience and the assembly members themselves (McCarthy, 2018). This moment was a reflection of just how much the community had moved forward in supporting the project and was now focused on investing in evidence-based strategies rather than just the frustration of business owners. Two years post opening, the community attitude about the program can be best illustrated by a local newspaper headline. This article, dated September of 2019, decried “Home Sweet Home: A Once Controversial Housing Program Seems Like a Success” (Hohenstatt, 2019, p. 1).
Following the first round of data sharing, Study authors were also contacted by tribal program managers, a foundation director, and several other non-profit programs to discuss the possibility of developing an affordable outcome assessment for new or existing projects under their purview, demonstrating a growing interest in the power of community-based program evaluation.
Research on all types of program interventions are often understudied in rural communities, and Housing First (HF) is no exception. This paper seeks to add a rural voice to the peer-reviewed literature on HF outcomes. Additionally, study findings were an essential tool for the community to make decisions related to serving individuals experiencing homelessness.
As described, the findings demonstrate a decreased use of all emergency resources in the six months following a move into HF. This is generally consistent with existing findings on HF type programs (Aubry et al., 2015; Patterson et al., 2013; Srebnik et al., 2013). Reduced emergency service use has been the most compelling data to share with the community and funders in this first phase of program evaluation, and study authors were able to provide funders and the city council this data only ten months after HF opened. This quick turn-around can be attributed to developing working relationships with data providers (i.e., the hospital, ambulance service, and police department) and pre-developed data systems that were ready before receiving the data after month six. Having preliminary service use findings to share continued the public conversation and garnered support from those who were skeptical of the program model. These study findings were picked up by local newspapers, and authors received multiple requests to present at local chambers of commerce, community groups, and to the city council. In the fall of 2018, the HF board had the opportunity to apply for expansion funding to double the program size, an opportunity that garnered unanimous city support in the form of matching funds.
The rapid assessment process has also benefited the HF program in providing a process of understanding program strengths and in informing areas in need of process improvement. One poignant data point is demonstrated in the trend line (Figure 1) at month three post move in. The HF program director reported at this time there was a great deal of tension between HF staff and local EMS staff. At month three, the housing director hosted a problem-solving meeting between HF staff and the leadership and direct service providers from local EMS. Through this process, the two groups gained a better understanding of the needs and expectations of each program, and EMS staff were able to assist HF direct service staff with developing a process for deciding if EMS needed to be called. The trend line demonstrates that EMS transports dropped substantially after this process improvement.
Another critical data point used in process improvement by HF the statistically significant increase in the number of days residents felt disconnected after moving in off the street and into their residences. Study authors were able to share data with the program manager that highlighted the unexpected challenges and cultural changes residents were navigating, and this, in turn, has led the program to develop and promote more engagement activities and events to help residents build new support systems and feel more connected. This data point also led HF staff to incorporate more indigenous focused activities to better engage the community; activities such as talking circles, traditional beading, and other arts.
A final benefit of this rapid assessment has been a general promotion of the use of program evaluation as a doable goal for small, minimally funded programs. City housing officers have expressed an interest in establishing evaluation plans for other new housing programs in development, and the HF staff checks in regularly for data updates and feedback. In this way, the community has been able to use, understand, and appreciate the program evaluation process. Some have changed their perception about the role and value of data and evaluation in the programs they run.
There are several limitations to this study. First, while the sample of 27 (and 25 in the wellbeing data) is close to a complete census of the 32-resident unit, it is still a small sample size. As not all residents were able to participate, findings are not a reflection of the experiences of the entire population of this facility. Additionally, this study does not have a control group, which limits the ability to attribute the changes exclusively to the intervention. Finally, it is essential to note that the changes are reflected for a relatively short time-frame, six months before and after moving into the HF facility, and ongoing evaluation may ultimately find different long-term service use and wellbeing outcomes.
Implications For Housing Policy and Service Provision
Data from this study demonstrated significant changes in both uses of community emergency services as well as an increase in the overall wellbeing of the tenants following their receipt of permanent supported housing. This strong correlation between establishing housing and decreasing expensive emergency service use was the primary goal set by the HF board when initially requesting this evaluation. By design, the HF leadership and community leadership were able to access this data a mere ten months after the opening of the facility, and this data was shared widely in the local community news outlets and city planning meetings. In this case, the outcome data on service use has been a tremendously useful tool for garnering increased community support for a practice model (HF) that was met with some resistance and skepticism in the community. This highlights the value that a rapid, community-focused program evaluation can have on the direction of policy discussion, informing community citizens with data to inform their attitudes and beliefs towards a data-informed approach and away from moralizing viewpoints.
For the HF agency itself, the data on wellbeing has helped shape service delivery and inform the project as they focus on staff training, community relationships, and program improvement. By meeting regularly with program staff and helping staff talk through the implications of the data, programming has evolved to expand tenant access to more community engagement, and outreach efforts to service providers, like EMS, have increased. Finally, since the data was made public, the study authors have been contacted by multiple service providers, the municipal housing authority, and a local foundation about designing and implementing outcome evaluation in their programs. This suggests that the community entities and service providers have developed a more positive attitude towards program evaluation, and, in this way, this rapid assessment process has been beneficial at both the service delivery and policy level.