Problem-solving is a critical skill for any interpersonal interaction and is essential for navigating, understanding, and successfully interacting with one’s social world (Vlachou & Stavroussi, 2016). The summer camp setting involves a number of relationships and interpersonal dynamics across multiple power-differentials (i.e., campers, counselors, and staff). Competency in problem solving ability may be particularly salient given these dynamics. Overall, participation in a summer camp setting has been shown to yield positive results for child campers (Henderson, Whitaker, Bialeschki, Scanlin, & Thurber, 2007) and counselors (Carter & Kotrlik, 2008). The American Camp Association estimates that 14 million children and adults attend camp in the United States each year (2013). Further, longitudinal data has suggested that participation in 4-H is associated with healthy and adaptive outcomes (e.g., increased self-regulation) within youth (Lerner, Almerigi, Theokas, & Lerner, 2005).
4-H and Summer Camp Setting
The 4-H organization is America’s largest youth development program with a network of 500,000 volunteers and 3,500 4-H professionals that empower nearly six million young adults nationally (4-H Organization, n.d.). The organization views each child as having a strength that can influence change and engages them in hands-on projects in areas such as health, science, agriculture, and citizenship. Additionally, the 4-H organization is delivered by a Cooperative Extension System, which includes over 100 public universities across the nation that occur in settings including in-school and after-school programs, school and community clubs, and 4-H camps, the focus of the present study.
To foster positive experiences for campers and counselors, summer camp staff undergo a variety of training experiences to effectively manage camper behavior and provide a positive experience to participants. Training developed internally by community organizations may suffer from a relative lack of awareness of empirical evidence regarding effective behavior management skills. Relationships between community organizations and university departments with expertise in these areas may be particularly beneficial in the summer camp setting given that staff members interact not only with campers, but also with counselors and other staff members. Multidirectional relationships might be especially vulnerable to interpersonal conflicts that arise among campers, counselors, and staff. Thus, adult camp staff members need to be well-versed in both camp policies and specific strategies for dealing with such issues.
Given the complexity of levels of interactions and ever-changing environmental demands faced by summer camp staff, there are endless skills, methods, and knowledge that can be imparted to increase efficacy of the position. Teaching specific skills that can be generalized to a variety of situations may yield greater and longer lasting benefits than focusing efforts on specific skills that can only be used in niche circumstances. Many cognitive-behavioral coping programs, such as Problem Solving Therapy (D’Zurilla & Nezu, 1982), were created to help acquire such skills. Thus, there is growing recognition of the need to assess various training methods that allow laypersons to better serve the community. Training modalities that focus on the process of interpersonal relationships, rather than idiosyncratic content issues, may provide a foundation that can be applied across settings and situations. The current study uses an implementation research framework (Damschroder et al., 2009), with a focus on implementation for adoption of evidence-based practices, in collaboration with the 4-H organization.
Problem Solving Therapy
D’Zurilla and Nezu (1982) have defined effective problem solving as the capacity to identify effective ways of addressing everyday problems. Many studies apply problem solving training to target a range of difficulties, including oppositional child behavior, stressful life events, and daily stressors (Heppner, Pretorius, Wei, Lee, & Wang, 2002; Kazdin & Weisz, 2010). Problem-solving therapy (PST) teaches problem solving strategies and focuses on employing a cognitive-behavioral intervention to employ approaches for individuals to identify effective coping with problematic situations (D’Zurilla, 1986; D’Zurilla & Nezu, 1982). These are often difficulties encountered in the course of everyday living, for individuals who have clinical or non-clinical presentations. The foundation of this approach centers on a relational/problem-solving model of stress and well-being that includes physiological, social, and health-related dimensions (D’Zurilla & Nezu, 1999, 2007). With this framework, it is posited that problem solving is critical to navigating the relationship between stressful life events, which can be major events or daily interactions, and one’s well-being (D’Zurilla & Nezu, 1999, 2007). Specifically, PST focuses on taking a positive problem solving orientation (e.g., viewing problems as challenges rather than threats); challenging dysfunctional attitudes toward problem solving; regulating negative emotional responses; generating alternative solutions; evaluating solution outcomes non-judgmentally; and self-monitoring. A meta-analysis of PST randomized controlled trials revealed that PST is most effective when it includes all of the key components mentioned above, rather than problem solving skills only, as well as the problem solving orientation (Bell & D’Zurilla, 2009).
PST has been applied as a means of reducing distress across diagnostic and non-clinical populations. For example, mothers with a newly diagnosed child with autism spectrum disorder (ASD) who participated in PST showed significant decrease in distress (Nguyen, Fairclough, & Noll, 2016) and adult cancer patients and their partners showed sustained reductions of distress six months after treatment (Nezu, Nezu, Felgoise, McClure, & Houts 2003). Though this approach has been employed with many populations who have experienced everyday distress, it has not been studied in camp staff members who often face difficult situations in a fast-paced milieu.
Problem Solving Therapy in the Community
Until recently, the majority of intervention and training outcome research has focused on establishing efficacy under carefully controlled laboratory settings (Chorpita, 2003). However, treatments that perform well in lab settings often do not translate as readily in real-world contexts (Chorpita, 2003). This discrepancy in training dissemination has led to calls of tailoring training to non-clinical contexts, including camp-based programming (e.g., Carney & Nottis, 2008; Santucci, Thomassin, Petrovic, & Weisz, 2015). As outlined by Schafer (2007), when specifically educating camp staff members, training programs must incorporate the needs of staff, utilize active learning techniques, and be directly tethered to a developmentally appropriate framework. Further, some propose that increased involvement of stakeholders, including community organizations, from the beginning stages of planning and program implementation, increases the positive impact that a community can have to protect children and prevent neglect and abuse (Harper, Echohawk, Bigpond, Cloud-Koenen, & Slotted Eagle, 2002).
In addition to implementing treatments in real-world settings, there has also been an emphasis in exploring partnerships between universities and community organizations in applying research (Buys & Bursnall, 2007). Maintaining these partnerships is complex and can involve time and effort to coordinate these collaborations and the notion that the work benefits both agencies involved is also paramount to success (Suarez-Balcazar, Mirza, & Hansen, 2015). A multi-stage process that involves talking to stakeholders, data collection, and dissemination has been outlined and supported in working with children (Harper et al. 2002). Additionally, it is paramount to understand and address biases that influence the decision making process at each step, as outlined in the transformative paradigm (Mertens, Holmes, & Harris, 2009).
As community samples transition from the role of participants to that of collaborators, it might be that results of such research are more translatable and culturally relevant to real-world situations (Jacquez, Vaughn, & Wagner, 2013). Collaborations between universities and community organizations realizes the concept of “engagement,” which has been shown to be critical in furthering research as well as in influencing policies and practice (Coldstream, 2003, p. 6). Further, research has suggested that the most important factor in successful university and community partnerships is interpersonal relationships (Buys & Bursnall, 2007).
This paper focuses on the collaboration between the authors’ affiliated university (Virginia Polytechnic Institute and State University) with the 4-H organization, which can be divided into four distinct phases. The partnership began in Phase I, with the 4-H organization requesting training in psychoeducation and behavior management strategies specifically for children with ASD from the university, which was presented to over 80 4-H Extension Agents. Several in-person meetings with the Camping Specialist for the 4-H department of Virginia Cooperative Extension were arranged to fine-tune presentation materials to incorporate challenges faced by staff. Specific training needs included ASD psychoeducation and how to make camp inclusive for all individuals. Phase II lasted for two years, starting directly following the initial training. Requests were made by individual 4-H Extension Agents to have similar trainings (i.e., psychoeducation of ASD, behavior management strategies) presented to teenage camp counselors and adult staff. To best tailor each talk to the specific audience, 4-H Extension Agents provided examples of difficult situations commonly encountered at camp and the Clinical Science doctoral psychology students and faculty adapted psychoeducation materials and behavioral management skills.
In Phase III, a 4-H Center Program Director for Southwest Virginia region requested a broader training of behavior management strategies for teen counselors and adult volunteers who would directly supervise campers for one week of summer camp. The university team worked with the 4-H Center Program Director to identify the unique challenges for the teen counselors and adult volunteers, and presented ideas from evidence-based practices, specifically using behavioral-based strategies for internalizing and externalizing behaviors. To evaluate these trainings, consider their potential for adaptation to other settings, and contribute to extant literature on behavior management training in community settings, the university team began collecting data pertaining to efficacy and knowledge. Preliminary data showed high ratings of satisfaction and pre to post increases in confidence for managing campers’ problem behaviors (unpublished). Finally, Phase IV, the focus of the current study, included a similar request for behavior management strategies for a different population (i.e., 4-H Center summer staff). These staff members work throughout the summer at a specific 4-H Educational Center and oversee campers, teen counselors, and adult volunteers. In Phases I through III, the method for determining trainees’ needs and selecting the appropriate training included only the university team and the individual who requested the training. To better cater the training to the challenges perceived by 4-H Center summer staff, a needs assessment was utilized for the current study. This study addresses the call to tailor training to specific populations (i.e., camp staff members) with specific expertise (i.e., each side providing unique value to the partnership).
Aims and Hypotheses
The goals of the current study were to (a) assess the training needs of adult staff from multiple Southwest Virginia 4-H Centers related to interpersonal skill knowledge (e.g., with campers, counselors, other staff); (b) find and adapt evidence-based practices to address these specific needs; and (c) report preliminary outcomes of a novel training program to examine feasibility. Specifically, results examined changes in perception of problem solving ability and knowledge of PST materials. We hypothesized that the training model (described below) would increase 4-H Center summer staff’s problem solving ability and knowledge of PST materials. Exploratory data was also collected at a follow up time point to assess skills after implementing them at camp.
Phase IVa: Needs Assessment
Study Design. The study implemented a two-phase design to create and implement a comprehensive and relevant training series to 4-H Center summer staff. The primary aim of the first phase was to determine, through a participatory process of stakeholders, the challenges and needs encountered by camp staff members from Southwest 4-H Centers. This data was obtained to facilitate the creation of in-person training materials.
Staff completed an anonymous survey to collect data on perceived training needs. A convenience sampling method was used to gather representative areas of struggle and need. Inclusion criteria consisted of self-identification of past or future employment at a Southwest VA 4-H Center as a summer staff member. An anonymous online survey was created in REDCap (Research Electronic Data Capture; Harris, Taylor, Thielke, Payne, Gonzalez, & Conde, 2009), a secure, web-based application designed to support data capture for research studies. Data collection occurred over a two-week period. After receiving the online survey link, individuals had to consent to participate in the study before they were prompted to answer demographic and online survey questionnaires. If they chose not to consent, they were thanked for their time and shown an exit screen. The institution’s human research ethics committee approved this study.
Participants. Any 4-H Center summer staff member was eligible to participate in the online needs assessment study. Fifty-one adults began the survey and 41 individuals completed the online survey (see Table 1 for demographic characteristics). Participants who did not finish the survey were excluded from data analysis. Participants averaged 20.66 years of age (range = 18–54 years, SD = 5.30 years) and had approximately 5.27 years of experience working in a camp capacity (range = 0–26 years, SD = 4.07 years). All participants who completed the online needs assessment were invited to participate in the training described below. Given that the online needs assessment was anonymous, we are unable to determine whether these same individuals who completed the needs assessment also participated in the training.
|Variable||Needs Assessment (n = 44)||Pre-Behavioral Training (n = 80)|
|Less than $24,999||14||31.80||24||30.00|
|$25,000 to $49,999||10||22.70||23||28.80|
|$50,000 to $99,999||14||31.80||16||20.00|
|More than $100,000||6||13.60||17||21.30|
|Hispanic or Latino||2||4.50||6||7.50|
|Not Hispanic or Latino||41||93.20||74||92.50|
|Black or African American||5||11.40||15||18.80|
|Highest education completed|
|Some college/technical school||32||72.70||65||81.30|
|Bachelors/4-year college degree||2||4.50||4||5.0|
|Prior participation in 4-H Behavioral Management Training|
|Variable||n||M (SD)||n||M (SD)|
|Age (years)||44||20.66 (5.30)||78||19.81 (1.38)|
|Years of experience as camp staff||44||5.27 (4.07)||79||4.80 (2.76)|
Evaluation Measure. The online survey content was developed based on the 4-H camp training staff competencies (i.e., specific training topics) and theory (e.g., D’Zurilla, 1986; D’Zurilla & Nezu, 1982). The survey took an average of 17 minutes to complete (range = 5–89 minutes) and consisted of 77 questions, which were created for the purpose of this needs-assessment survey. Domains assessed included: positive engagement with campers and staff; problem solving with campers, teen camp counselors and adult camp staff; managing bullying situations; being an effective leader; identifying multiple mental health concerns; handling loneliness and isolation; fostering peer social support; managing camper conduct and behavioral issues; managing intense emotions of campers; managing camper adaptive skills and attention; disciplining campers; advocating for personal needs; displaying self-confidence; knowing when to intervene; and enforcing the rules. Participants were instructed to rate the following: (a) how often situations result in difficulties or challenges for camp staff, (b) how personally challenging situations are in the camp setting, and (c) additional training domains requested. These domains were assessed quantitatively on a five-point scale for frequency (1 = Never a problem for me, 5 = Almost always a problem for me) and intensity (1 = No problem/challenge for me, 5 = A huge problem/challenge for me), qualitatively (e.g., “Describe areas of difficulty”), or via a percentage designation (e.g., “Please rate your level of interest in the following topics”) on a scale from 0 to 100. The study specific measure may be available from authors upon request.
Study Design. The secondary study aim was to create an applicable training and subsequently examine potential changes in adult camp staff’s knowledge and competencies following the training. Results of the needs assessment indicated the need for a training that focused on a generalizable skill for multiple populations versus skills for a specific population (e.g., social skills for children with ASD). As described above, the university team presented the training idea, framework, and rationale to the 4-H Center Program Director. After careful consideration and collaboration, PST strategies were selected and adapted for the training.
All participants provided informed consent during the start of the pre-assessment. The training was free of charge to the 4-H organization, as they agreed to participate in research regarding this training. Following completion of the pre-training assessment, adult 4-H Center summer staff were invited to participate in a training which addressed how to best apply problem solving techniques across the three scaffolded levels of 4-H personnel (e.g., campers, teen camp counselors, and other adult staff). To more specifically delineate staff roles, 4-H Center summer staff are directly supervised by the 4-H Center Director and actively supervise child campers as well as teen camp counselors. During the training, participants were broken into three groups and rotated through each to discuss and practice the application of skills as they relate to different interpersonal interactions with campers/staff. This workshop took place amidst a day of training for staff members. After workshop completion, 4-H Center summer staff completed a post-training assessment to examine any changes in responses following the workshop. Lastly, a follow-up assessment was sent three months following the training to examine whether the workshop had a lasting impact on problem solving behaviors and competencies and whether problem solving strategies were implemented during camp. For a visual depiction of the number of participants in each phase, see Figure 1.
Training. 4-H Center summer staff participated in a 3-hour training led by members of the research team. Training leaders (n = 6) were graduate students in a clinical psychology Ph.D. program (Mage = 23.83 years; females = 5). The training was centered on addressing concerns noted on the needs assessment using a problem solving oriented framework, as defined by principles consistent with PST (Nezu, Nezu, & D’Zurilla, 2013). Sessions focused on the following guiding principles consistent with PST: taking a positive problem solving orientation (e.g., viewing problems as challenges rather than threats); challenging dysfunctional attitudes toward problem solving; regulating negative emotional responses; generating alternative solutions; evaluating solution outcomes non-judgmentally; and self-monitoring. These components were conveyed in an acronym SSTA (i.e., Stop, Slow Down, Think, and Act; Nezu, Nezu, & D’Zurilla, 2013) to help staff recall the steps. Further, adult 4-H Center summer staff were instructed on how to recognize problems when they occur, gather relevant information about the problem in order to set a realistic goal, maximize the likelihood of an effective solution, and evaluate the effectiveness of the solution. Lastly, a self-monitoring approach was introduced in order for staff to objectively evaluate the solutions outcome.
First, all 4-H Center summer staff attended a 10-minute presentation, which focused on conveying the results of the needs assessment, as well as general PST principles. Following the presentation, the PST principles were further explored within three separate 35-minute breakout sessions to address noted concerns across the three levels of 4-H personnel (e.g., campers, teen camp counselors, and all higher level staff members). Examples discussed in each breakout session were topics ranked highly in the needs assessment or individual concerns voiced by participants completing the needs assessment.
Participants. Participants were adult 4-H Center summer staff who were completing a general staff training in preparation for summer camp. A total of 80 individuals completed the pre-training survey (see Table 1 for demographic characteristics). Initially, 86 people began the pre-training assessment; however, six participants did not finish the survey and were excluded from data analysis. Participants averaged 19.81 years of age (range = 18–26, SD = 1.38 years) and had approximately 4.80 years of experience working in a camp capacity. The majority of participants were female (63%), white (81%), and had completed some college (81%). A total of 73 adult 4-H Center summer staff completed the post-training assessment. Lastly, nine adult 4-H Center summer staff completed the follow-up assessment 3 months after the training.
Problem Solving Inventory (PSI; Heppner & Petersen, 1982). The PSI is a validated and widely used self-report measure (α = .90). The PSI consists of 32 items, which constitute three factors: Problem-Solving Confidence (i.e., confidence in ability to effectively solve problems), Approach-Avoidance Style (i.e., approaching versus avoiding problems), and Personal Control (i.e., self-control of emotions and behaviors when attempting to solve problems). Participants were instructed to respond to the 32 items in Likert fashion (1 = Strongly Agree, 6 = Strongly Disagree), indicating the extent to which they agree or disagree. The PSI was obtained before the training, following the training, and at a three-month follow-up time point. Lower scores reflect better problem solving abilities. Internal consistencies were acceptable across time points for the factors assessed (Cronbach’s alpha = .82 – .87).
Problem Solving Therapy Knowledge Questionnaire (PST-KQ; unpublished, available upon request). The PST-KQ was created for the purposes of the present study to gauge participant understanding of PST concepts. Questions were developed prior to the workshop by the project’s staff and were formatted to be consistent with principles discussed during the training. For example, as part of the survey, participants were asked to rate: Which of the following is an example of a specific goal? A) I want to excel at my job, B) I want to set at least 2 clear expectations for my campers during our meeting tonight, C) I want to be a good model for the teen camp counselors, D) I want to have fun as a staff member. Participants were prompted to answer ten questions, which were presented in both true/false and multiple choice formats. The same questions were presented on the pre- and post-assessments to gauge change in knowledge of PST concepts from before the training relative to following the training.
Satisfaction Survey. In addition to standardized measures, a series of open-ended questions were included during the post-training survey to allow participants to report components of the training they found particularly helpful, those they did not enjoy or find helpful, and aspects that they would change if they were to repeat the training. Participants were also asked to rank the level of satisfaction and perceived usefulness of the training on a 0 to 100 scale as well as whether or not they would recommend it to someone else.
Problem Solving Self-Monitoring Form (PSSM; adapted from Nezu, Nezu, & D’Zurilla, 2013). This form was adapted from the PST manual to apply to solving problems in the camp setting. During the follow-up survey, staff members were instructed to record information about a significant problem that occurred during the summer while working as a 4-H Center Staff member and to explain how they attempted to solve it. They were asked to describe a problem, including involved parties, type of problem, and their goal in solving the problem.
Phase IVa: Needs Assessment
Phase IVa focused on determining difficulties encountered by 4-H Center summer staff in their roles at 4-H summer camps as well as their general interest in training topics. The results of the needs assessment were used to inform the training, a strategy highlighted in multiple domains of work (Altschuld & Witkin, 1999; Reason & Bradbury, 2005). For example, respondents identified that they frequently encountered situations where they needed to display self-confidence in everyday situations (M = 1.93, SD = 0.97), advocate for their own personal needs with other camp staff (M = 1.89, SD = 0.92), and problem solve with other camp staff (M = 1.8, SD = 0.73) and teen counselors (M = 1.73, SD = 0.62). Qualitative responses echoed the frequency of these domains. For example, “being confident in myself” was frequently endorsed among respondents, as was “managing relationships with co-workers.” The theme of problematic behavior emerged with great frequency and encompassed several domains including managing anxiety disorders/depression in campers (M = 2.02, SD = 0.88), camper attention (M = 1.95, SD = 0.86), and intense camper emotions (M = 1.86, SD = .85).
In addition to the frequency with which they encountered specific situations, 4-H Center summer staff also reported on the difficulty and challenge of situations regarding camp members on a five-point scale (1 = no problem/challenge for me, 5 = huge problem/challenge for me). The challenging nature of the camp situations largely echoed the endorsed frequency. Identifying anxiety/depression mental health concerns among campers was highly rated situation (M = 1.89, SD = .81). 4-H Center summer staff found managing the intense emotions of campers to be challenging (M = 1.86, SD = .73), in addition to handling camper loneliness and isolation (M = 1.82, SD = .97). Qualitatively, participants noted, “I find it to be difficult more than anything to work with homesick kids.” Staff also endorsed the difficulty in advocating for their own personal needs to other staff members (M = 1.82, SD = .90) and self-disclosed difficulty in handling both camper needs as well as their own needs.
Individuals were next asked to indicate which interpersonal skills and behavior management techniques they believed were most important and relevant to their work, as well as those they were most interested in learning about. Again, emerging themes related to interpersonal engagement and managing interactions at multiple levels (e.g., staff-camper, camper-camper, and staff-staff), in addition to concerns about identification of mental health problems. Although mental health concerns were reported as the most desired topic (M = 75.25, SD = 25.76), items related to positive engagement (M = 72.23, SD = 25.11), problem solving (M = 68.68, SD = 22.00), and leadership (M = 68.68, SD = 29.18) were also among the most-desired topics. Topics rated as the most- or second-most important included the same items pertaining to engagement (21% of sample rated as most important) and leadership (16%). Identifying mental health concerns was rated as a desired (M = 75.25, SD = 24.77) and important topic (11%).
Qualitatively reported descriptions of challenges further underscored the desire of staff to receive training to improve their ability to solve multiple types of responses. Several responses indicated a need for help in problem solving ability as well as experiencing interpersonal difficulties. For example, one staff member indicated a need to understand how to prioritize issues that arise with campers and specifically “how to deal with multiple needs all at once with different variations of the problems.” Several responses focused on interpersonal communication skills (i.e., how to effectively communicate with supervisors, how to voice complaints) and forming valuable connections with peers, while balancing the professional and social roles of the job. Finally, multiple staff indicated that they sought instruction on how to effectively communicate opinions of frustration with peers and advisors.
Phase IVb: Pre-Post Data
Paired-samples t-tests were conducted to compare pre- and post-training ratings of problem solving confidence, approach-avoidance style, personal control, and knowledge of problem solving training (see Table 2 for details). There was a significant difference in the scores for approach-avoidance style (t(68) = 4.14, p < .001), personal control (t(68) = 2.66, p = .010), and knowledge of PST (t(66) = -3.68, p < .001). There was no significant change in problem solving confidence (t(67) = 1.02, p = .310).
|Measure||Pre-Training M (SD)||Post-Training M (SD)||t||p|
|PSI: Confidence||24.91 (8.19)||24.21 (12.88)||1.02||.31|
|PSI: Approach-Avoidance||45.26 (10.19)||41.62 (12.88)||4.14||<.001|
|PSI: Personal Control||16.29 (3.38)||15.36 (4.19)||2.66||.01|
|PST-KQ||.67 (.18)||.75 (.18)||–3.68||<.001|
In addition to analyses evaluating change in problem solving confidence and knowledge from pre- to post-training, we examined responses to ratings and open-ended items regarding components of the training that participants liked most and least, as well as those they would change if they were to repeat the training. Overall, participants appeared satisfied (M = 80.88, SD = 15.36, range = 28–100) and rated the training as helpful or useful (M = 80.23, SD = 17.21, range = 27–100). Finally, 97.3% of participants (n = 71) indicated that they would recommend this training to another 4-H summer staff member in the future.
From a qualitative standpoint, a substantial proportion of respondents reported that they most enjoyed the group-based, interactive nature of the training, including the extent to which participants were provided with opportunities to engage in discussion. One participant noted that they “…liked how [researchers] broke us up into different groups and kept us moving through the time they had us instead of doing one thing the entire three hours. It kept us engaged and alert throughout the presentations and learning.” Despite the frequency with which respondents indicated that they enjoyed the interactive nature of the training, many participants reported that the aspect they liked least was the length of training, that content felt repetitive, or that they felt “…like we were being talked at.” Finally, participants reported on aspects of the training they might like to change. Most frequently, participants reported that there was nothing they would change. Those who did suggest changes recommended reducing the size of each of the three groups, providing more specific and detailed examples, and making groups more interactive.
On the follow-up questionnaire, participants were asked to complete the PSSM in which they described a problem in detail. In addition, participants rated the extent to which they found the problem to be a challenge, their perceived ability to change the situation for the better, how intense their negative feelings about the problem were, and the extent to which the situation was ultimately changed for the better.
Of the 73 participants that completed the post-training questionnaire, nine completed the PSSM with responses based on a problem they had encountered in their work throughout the summer as a 4-H Center staff member. Seven of nine (78%) problems described by participants were interpersonal in nature, including disputes or other difficulties associated with interpersonal relationships. The remaining two consisted of one domestic problem (i.e., the individual’s living situation in a camp setting) and one personal problem (i.e., involving personal issues focused primarily on the individual’s thoughts, feelings, or behavior in non-social aspects of their environment). Problem-solving goals as reported by participants varied widely and frequently consisted only of further description of the problem. However, some participants were more specific, including one who described their goal as “[To] change the way the staff member spoke with me when there was an issue.” Another reported that they “…wanted to not only get [campers] to stop arguing but to have them get along before the end of the week.” In terms of their appraisal of the problem solving challenge they reported on, participants reported on a four-point scale (1 = no challenge, 4 = major challenge) that problems were generally between minor and moderate (M = 2.67, SD = .94). Regarding outcome expectancy, participants demonstrated a moderate (M = 2.56, SD = 1.17) belief that the problem could be changed and a high level of confidence that they had the ability to control or improve the situation (M = 3.22, SD = .79).
Next, participants rated the extent to which the situation changed for the better on a seven-point scale (1 = very much improved, 7 = very much worse). Participants generally reported that the situation improved somewhat (M = 2.89, SD = .99). Satisfaction for the overall outcome was rated on a 0-100 scale, with participants reporting moderately high satisfaction, (M = 64.17, SD = 25.17). Finally, participants reported whether they had used the “Stop, Slow Down, Think, and Act” model in solving the described problem. Four of nine (44%) reported that they used the model in the example described in the PSSM form, while five (56%) indicated that they had used the model in other situations throughout the summer. Two participants (22%) reported that they had incorporated other aspects of the training into their experience as 4-H Center summer staff, including one who reported that they “…handled problems more openly and tried to hear people’s opinions before acting upon [their] own judgments.”
The current study serves as a training model illustrating the successful assessment of client need and adaptation of evidence-based materials. Moreover, this serves as an example of a community partnership that demonstrates training and research in an applied setting. Overall, findings replicate previous reports suggesting participating in a camp setting increases levels of self-efficacy and problem solving (Carter & Kotrlik, 2008). To our knowledge, this is the first study to evaluate a problem solving training in a camp setting, with targets of interpersonal difficulties and behavior management. Additionally, this study applies the PST framework within an organizational context, with the assumption that it will then be implemented to specific relationships. Thus, this provides initial support that PST can be taught within an organizational framework and what assessments might be utilized for targeted training. Preliminary findings suggest a need for adapted training materials and feasibility of implementation to a large group of 4-H Center summer staff members.
Results from the needs assessment indicate that the need for effective coping skills to target a myriad of interpersonal difficulties is increasingly at the forefront of camp staff’s awareness, and 4-H Center summer staff members are challenged in determining how to best handle the situation. Many staff members indicated the need for training on how best to attend to the interpersonal difficulties emerging across multiple levels of communication to ensure the best possible experience at camp. Needs assessment results revealed that staff members not only experience difficulties with emotional and behavioral problem behaviors in campers but also with conflicts among counselor and other adult interactions. Results indicated high levels of satisfaction and perceived helpfulness. Significant changes in pre- to post-training knowledge of PST material support the efficacy of gaining skills during a brief training session. Therefore, it seems that the training is effective in teaching these skills. Additionally, 4-H Center summer staff significantly improved their rating of approach-avoidance style and personal control. More specifically, after the training, staff members reported more willingness to deal with and less tendency to avoid difficult situations. Further, changes in scores on personal control suggest that the training led to more positive perceptions of one’s ability to control problems, similar to the concept of self-efficacy. There was no difference in pre- and post-training scores on the Problem-Solving Confidence subscale of the PSI. The questions in this subscale assess the participant’s perceived belief and confidence in his or her ability to effectively solve problems. The training may have illuminated difficulties in problem solving skills that the staff were not previously cognizant about when assessing their pre-training levels of confidence. However, after the training, they may have become more aware of these difficulties and nuances of problem solving, thus this did not lead to a change in confidence captured on this measure.
University and Community Center Collaborations
This work also highlights the ability for a university and a community-based center to collaborate to further research as well as enrich the training experiences for summer camp staff members. The ability to effectively implement the brief workshop provides evidence and momentum for more future collaborations of this sort. Furthering work that explores these university-community partnerships, both in feasibility and in the benefits and outcomes they produce, is essential in establishing sustainability of such work (Suarez-Balcazar et al., 2013). Previous research has indicated that university-community partnerships lead to benefits including in research and learning, on the university side, and in organizational recognition and learning as well for the community organization (Buys & Bursnall, 2007). While university students and faculty often receive training and demonstrate efficacy in a variety of evidence-based practices, specific to a certain skill or population, we encourage the university-side of the partnership to work collaboratively with the identified local organization to best select and adapt the evidence-based practice to meet the needs of the organization members. Continuing work between 4-H centers or other local organizations and universities, beyond the two featured in the present study, could result in further dissemination of problem solving strategies and might continue to improve on services and trainings that are currently offered.
Bringle and Hatcher (2002) presented the idea of campus-community partnerships being analogous to interpersonal relationships. The current study provides an example of relationship initiation, development, and maintenance. Within the context of initiation, the current study provides an example in which the community began the partnership; however, similar collaborations could also be commenced by campus researchers. Regardless of the directionality of initiation, strong consideration should be given to compatibility and communication about objectives and goals.
Although results from this training are promising in suggesting that the application of PST to camp staff training might be beneficial, the study is not without limitations. Notable in the interpretation of these comments is the fact that the training occurred at the end of a full (approximately 8-hour) day of trainings on other aspects of working as a 4-H staff member. Specifically, the use of the PSI to demonstrate pre- and post-training relies on the participants’ perceptions of skills as opposed to measuring problem solving techniques in a more objective manner (e.g., live observation). For this reason, we incorporated the PSSM form in the follow-up survey; however, due to the decreased response rate, results should be viewed with caution. The lack of response to the follow up survey is also a limitation, as we were not able to fully assess the way in which staff members implemented the skills and techniques taught in the training. In addition, the anonymous nature of the needs assessment made us unable to determine the extent to which the sample who completed the needs assessment overlapped with those who completed the training. Therefore, the degree to which the expressed concerns on the needs assessment were met during the training might not have been fully indicative of the opinions expressed by the original sample. Further, certain aspects of the community-university partnership may have contributed to this limitation. For example, ongoing communication between participants, supervisors, and university researchers may have increased investment in the research process.
Additionally, it is important to note that a multi-session intervention was consolidated into a single 3-hour training. The brief nature of the training may have resulted in changes of perception of abilities; however, it may not have provided sufficient time for the consolidation of information and the practicing of skills with feedback. Finally, although staff members were eager to learn about proper identification of and techniques to work with children with mental health concerns, they did not rank these situations as frequently occurring. The discrepancy in ratings supported our decision to teach general problem solving techniques that can be used in a variety of situations as opposed to highly specialized techniques that apply to specific situations.
Results of this study indicate that collaboration between graduate students at a large public institution and 4-H Center Staff in the state of Virginia produced didactic learning in a workshop setting in addition to later generalized problem-solving application amongst participants. The multi-phased approach of the study allowed for stakeholder-input and an opportunity to address the specific needs and desires of participants in a targeted manner. We view the inclusion of 4-H Center staff voices into the researcher-developed didactic training as a primary strength of the research design and a true example of a collaboration between a university and community organization. Specifically, by using this multi-phased approach, participants contributed to the second phase research questions and gave a framework for what content was deemed important and relevant to their individual needs. The incorporation of the needs-assessment findings not only engaged the staff members to voice their opinions and requests for help, it allowed the trainers to better understand and adapt the evidence-based techniques to specific camp settings. This model follows a patient-centered framework where including the perspective of the end-user may improve the likelihood of implementation (Frank, Basch, & Selby, 2014; Stewart et al., 2000). This model also realizes the notion of taking action based on assessment (Altschuld & Witkin, 1999; Reason & Bradbury, 2005). Thus, we argue that gathering perspectives of the target audience prior to programmatic implementation is an essential component, not just to promote outcomes, but also to foster and strengthen the collaborative relationship. The large drop-off in response from post-training to follow-up questionnaires warrants further investigation for the full impact of adapting PST to the camp setting. Although technical issues may have prevented some from submitting responses, we encourage future researchers to preemptively devise strategies to address the low follow-up response rate. Soliciting advice from the end-user may be helpful in obtaining meaningful change that is not researcher-directed.
The positive, yet preliminary, findings promote future implementation of the consultation model used in this study and suggest that the PST training can be implemented in group settings. We believe results suggest the effectiveness of a community-university partnership in implementing a brief training and using a problem solving framework might be successful in teaching summer camp staff skills for dealing with difficult camp situations.