The sport sector considers evaluation an essential organizational practice. Through evaluation, sport organizations can gain insights and solicit feedback about their programs. Evaluation can also inform decision-making, guide program improvements, and build stakeholders’ value and advocacy for programming (Holt et al., 2016; Shaikh et al., 2020).

Despite its value, evaluation also challenges many sport decision-makers, funders, partners, managers and coaches. The challenges such stakeholders face when engaging in evaluation stem from a lack of evaluation-focused training, human or financial resources, and time (Arnold et al., 2016; Mitchell & Berlan, 2016). The result is an increased demand for strategies that can help sport organizations enhance evaluation in cost-efficient and non-burdensome ways.

This blog summarizes the current state of evaluation practices within the sport sector. It also outlines 3 recommendations for sport organizations to consider when engaging in evaluation.

Navigating new waves

The COVID‑19 pandemic has presented sport organizations with an opportunity to ask and answer important questions, including how to adapt sport programming to meet stakeholders’ needs while adhering to public health regulations (Own the Podium, 2020; Patton, 2020). Given the pandemic, a research team at Brock University and the University of Ottawa sought the perspectives of 28 Canadian sport-sector stakeholders from the grassroots level to national sport organizations (NSOs). Their conversations revealed a priority shift in 1 of 2 ways about evaluation practices:

• Evaluation has taken a back seat. By prioritizing other operational aspects, including adapting programming, ensuring clients’ basic needs were met, and securing additional funding, some stakeholders were left feeling like they did not have the capacity to engage in evaluation.

• Evaluation has taken a front seat. Despite pandemic obstacles, the timing provided other stakeholders with an opportunity to ask important questions and reflect on program goals. Staff had time to focus on evaluation because they could take a breather from programming due to either drastic adaptations from in-person to virtual program delivery or a complete pause in programming.

Many stakeholders highlighted the benefit of embracing evaluation practices. For those who paused their programs, it allowed them to step back from the hustle of regular programming and dedicate time and energy to ask and answer relevant evaluation questions. In contrast, those who changed their programming spoke of the increased value of evaluation, including refining evaluation questions given the shift in program goals and/or delivery.

Regardless of how evaluation was prioritized, all stakeholders shared important reflections on evaluation-focused, capacity-building strategies to move sport organizations and the entire sector forward. These conversations led to the development of 3 recommendations that stakeholders can use to advance the field of evaluation. The recommendations can be applied both during and after the COVID‑19 pandemic.

Recommendation #1: Involve multiple stakeholders, it’s a team effort

Specific individuals within an organization are sometimes tasked with evaluation. However, applying a team approach that includes diverse stakeholders throughout the process can help to formalize evaluation, monitor activities, and ensure evaluation is aligned with organizational goals. An evaluation team should involve sport stakeholders at different levels, including individuals involved in on-the-ground program delivery, in managerial roles, and external to the organization.

On-the-ground stakeholders are those closest to the sport experience, such as coaches and athletes, who can offer lived perspectives of their experiences. Managerial stakeholders include program managers or administrators who can offer support by prioritizing evaluation within the organization and providing resources for evaluation. External stakeholders can include anyone invested in the future of the sport organization, such as community partners, funders, board members and industry experts. Intentionally seeking stakeholder diversity is important as it offers a variety of perspectives for informing evaluation foci and activities.

Recommendation #2: Evoke evaluative thinking

Simply knowing how to do evaluations is only 1 piece of the puzzle. The ability to think critically about a program can also be key for conducting high-quality evaluations. Evaluative thinking refers to consistently reflecting on how and why you engage in evaluation going forward (Buckley et al., 2015).

Without evaluative thinking, an evaluation may end up being superficial and without intention. Thus, by using evaluative thinking, the evaluation team members can keep themselves accountable for the questions they ask, the activities they engage in, and the way they interpret and communicate findings. In this way, evaluation becomes well-planned, activities are maintained, and the results support program improvement and continuity.

To promote evaluative thinking in an organization, stakeholders should meet with their evaluation team members and review their current evaluation practices. Some questions to ask to initiate discussions include (Shaikh et al., 2020):

• What is your purpose for evaluating your program(s)?
• What data do you collect?
• What is the quality of your data?

Recommendation #3: Capitalize on the power of storytelling

Once an organization has collected its data, storytelling can be used to communicate the findings as a narrative of the sport experience. Storytelling helps make sense of participants’ thoughts, experiences, and interactions with others, which helps participants to form their beliefs, identities and values (Wilson, 2018).

Storytelling can be especially empowering when told through the voices of stakeholders. For instance, coaches may share stories about how they delivered specific program practices, like teaching athletes about the value of team building. Sharing such stories could positively affect the athletes. For example, encouraging them to take initiative by mentoring younger players. An athlete could also share how their role as a captain of a soccer team helped them learn the value of being a leader in their community. By weaving data and statistics into these stories, the narrative can be placed within a program’s greater context.

Engaging stakeholders in communicating your data can offer a relatable and culturally relevant lens to ensure findings are palatable to diverse audiences. Recently, storytelling has emerged as a popular promotional vehicle and is even being used by funders as a way to communicate evaluation findings.

In sum, sport organizations encounter various successes and challenges with evaluation. By considering the 3 listed recommendations, sport stakeholders are provided with an alternative viewpoint for enhancing their evaluation practices. These recommendations are applicable during and after the pandemic.

---

This blog post and associated research are possible due to support from the Social Sciences and Humanities Research Council, the Government of Canada and Brock University.

---

About the Author(s)

Caroline Hummell is a doctoral student within the Faculty of Applied Health Sciences at Brock University, under the supervision of Corliss Bean, PhD. Caroline’s research interests involve positive youth development and the role of evaluation within sport and recreation.

Corliss Bean, PhD, is an assistant professor within the Department of Recreation and Leisure Studies at Brock University. She’s Co-director of the Centre for Healthy Youth Development through Sport and a member of YouthREX’s Provincial Academic Network. Her research involves working with youth-serving community organizations to develop, implement, and evaluate quality programming that fosters psychosocial development.

Maji Shaikh is a PhD Candidate in the School of Human Kinetics at the University of Ottawa. He focuses his research on positive youth development through recreation and sport. He has 6 years of experience in program evaluation with several youth-serving organizations in Ontario.

References

Arnold, M. E., Cater, M., & Braverman, M. T. (2016). Rethinking evaluation capacity in youth development programs. In K. M. Pozzoboni & B. Kirshner (Eds.), The changing landscape of youth work: Theory and practice for an evolving field (pp. 193-209). Information Age Publishing Inc.

Buckley, J., Archibald, T., Hargraves, M., & Trochim, W. M. (2015). Defining and Teaching Evaluative Thinking: Insights from Research on Critical Thinking. American Journal of Evaluation, 36(3), 375–388. https://doi.org/10.1177/1098214015581706

Holt, N. L., Deal, C., & Smyth, C. (2016). Future directions for positive youth development through sport. In N. L. Holt (Ed.), Positive youth development through sport (2nd ed., pp. 229–240). Routledge.

Mitchell, G. E., & Berlan, D. (2016). Evaluation and evaluative rigor in the nonprofit sector. Nonprofit Management and Leadership, 27(2), 237-250. https://doi.org/10.1002/nml.21236

Own the Podium. (2020). Canada’s COVID-19 Return to High Performance Sport Framework. Retrieved from: https://www.ownthepodium.org/en-CA/Resources/COVID-19-Resources

Patton, M. Q. (2020). Evaluation criteria for evaluating transformation: Implications for the coronavirus pandemic and the global climate emergency. American Journal of Evaluation. https://doi.org/10.1177/1098214020933689

Shaikh, M., Bean, C., & Forneris, T. (2020). Six recommendations for youth sport stakeholders when evaluating their programs. Journal of Sport Psychology in Action, 11(3), 165–182. https://doi.org/10.1080/21520704.2020.1746709

Wilson, M. (2017). Some thoughts on storytelling, science, and dealing with a post-truth world. Storytelling, Self, Society, 13(1), 120.

---

The information presented in SIRC blogs and SIRCuit articles is accurate and reliable as of the date of publication. Developments that occur after the date of publication may impact the current accuracy of the information presented in a previously published blog or article.