Movement-based approaches to program evaluation with children
May 25, 2021
This blog is part of a series in collaboration with Brock University. Written by a student in the ‘Program Evaluation in Professional Practice’ course, this blog draws on a student’s first-hand experience conducting a program evaluation during a placement with the Bounce Back League. In this blog, Ashley Romano, a 4th year undergraduate student, offers practical tips to engage children in program evaluations, and make evaluation fun!
Designing activities that cater to the children’s skills and interest is a routine part of any children’s sport program. In the context of program evaluation, choosing data collection methods that work for kids is just as important.
Research has shown children are capable, accurate, and valuable communicators, yet many data collection methods come with considerable limitations related to effectively engaging and understanding children’s experiences (Driessnack & Furukawa, 2011). To overcome these limitations, creative approaches to data collection account for children’s skills and abilities while making evaluation engaging and fun.
Using an evaluation of a children’s sport program—the Bounce Back League—as an example, this blog explores creative approaches to data collection, including movement-based methods, and offers tips and tricks for implementing movement-based methods with children.
Limitations of traditional data collection methods
Traditional approaches to data collection, such as surveys or assessments, may not be suitable for children for several reasons. First, methods that rely on adults to observe and/or report on children’s progress or experiences (e.g., physical literacy assessments, progress reports) diminish what children have to offer. Children think differently than adults, but not less (Carter & Ford, 2013). Second, many self-report techniques, such as surveys, were developed for adults and adapted to be “child friendly” (Driessnack & Furukawa, 2011), but these adaptations do not always ensure sensitivity to the childhood experience. Often, these methods receive low response rates, have poor reliability, and are influenced by social desirability (i.e., wanting to give the “right” answer; Soland et al., 2019). In contrast, creative approaches to data collection can be developmentally tailored, provide valuable insight directly from children’s experiences, and save time by doubling as a program activity.
Creative approaches to data collection
The term “creative” fits nicely within the scope of methods for collecting data with children. Creative approaches incorporate developmentally appropriate strategies, including but not limited to, individual or group interviews, creative play, or thinking (Christian et al., 2010). Creative approaches resonate with the childhood experience and cater to children’s skills and interests (Carter & Ford, 2013). Movement-based methods are one example of a creative approach to data collection that is particularly relevant for evaluations of sport and physical programs. These methods tend to be low-cost, easy to implement, are typically well-received by children due to the active nature of involvement, and can help improve on-task behaviour and performance when engaging in these methods (Savina et al., 2016).
Movement-based methods can be understood as integrating physical activity into data collection and can take many forms, such as cooperative games or team sport in which children use the structure or rules of the game to provide feedback about a program or their experiences. For example, in Ship, Shore, Anchor, children run to the wall labelled “yes/agree” or “no/disagree” when questions are called out about the program. This method is explained in more detail below.
Movement-based methods can be used to capture how and why sport fosters development through children’s perspectives. Using movement is an engaging process that helps children ‘buy in’—and it is fun!
Movement-based methods in the Bounce Back League
The Bounce Back League (BBL) is a nationally-run trauma-sensitive sport program for children aged eight to 12 years old. BBL uses a proactive approach to build children’s skills, including resiliency, as many who attend BBL are at risk of experiencing trauma or adversity. In the BBL, program evaluations are crucial to understand children’s program experiences, and given its sport-focused nature, movement-based methods are a natural fit. While several movement-based methods have been used to inform BBL evaluation, Ship, Shore, Anchor is a notable example.
Ship, Shore, Anchor uses space and movement to elicit children’s thoughts and ideas about their program experience and can be run by program staff or an external evaluator. To prepare, staff label one wall “yes/agree” and the opposite wall “no/disagree.” Another space or object (e.g., a hula-hoop or mat) is labelled the anchor—a neutral space for children who are unsure or do not feel comfortable answering. A staff member then calls out “I-statements” that relate to children’s experiences (e.g., “I feel safe at BBL”) or outcomes (e.g., “I learned to not give up when things are difficult”). How children respond to the questions can be documented by a facilitator. At the end of the activity, staff can foster a reflective discussion in a debrief with the children that is aligned with the evaluation purpose. This discussion can include questions such as “What skills did you use today?” or “What did you learn about yourself through this activity?”. If feasible, the debrief can be audio-recorded or notes can be taken and analyzed as part of the evaluation data.
Tips and tricks for implementing movement-based methods
- Use activities that children know and love: It can be difficult to create a movement-based activity from scratch, so start with an activity that program participants already know and love. Making slight modifications for data collection will be easier knowing the activity is familiar and well-received.
- Demonstrate directions: Making modifications to well-known activities, even if minimal, can be confusing for children who are used to playing an activity a certain way. When explaining instructions, it can be beneficial for facilitators to physically or visually demonstrate changes to the structure or rules.
- Keep it simple: Children have short attention spans and benefit from clear instructions (Carter & Ford, 2013). Keeping language short and simple is important for both the instructions and evaluation questions. “I-statements” and signs/labels are easy ways to keep evaluation questions and instructions clear and simple.
- Check in: It is important to continuously check in with children to ensure they understand what is going on, what is being asked of them, and why. A check in could be as simple as seeking a thumbs up for “I get it,” thumbs down for “I don’t get it,” or a thumbs sideways for “I kind of get it, but not completely.”
About the Author(s)
Ashley Romano (she/her) is a 4th year undergraduate student in Recreation and Leisure Studies at Brock University, where she was involved in the Program Evaluation in Professional Practice course. Ashley was also a research assistant under Dr. Corliss Bean and YouthREX where she developed an evaluation plan for the Bounce Back League.
Dr. Corliss Bean is an Assistant Professor within the Recreation & Leisure Studies Department at Brock University. She is the co-director of the Centre for Healthy Youth Development through Sport and a member of YouthREX’s Provincial Academic Network. Her research involves working with youth-serving community organizations to develop, implement, and evaluate quality programming to foster psychosocial development.
References
Carter, B. & Ford, K. (2013). Researching children’s health experiences: The place for participatory, child-centered, arts-based approaches. Research in Health & Nursing, 36, 95-107. doi:10.1002/nur.21517
Christian, B.J., Pearce, P.F., Robertson, A.J., & Rotherwell, E. (2010). It’s a small, small world: data collection strategies for research with children and adolescents. Journal of Pediatric Nursing, 25, 202-214.
Driessnack, M., & Furukawa, R. (2011). Arts-based data collection techniques used in child research. Journal for Specialists in Pediatric Nursing, 17. 3-9. doi:10.1111/j.1744-6155.2011.00304.x
Savina, E., Garrity, K., Kenny, P., & Doerr, C. (2016). The benefits of movement for youth: a whole child approach. Contemporary School Psychology, 20. 282-292. doi:10.1007/s40688-016-0084-z
Soland, J., Zamarro, G., Cheng, A., & Hitt, C. (2019). Identifying naturally occurring direct assessments of social-emotional competencies: The promise and limitations of survey and assessment disengagement metadata. Educational Researcher, 48(7), 466-478. doi:10.3102/0013189X19861356
The information presented in SIRC blogs and SIRCuit articles is accurate and reliable as of the date of publication. Developments that occur after the date of publication may impact the current accuracy of the information presented in a previously published blog or article.