Table 2 Description of reach, effectiveness, adoption, implementation, and maintenance drivers of success, facilitators and barriers, and impact across district- and school-level personnel who administered and/or worked with the PE Works Audit and Feedback (PEAFC) tool in the New York City Department of Education (NYCDOE)
District-level administrators ( n =17); Facilitators (+), barriers (–), impact (•) | School-level administrators/PE teachers ( n =24); Facilitators (+), barriers (–), impact (•) |
|---|---|
REACH, Driver of Success: First ensure highest need schools receive necessary attention/support, then move on to lower-needs schools | |
 + Prioritized highest need schools during pilot cohort, which included more focused time and attention on highest needs schools  – Students not reached because PE was not a priority at that school, relative to other subjects  – Students not reached because principal had prior bad personal experience with PE  • PE Works audit and feedback reached students across schools, districts, and boroughs, regardless of school-level demographic characteristics prior identified as associated with poor PE quantity | + Principal is open and willing to accept the help to improve PE + School community, including teachers and parents, is bought-in – Students not reached because PE was not a priority at that school, relative to other subjects • PE audit and feedback increased reach of PE program within the school • Improved PE impacted the entire school community |
EFFECTIVENESS, Driver of Success 1: Ensure everyone is on the same page: a unified goal is necessary for PEAFC effectiveness. | |
 + Using audit and feedback as a tool to communicate about, and advocate for, PE  + To meet goal, need to first assess and identify needs and next support schools in meeting needs  • Audit and feedback helped increase administrators understanding of PE programming and its importance | + To meet goal, need to embed PE structures and processes in school • Audit and feedback helped increase administrators understanding of PE programming and its importance |
EFFECTIVENESS, Driver of Success 2: Meet schools where they are and provide tailored on-the-ground partnership and supports for improving PE | |
 + Supported, rather than penalized, for being non-compliant with PE law  + Laid out clear components for successful PE program  + Supports were structured, layered, and gradual (change takes time)  • Helped schools develop a personalized plan for establishing components of strong PE program | + Supported, rather than penalized, for being non-compliant with PE law + Laid out clear components for successful PE program – Without plan and supports, audit and feedback wouldn’t be as effective • Helped schools develop a personalized plan for establishing components of strong PE program |
EFFECTIVENESS, Driver of Success 3: Provide schools with a PE teacher(s) and the appropriate teacher(s) supports, for PEAFC to be as effective as possible. | |
 + Schools need help in identifying qualified teachers and then in supporting them  + Initial full funding of PE teacher was critical to get foot in the door  + Create or ensure a strong, navigable pipeline from PE credentialing programs to schools, so that there is a pool of eligible teachers for schools to choose from  • Helped schools who wanted and needed a PE teacher find someone who fit well within that school and provided adequate and appropriate training and support for new PE teachers | + Helping schools identify qualified teachers and then provide them resources + Providing trainings and continued professional development for PE teachers + Professional learning communities help support PE teachers establish effective programs – Tell schools to get a PE teacher but don’t help them in finding or training that teacher • Helped schools who wanted and needed a PE teacher find someone who fit well within that school • Provided adequate and appropriate trainings and support for new PE teachers |
ADOPTION, Driver of Success 1: PE needs to be a priority at the district level, which in turn drives priority at the school level | |
 + Audit and feedback needs to be prioritized and funded at the district level  + If PE is valued by district, it will increase priority in schools, making adoption more likely  + PE needs to be prioritized/valued across the school (not just by principal/PE teacher)  + There needs to be a clear vision for what PE should look like both across the district and within schools  • Program demonstrated increased district-level PE priority that was felt at school level  • Priority for PE was increased in most PE Works schools | + Seeing PE prioritized by superintendents and district-level personnel increases importance of PE within schools. + Still, many principals need support in making PE a priority at their school + Receiving funding for PE (via PE teacher salary, equipment, professional developments, etc.) increases priority for PE in schools + Initial funding for PE teacher critical to getting program started in schools • Program demonstrated increased priority for PE at the district level that was felt at the school level and priority for PE was increased in most PE Works schools |
ADOPTION, Driver of Success 2: Need appropriate, actionable indicators for auditing PE programs | |
 + Data should be collected as part of a conversation with principals, not just reading off survey questions  + Positive data (things school is doing well) should be highlighted as much as what needs to be worked on.  + Collect data for schools themselves, but also to indicate need across districts  – Too much data can lead to paralysis. (i.e. collecting data on nine components is too much); limit to only key components if personnel or resources are limited  • Actionable indicators helped schools clearly see their PE-related strengths and deficiencies, and forms strong foundation for technical assistance work to help address the deficiencies | + Collect data and report on PE indicators that can be improved + Don’t just identify the problem, help schools solve it – Indicating lack of space is frustrating because it’s not easily actionable for NYC schools • Actionable indicators helped schools clearly see their PE-related strengths and deficiencies, and forms strong foundation for technical assistance work to help address the deficiencies |
ADOPTION, Driver of Success 3: Need an efficient data entry system and automated process for producing feedback reports | |
 + Need to train district personnel on quality data collection and data entry  + Data entry needs to be user-friendly for non-techy district-personnel  + Manual work to ensure feedback/action plans are appropriate for each school  + Strong template helps, but personalization is often needed  – Limited capacity at the district-level to analyze data across district Streamlined data entry and reporting system enabled district to provide structured feedback to schools  • Too much data and not enough people at the district level to help analyze the data led to some inaction. | N/A |
ADOPTION, Driver of Success 4: Need appropriately qualified and passionate district-level personnel who are working collaboratively to work with schools | |
 + Need district personnel who know school administration  + Need district personnel who know PE pedagogy and teaching  + Need district personnel who can work with data  + Having a smaller caseload of schools, so district personnel can spend more time with each school and really get to know the school and its community  + Need coordination of/trainings for district-level teams so everyone is on the same page  • Teams of district personnel (one with an administrative background, one with a PE background) worked well together to support both principals and PE teachers through the audit and feedback process | + Need district personnel who know school administration, so they understand challenges of principal in overseeing the execution of a strong PE program + District personnel having prior in-school experience is very helpful + Need district personnel who know PE pedagogy to directly help the PE teacher • Teams of district personnel (one with an administrative background, one with a PE background) worked well together to support both principals and PE teachers through the audit and feedback process |
IMPLEMENTATION, Driver of Success 1: Need a personalized assessment of, and structured plan for, each school, which includes proactive coaching to improve PE | |
 + Use real, relevant data because principals value data on their school  + Plan to improve PE and supports need to be targeted to each individual school  + Plan needs to meet schools where they are and have realistic expectations  + Plan needs to be layered to evolve over time  + Coaching is proactive, as opposed to reactive  + School-level supports are available, personalized, and helpful  – Changing PE is a journey that takes time; rushing shows you don’t understand schools  • Schools received a structured, personalized plan to improve PE, which, when followed and supported, enabled them to significantly improve their PE programs  • Technical assistance and layered related supports led to improved PE quantity/quality | + Use real, relevant data because principals value data on their school + Plan to improve PE and supports need to be targeted to each individual school + Plan needs to meet schools where they are and have realistic expectations + Plan helps schools check progress and see gains + Coaching is proactive, as opposed to reactive + School-level supports are available, personalized, and helpful – Changing PE is a journey that takes time; rushing shows you don’t understand schools • Schools received a structured, personalized plan to improve PE, which, when followed and supported, enabled them to significantly improve their PE programs • Technical assistance and layered related supports led to improved PE quantity/quality |
IMPLEMENTATION, Driver of Success 2: Building strong district-school relationships is necessary for successful PEAFC implementation | |
 + Understanding schools’ unique complexities helps build trust  + Need to build trust with the principal  + Persistence and patience are key in building relationship  + Need to understand complex nature of schools  – Being judgmental or hounding a school repeatedly/being smothering  • District personnel built strong and lasting relationships with school administrators and PE teachers that helped ensure successful implementation  • Coaching and supports led to improved PE quantity/quality | + Need to build trust + District-level personnel needs to be available + Need balance between being available and not smothering + Follow-up with principal is key + Need to be flexible – Hounding a school repeatedly/ being smothering • District personnel built strong and lasting relationships with school administrators and PE teachers that helped ensure successful implementation |
MAINTENANCE, Driver of Success 1: District-level personnel should remain available to schools to continue to support PE past the life of the program | |
 + District personnel maintained relationships with schools and provided support throughout program and past life of program  – Funding was available only for one audit per school  • District personnel continued to support schools past the life of the program, which helped schools to maintain strong PE programs | + Intensive support was good at program start, not necessary once program was established + District personnel were available, if needed, to help support schools once program was complete • District personnel continued to support schools past the life of the program, which helped schools to maintain strong PE programs |
MAINTENANCE, Driver of Success 2: PEAFC should provide a long-term plan and structure for schools to successfully build and maintain a PE program | |
 + Improved PE, and its impact on the students and school community, motivated maintenance of a strong PE program  + PE teachers took on leadership roles within schools, which motivated maintenance  • Most schools developed and maintained a long-term vision for PE and the structure to facilitate the maintenance of a strong program, which lasted beyond the funded length of the intervention | + Improved PE, and its impact on school community, motivated maintenance + Student buy-in, including increased expectations for PE, motivated maintenance + Parent and family buy-in motivated maintenance + Integrating PE across departments helped with maintenance • Most schools developed and maintained a long-term vision for PE and the structure to facilitate the maintenance of a strong program, which lasted beyond the funded length of the intervention |