Practice-Based Inquiry (PBI) is a school visit process that provides interested constituencies insight into how teaching and learning happen at a school.  PBI is based on the English tradition of school inspection, by which all UK schools receiving public funds are inspected every four years.  The principles of PBI were also the process by which the State of Rhode Island conducted the “state-wide, school accountability system designed to support the implementation of the Board of Regents’ strategic agenda” from 1997 until 2009 (SALT Blueprint).  The process came to Chicago for the first time in 1998 through the Chicago Schools Alliance at Business and Professional People for the Public Interest (BPI).   ACT Charter School requested PBI visits as a part of their charter renewals during its years of operation . Since that time, the Network for College Success (NCS) at the University of Chicago has become a PBI center, sponsoring visits as a way for its member CPS high schools to better understand learning and teaching at the school.  After a visit, NCS, rather than asserting control over the school, instead offers assistance (Wilson, 1996) towards its improvement plans based on the resulting PBI report.  (If you are familiar with PBI, jump to implications for connected learning and equity.)

The Protocol

Each 5-day PBI visit follows a rigorous protocol that lays out how the visit is conducted. Protocol guidelines determine:

• The size and composition of the visit team
• The conduct of team members while in the school
• Focus questions that guide and limit the scope of the team’s inquiry
• The nature of the evidence the team collects
• The structure of the team’s written report
• The process for how the report is written
• Tests for each conclusion in the report
• Tests of the report itself

Team Composition

Each team in purposefully composed for the school requesting the visit and requires a complex balancing of criteria.  The protocol sets requirements regarding the number of administrators, special education teachers, grade-level and subject area teachers.  Further, at least 50% of the team must be practicing teachers.  On top of these requirement, NCS seeks to balance teams in regard to gender and race to match that of the student population of the visited school. Finally, each team is composed of educators currently working in or with CPS schools. After each visit, that team is disbanded.  Thus, each visit is conducted with a completely new, unique team, thereby making it impossible to develop a “professionalized corps of inspectors”.   While assembling a team primarily from working, school-based educators arguably makes for the most daunting logistic of any visit, nearly all team members who experience a PBI visit report that it is one of, if not the most, powerful professional learning experiences they have had.

The Focus Questions

Three nested questions provide the focus for each visit: 1) How well do students at [school name] learn?  2) How well do teachers at [school name] teach for learning?  3) How well does the administration at [school name] support teaching and learning?  The nested structure of the questions with student learning at the center of the inquiry anticipates related conclusions in teaching and support.  While these questions seem generic, that is, in fact, what gives the visit process and resulting report their credibility.  Teams do not go into schools with rubrics and checklists.  In such a context, the use of such tools are far too random and subjective and thus suggest that if teams don’t find what is on the rubric or checklist that the school must not be “good” or “effective”.  Instead, team members observe the school through the frames of these questions and collect the evidence that provides answers, applying their professional judgement to that evidence to determine whether and how well students learn, teachers teach, and administrators support teaching and learning.

5 Days in the School

Team members spend their first day (Monday) shadowing a student through their full schedule to get the students’ perspective and experience being a learner at the school.  Throughout the day team members observe their students’ learning in each class, talk with them as well as other students they encounter.  Everything they observe throughout the day must focus on evidence of student learning and to draw on their professional judgement to determine whether and what their student learned that day.  On the second day (Tuesday) team members fan out to observe every teacher in the building and collect evidence to answer how well teachers teach.  In addition to observations, team members converse with teachers during their preps or lunch periods to get the teachers’ perspectives about what it is like teaching at the school as well as their thoughts about their students and administrative support for teaching.   The team spends the third day (Wednesday) meeting various school constituencies.  By meeting with the principal, the assistant principals, counsellors, parents, students, LSC members, community partners, security, lunch staff, the team determines how well the school supports teaching and learning.  Each meeting lasts at least an hour.  As a result of the Wednesday meetings, team members begin to see connections between and among the evidence they collected during their first two days in the school.  The first half of the fourth day (Thursday) is for conducting “special inquiries”.  By this time, team members have a high level of certainty about what they want to say in the report.  So the time in the school is about collecting the final evidence and determine whether it strongly affirms or challenges the existing evidence and the conclusions they believe are the most important for the school to hear about learning, teaching, and support. All inquires and evidence collection must be completed before noon on Thursday, after which the team is sequestered in the team room for the remainder of the visit to deliberate and write the PBI report.

When the first draft of the report is completed by the end of the day on Friday, the team has spent a significant amount of time in the school.  On average, a PBI team is convened at a school approximately 60 hours.  A team of 12 will spend nearly 125 person hours directly observing the classrooms at a given school with a goal of visiting every teacher’s classroom at least once. Teams average around 60 person hours in one-on-one or small group conversations with a school’s students, faculty, administrators, parents, and staff.  They will also spend several hours observing school meetings if school and team schedules allow it. Teams convene meetings that total around 25 to 30 hours to ask questions of students, teachers, the leadership team, the principal, and senior school administrators.  In addition to the time they spend interacting with the school, a full visit team meets privately in the team room for anywhere from 30-40 hours spanning the five days of the visit. During most of this time, teams are in intense discussions to thoroughly consider the evidence they have collected and to build the conclusions, commendations and recommendations presented in the report.

After each day’s evidence collection, team members return to the team room and review their notes from the day.  Every team member shares the evidence they collected to answer the focus question for that day.  They share what they believe is the most important information first.  They finish by asking a question about the school they think they need to attend to during the next day based on what the present day’s evidence seems to suggest is important to learn.  Finally, the team spends some time considering what they heard in the debriefing of the day, what patterns they heard, and common themes.  These questions and common themes are captured in team notes and drive the inquiry forward.

Professional Judgement, Deliberated Consensus, & Dynamic Evidence

Several elements about the PBI protocol make it a compelling process, not to mention unusual, in this age of data-driven decision-making.  What is unique is the emphasis placed on the roles of professional judgement, deliberated consensus and the dynamic nature of evidence.  A PBI visit is different from the kinds of “data dives” so many schools do these days.  The differences are primarily a result of the process the team follows and the evidence it collects, both of which take a significant amount of time relative to spending a couple of hours in a faculty meeting looking primarily at charts, graphs, and numbers and trying to divine from them any specifics about learning and teaching.

During the 3.5 days the team is in the school collecting evidence, each team member draws on their professional judgement to determine which pieces of evidence are the most important for the team to hear in light of the previously collected evidence and team discussions about it. They consider if and how that evidence answers the questions raised from what was shared the day before.  Each day, the questions team members ask and the themes they identify in the shared evidence pushes the team farther and provides direction for the next day’s observations.  While the team is open to all evidence they encounter, what they have collected also suggest the kinds of evidence they look for as they move forward.

By about the third day of the visit, the team starts to identify themes they believe are the most important for the school to hear.  These themes become candidates for conclusions in the report.  Thus, what the team believes is the most important findings for the school to hear evolves over time:  The evidence they collect each day and the questions that fuel further inquiry are informed by their professional judgements — both individual and corporate — which in turn are informed by each day’s deliberations. The evidence becomes dynamic as the team deliberates about the meaning of the evidence they find.   What the team believes is the most important conclusions to write about learning, teaching, and support on Monday are usually not the same by Thursday.

The PBI Report

Each PBI visit results in a report that is written by and tested for full team consensus.  The report is the collective voice of the team.  Each report includes the following sections:

• Introduction
• Profile of the School
• Portrait of the School at the Time of the Visit
• Findings on Student Learning
• Findings on Teaching for Learning
• Findings on School Support for Learning and Teaching
• Final Advice
• Catalpa Endorsement
• Report Appendix

Of these sections, each visit team writes the Profile, Portrait, Findings, and Final Advice.  The Profile is drafted by the team chair before the visit begins.  It included the history of the school and its community, drawn from the chair’s own research, public documents, and formal interviews with the school principal.  It is shared with the principal for any factual corrections.  The team also has an opportunity to make any revisions they feel are necessary.  The Portrait, while one of the first parts of the report is one of the last parts written after most of the conclusion writing is complete.  It serves as an overview, or a snapshot of the state of the school at the time of the visit. Conclusions the team writes based on the three focus questions constitute each of the Findings sections.  Teams write 3 to 7 conclusions each for Student Learning, Teaching for Learning and School Support for Learning and Teaching. Each conclusion is coded from a list of evidence sources and must include a minimum of two different sources.  In addition to the conclusions for the latter two Findings, teams may include Commendations for areas where teachers and the school excel.  They also write brief Recommendations based on the conclusions in the visit report.  Final Advice is just that: The last chance for the team to speak to the school community, rally them to the work ahead, and give the school a sense of what the team believes, only in the broadest sense, what the school’s next steps should be.

After the team adjourns on Friday night, the chair works with both a PBI master chair and copy editor to make sure the report is soundly written grammatically, stylistically, and effectively communicates the team’s intent.  Then, within two weeks of the end of the visit, the chair and as many team members as possible, return to the school and read a working draft of the report to the entire faculty and staff.

Endorsement occurs after a Catalpa representative has a post-visit conversations with the team chair and the school principal to confirm procedures to ensure the legitimacy of the report were followed. [Link to Catalpa Procedures page]  With the attachment of the written Endorsement to the report, the final draft is finalized, copyrighted, and becomes the property of the school.  Copies are sent to the school and NCS with a third retained by Catalpa.

What PBI Suggests About Learning Technologies Use in CPS High Schools

The details of what can be shared here are limited by confidentiality agreements each team member signs in order to serve on a PBI team.  Therefore, the nature of the evidence shared here is generalized.

Nothing in the PBI visit protocol explicitly requires teams to consider a school’s use of information and communication technologies (ICT).  Nor should it since conclusions about ICT are easily validated within the bounds of the three focus questions.  Indeed, in the past few years some teams have decided to include technology use as evidence examples of broader conclusions about learning and teaching.  On one level, this is encouraging, especially since nearly all schools at the time of their visit demonstrated typical ICT use. That is teaching characterized by an overwhelming use of traditional, teacher-centered, textbook-based instructional methods even though “contemporary educational programs recommend the integration of ICT’s in teaching and learning” (Barak, 2014).  Only one team of those on which I have been a part has written entire conclusions specifically about technology use in the school that one would recognize as informed by ISTE standards [link].  Even in this instance, the conclusions specifically about ICT use made it into the report only because a single team member with deep knowledge and current beliefs about the role ICT need now play — made a very strong case and successfully swayed the team.  Among his arguments was that conclusions about ICT use had to be a priority since the school being visited had a curricular focus on technology.  Thus the faculty and staff needed to hear what the team thought about its use of technology for learning, teaching and support.  Given that the PBI protocol requires true consensus, eery team member had to agree both that the conclusion was important enough to write and that they had the evidence to write it. Nevertheless, this teacher was selected specifically to reflect the programming of a technology-focused school.  To have such  highly knowledgeable, ISTE-steeped teacher on visit teams for schools that do not have a technology focus is not the norm.

This raises an issue about intentional composition of PBI teams.   Excepting the example above, conclusions in previous reports that even mentioned technology treated it in their reports as so many teachers do in the classroom — as a mere add-on.  Yet this may not be because previous teams didn’t deem ICT use important per se, but because to date, teams have not had the current and requisite knowledge or beliefs about pan-curricular, ICT-based teaching and learning sufficient enough to make such conclusions a regular part of visit reports even if such conclusions were warranted.  Song et al. in their study of the effects of prior knowledge, self-regulation and motivation on learning performance reference the work of Baldwin et al, noting “the close relationship between prior knowledge and interest from the viewpoint of adult education….  They state that ‘as people get older and increasingly specialized, interest and knowledge may come to correspond closely” (2016).  Team members with more prior knowledge and interest in the uses of ICT for teaching and learning across the curriculum would “set larger and more consistent goal structures” and “behave in a more goal-oriented way” (2016).  The implications of intentionally selecting educators with such specialist interests to serve as PBI team members would have significant implications for team inquiries into ICT-based teaching and learning.  Not the least of which is raising ICT use to the level of stand-alone conclusions in their own right, thereby putting technology skills and knowledge on a par with students’ abilities to read, write, and think critically.  Such an elevation is necessary for any school that intends to be a 21st Century institution or one that positively impacts educational equity.  Then, to round out the nested focus questions then, the same must also be said about technology conclusions about teachers’ use of ICT to facilitate learning and administrators’ expectations, messaging, and professional learning supports.

PBI, Connected Learning, and Equity

Why does NCS exert so much energy to sponsor, support, and in some instances, cajole their member schools into hosting PBI visits?  Because PBI is a unique and powerful reflective opportunity to support continuous school improvement. As such, the best teams will uncover issues of equity in their learning about the visit school and raise it to the level of a conclusion in a report.  It is worth emphasizing that all NCS schools are CPS schools. As such, NCS member schools serve mostly urban students of color, many of whom are economically disadvantaged. NCS is an equity organization that works from the understanding that education is a social justice issue, where the future success of our society’s most vulnerable citizens is directly linked to quality education.  Thus, to host a PBI visit has potential benefits for school equity when the school uses the resulting report as a basis for setting priorities and improvement goals that positively impact educational outcomes for the public school students of Chicago.

As I wrote in parts one and two of this blog post, when schools make learning connected learning, they positively impact equity.   The correlations between socio-economic class, race, and teacher beliefs, and the kinds of opportunities students have to experience connected learning through ICT-based instruction have implications for how NCS selects educators for PBI visits.  In order to impact matters of equity linked to connected learning and ICT-based learning, PBI teams should be more intentionally comprised to include computing teachers (as opposed to computer teachers) [anchor tag to point in previous post] .  As noted in the example above, when teams have members who can employ their technological and pedagogical professional judgements to evidence collection and team deliberated consensus, they will include more specific conclusions about technology use in PBI reports.

PBI has been been a mechanism for schools to have discussions about matters of equity all along.  Whether it is about which students in a school take full advantage of available learning opportunities and programs or do not, how the curriculum does or does not prepare students for their post-secondary future, how faculty and staff beliefs expand or curtails students’ access to quality learning, or how support structures make access to programs available to all members of the learning community or privilege certain parts  over others.  ICT-based instruction plays a significant role in students’ development of the growth mindsets necessary for the lifelong learning, just-in-time learning, creative abilities, and collaborative abilities required to thrive in a 21st Century post-secondary world.  PBI visits and reports, therefore, need to be able to raise such issues for a school as a part of their post-visit discussions and improvement plans regarding student, teacher, and school uses of technology.

Team Composition for the 21st Century

Understanding ICT-based learning and instruction as an equity issue means it must be an explicit part of school transformation efforts.  Inasmuch as PBI visits are a part of such transformation efforts, then it is also vital to keep elements of connected learning with ICT-based, student-centered learning and teaching on the front burner of future PBI visits.  Intentional team composition is a means towards this end.  It calls for finding educators who understand the critical role learning with technology now plays.  “A range of other scholars have argued that knowledge about technology cannot be treated as context-free, and that good teaching requires an understanding of how technology relates to the pedagogy and content”  (What is TPACK?, 2012).   Where “good [inquiry] requires specific knowledge about the teaching of those disciplines….  [S]ubject specialization helps [team members] make judgements about the quality of teaching” (Wilson, 1996).  Since most schools do not effectively use ICT for teaching and learning when it is widely acknowledged how vital it is to do so, PBI practice would be strengthened and visit schools better served if the criteria for purposeful PBI team construction included educators with deep TPACK knowledge and positive belief sets about ICT use who can understand contemporary classrooms through a sort of “TPACK lens” .  As ICT-knowledgeable team members collect and interpret evidence about ICT-based learning and instruction and cast their professional judgements about the evidence, they would bring their technological knowledge as well as their pedagogical and content knowledge to the team deliberations and report conclusions.

Similar understandings and belief sets in PBI chairs would also strengthen visit practice.  Knowledge about ICT and connected learning would allow chairs to raise appropriate questions for the team, guide collection of useful evidence, and effectively facilitate team discussions such that visit reports include findings and conclusions about how well students learn with, teachers teach with, and administrators support uses of digital, networked technologies.

PBI As Mechanism For Changing Teacher Beliefs

PBI visits are powerful because they are conducted by professional peers.  PBI teams are not, as mentioned above, a professionalized corps of outsiders coming in to pass judgement on a school community’s effectiveness and to deliver a “gotcha!”.  PBI teams are working educators, fellow teachers and administrators from Chicago Public Schools.  They are allies working in support of common efforts to improve teaching and learning for the children of Chicago.  As such, the conclusions teams choose to write and the words they choose to write them means reports have a sometimes uncomfortably unavoidable credibility to the faculty and staff of host schools.  By including conclusions about how well the learning community is or is not using technology can set the stage for changing teacher beliefs about their own abilities and those of their students.  It is yet another opportunity for “perceived social influence from referent others [to have] a significant positive influence on individual beliefs about the usefulness of technology”  (Gu et al, 2011).  PBI team members — that is CPS colleagues — and their collective voice in the body of a report are the “referent others” who exert “social influence positively and significantly affects IT utilization” (2011).

Conclusion

One of the mottos that guides PBI teams is “Know what you see.  Don’t see what you know.”  It reminds team members that when examining evidence and deliberating in the team room, they cannot be led by what they do at their school or by whether or not a particular program or framework “should” be deployed as it is at the visit school.  It reinforces that they must bring their professional judgement to bear only on what they see and hear during the week of the visit and what that tells the team about teaching and learning at the visit school.  Yet it is fascinating how new learning changes what one notices.  How in light of that learning that one discerns things that were never noticed before.  Two years ago, I did not know about ISTE with its Essential Conditions  and technology standards for educators, students, parents, and coaches .  Or about the Connected Learning Research Network with connected learning as a social constructivist path toward redefining school for the 21st Century and a digitally networked era.  Or about the Partnership for 21st Century Learning and the links between technology, lifelong learning, and citizenship.  These are just a few of the organizations, concepts, or frameworks that in this digital, networked era so clearly connect the social justice and equity agendas of schools to the use of technology generally and technology-infused learning throughout the curriculum specifically.  Those connections have particular implications in schools that seek to serve poor children of color.

For years as an educator, a PBI team member, and as a PBI visit chair, I regarded technology use during PBI visits no differently than the educators I was leading on a team.  I still cannot lead PBI visits with a particular agenda.  Yet at a time when digital technology and media comprise an entirely new literacy that will determine the kinds of work students will be able to do as adults, it is vital that schools understand how well they are educating their students in its use.  With the considerable flexibility the PBI protocol allows, the times warrant more conscious and conscientious inclusion of knowledgeable and experienced computing educators on PBI teams and the ICT-learning-related conclusions for visit schools that will result.

HM Senior Chief Inspector Bill Maxwell talks about OFSTED school inspection.

See the first two parts of this series:

Navigating Teacher Beliefs, Connected Learning, and Practice-Based Inquiry for Equitable ICT Integration (Part 1)

Navigating Teacher Beliefs, Connected Learning, and Practice-Based Inquiry for Equitable ICT Integration (Part 2)