Measuring Quality


Email, Print, send to Twitter, send to Facebook, and more

As part of its efforts to devise a plan to improve the student assignment system, the External Advisory Committee created a definition of what constitutes a quality school. This is important as the EAC wants to ensure that any plan selected will provide children with equitable access to quality. Of course, there's no way to know whether access to quality is equitable if we don't know which schools are of high or low quality.

Furthermore, the EAC is now considering plans that would group schools or zones in a way that would provide equal access to quality. Again, it is not possible to do this if we can't measure quality in schools.

Here is the definition of quality developed by the EAC:

Acknowledging that quality varies for each individual, the EAC Defining Quality and Equitable Access Subcommittee - with its members’ expertise, community feedback, and BPS research on quality - has drafted the following preliminary definition of a quality school to include:

  • Academic excellence and student academic growth in all grades, across all subgroups of race, ethnicity, English Language Learners and students with disabilities
  • Principal effectiveness and teacher excellence with caring teachers and school staff
  • Parent engagement and a sense of community within and outside of the school
  • Effective community partnerships
  • Focus on the development of the whole child and the needs of all learners, though arts, music, athletics, and program and course offerings
  • Safe and positive school climate including social and emotional support
  • Adequate and appropriate facilities
  • As close to home as possible

So now comes the difficult part. How do we measure these factors? Academic excellence can be measured using MCAS scores. It's certainly not a perfect way to measure academics, but it's objective and it's available for all schools. After that, it gets more difficult. Teacher evaluations have not been done consistently. For safety, there's some crime data available, but not enough to compare schools.

Many of these factors could be measured with data from the School Climate Survey. This survey is filled out annually by teachers, parents, and students. The EAC considered using data from this survey, but they ran into a serious problem. Return rates for the survey vary considerably from school to school. This is especially true for the parent survey where some schools have a return rate over 50% and others have a 0% rate. Low rates essentially make the data unusable because it's impossible to know if the responses are representative of the entire school community.

Unfortunately, the school choice improvement process is going on without this information, and it's probably too late to do anything about that. But hopefully more data will be available in the future.

Massachusetts law and the new teachers' contract both call for annual teacher evaluations. So going forward, we should have a way to measure teacher effectiveness.

And BPS is hoping to increase the response rates for the school climate survey. One way to do this is simple, just tie some small amount of each school's funding to their response rate. This is the case with applications for lunch subsidies. Schools work hard to get those applications back, even from families that won't qualify because some of their funding is tied to the number returned.

Another way to get surveys back would be to give kids an incentive to get their parents to fill them out. Several years ago the Hurley School conducted a survey of parents. They bought several very cheap toys (probably from Oriental Trading) and kept them in the office. Any kid who brought in a completed survey got to choose one of the prizes. Other kids saw their friends with them and harassed their parents until the survey was completed. I can't remember what the final response was, but it was quite good.

For enrichment programs, BPS could develop a measure based on number of programs available, number of children served, and time spent on enrichment each week.

What are your ideas for measuring quality?

Skip Comments


Thanks for this post. It got me thinking about coming up with ideas for updating the BPS mission statement and quality improvements. After the Tuesday EAC meeting I expect I'll send it out to BPS and the EAC.

Here's a part of it:
School assignment should make it easier for families to get their child into a school that will be a good fit. If there are overlays this should make the process easier, not more difficult. Also a child with a 504 plan should not have to wonder if they need an IEP to get into a school that is a good fit for them, or if they would have better luck with the general lottery. As the new assignment process is working on creating equitable access to quality schools I think there needs to be some mechanism that puts into the choice process the family's beliefs about what quality means to them. For one family a partnership with Boston Ballet might be huge, or the school climate in another school might make it the only school that seems like a good fit for their child."

I'll keep working on it..

:) LJ

The quality measure is a big driver of the assignment process.

One of the powerful things about the proposed zone-less option is that it is more of a framework or a tool, than a final outcome. One might imagine they could adopt the tool and define it, subject to improving some of the components such as the quality measure. (This would never work when drawing lines on a map, with the original zone proposals - those are outcomes, not frameworks.)

Does it work if they adopt the framework, with a signed-in-blood commitment to improve the quality measure by a certain date - in 3-4 months or so? And also commit to review it again every year or two? Work backward from 2014 assignments and figure out how much time we have to improve the quality measure and finalize school groupings.

That might be the best (and maybe only realistic) option. And it still seems to allow them to meet the timeline for committing to a new, better assignment process.

Josh Weiss's picture

Josh Weiss's picture

I agree that they don't have to decide on the particular groupings in order to recommend the Grouped Schools Plan. I think there are a couple of potential issues, though. One is that it might be difficult to get people (both the EAC and the public) behind a plan where it's not possible to see who would have access to what schools.

The other issue is how long it might take them to get some of the new data they need. If it could be done within a few months it might be possible to use it for the 2014-15 school year. But I'm concerned that it could take more like a year.

The good news is that a plan like Grouped Schools is flexible. A better measure of quality could be phased in after the fact, but it would have to be done over several years to meet the goal of not changing families' school choices very frequently.

I agree that stopping this process until we have better quality measures is probably not realistic. But let's all keep the pressure on BPS to continue to improve the data that's available even after the EAC is done with their work.

Back to Top