How Do We Measure Absolute Quality?
Addthis
One problem with implementing the proposed "home-based" student assignment plans is that BPS is currently assigning schools to tiers using quartiles. This means the top 25% of schools are in Tier 1, the next 25% in Tier 2, etc. The problem is that even if schools improve significantly, there will be four tiers with approximately equal numbers of schools. Ian pointed this out in a comment on a previous post and it has been discussed by the EAC.
One of the advantages of the home-based plans is that they can be flexible over time. If, as we all hope, quality improves significantly, families will automatically be given fewer choices because they won't need to go as far from home to get to high quality schools. But if we always have the same number of schools in each tier, we'll just be moving school choices around. We'll never get to the point where we can reduce the number of schools offered or the distance traveled.
I think that the EAC would like to set some absolute measure of quality. For example, they could say that if at least 70% of students in a school scored advanced or proficient on the MCAS, that would be a high quality school. Of course the problem with this is that there's no clear number that we can all agree is "high quality." And to try to agree on three breakpoints that would allow us to group schools into four tiers seems impossible.
One solution would be to simply use the existing four tiers and take the breakpoints from the quartiles. BPS is creating a single number for each school based on two years of MCAS scores. Let's say that in order to be in tier 1, that number has to be 60 or higher, tier 2 ranges from 52 - 59, etc. We simply set the tiers to be driven by those numbers in the future. This is still fairly arbitrary, but there's really no way to avoid that if we're going to break schools into tiers based on some kind of quality score. What it does mean is that three years from now we'll be using the same absolute measurement of what a tier 1 school is that we're using now.
Most importantly, it would mean that if many schools improve we would have far more tier 1 and 2 schools and far fewer tier 3 and 4 schools. Under the home-based plans this would result in families having fewer choices covering smaller geographic areas. The families most affected would be those who would have a lot of schools over a wide area to choose from in the first years of the plan. Those families would also see a higher percentage of their choices move into the higher tiers.
Comments
quality
I like this idea. I particularly like that this gives BPS an extra incentive to improve quality. If more schools go into the higher tiers they will save money on travel expenses.
Still anchored to current boston levels
A good idea, and it has the virtue of continuity from the current levels.
But longer term, does it make sense to anchor the measure to the current levels of Boston schools? It may make sense to try to use a more meaningful measure of long term success - such as a statewide, rather than citywide, benchmark.
http://profiles.doe.mass.edu/mcas/performance_level.aspx?linkid=32&orgco…
Let's get to where people want to stay in Boston for the good schools.
Reasonable Benchmark
It does seem like setting a benchmark now, and then narrowing the choices as time goes on if schools improve would be an improvement. After all, the whole reason that people want the lotto so much is because of their local school not being good, so if their school is now meeting the benchmark then you should not need a plethora of choices anymore.
I do think though that you cannot really go to a state benchmark. Since we are not going to do anything about the lotto there is no reason to expect that the system is going to move from having 85% of the kids in it be classified as "High Needs" as the state does today. If you want to have the goal of BPS be to get its schools up to the "High Needs" average of the state, that seems like it would be a reasonable goal. If you set up benchmarks that are not correlated to the demographics of the system, or where it is today, they may not be reachable.
Actually, I never really noticed this before if you click around on the state site and look at other districts where "High Needs" kids represent the vast majority of students (Brockton, Fall River, Holyoke, etc.), Boston is actually not that bad. The "High Needs" state average is probably pulled up by the higher test scores in places where they represent a smaller concentration of the student population.
Not good enough - so a relative measure may be better
Maybe using the state levels wouldn't work to set a benchmark, but I'm still concerned that setting the definition of quality as being schools in the top 50% of Boston's current schools raises a number of issues. That is, the implicit statement of the analysis of the plans is that Boston families are getting a "good enough" school, and in fact are lucky, if they get a school in the top 50% of BPS' schools.
But as this shows, that's not good enough:
http://www.bostonpublicschools.org/files/2012-09-19_bps_quadrant_analysi…
If you read the legend on the vertical bars, the median school is 46% proficient or higher in ELA, and 40% proficient or higher in math. That's just not good enough. It's a little harder to guess the top 25%, maybe 57% for ELA and 52% for math. That's also not where it should be.
The answer isn't to accept that as a consequence of a higher-need population, and let yet another generation of children grow up without an adequate education. Instead we need to recognize the issue and try to figure out what we can do to address the higher needs for this generation of children, so they and we overcome those challenges.
While a more complete measure of quality might value other dimensions of quality better than MCAS alone - this issue is still going to be there. In fact, having looked at these facts - I think we'd be better off using the relative measure of quality, based on the quartiles, for some time to come.
Also, only a portion of the students in the schools are included in the MCAS score analysis. A little web searching hasn't given me the complete answer why. Some of it is just grade level - if only grades 3-8 are tested, then only 60% - 65% of children could possibly be included. Some of the SWD and ELL are apparently excluded too, according to the PDF above. And kids who don't take the test, of course. So this already excludes a lot of children.
It may need to be flexible
I agree that the bar is pretty low right now. One thing I would say is that there are two different things going on right now. One is that schools are being broken into four tiers for the purposes of the home-based plans. The other is that they're being broken into two tiers (quality vs. non-quality) for the purposes of analyzing the plans. I think a lot of people would disagree that the top 50% of schools are quality. This is why I urged the EAC to look at how the various plans affect access to tier 1 schools.
Maybe some flexibility is in order. Let's say a few schools improve in quality. We probably want to allow schools to move into higher tiers. But hopefully all the schools will improve. In this case, we might adjust the cut-offs to prevent most of the schools to be grouped into one or two tiers even though there would still be significant disparities.
Maybe it would make sense to set some benchmarks in advance. If x% of schools are tier 1, we're going to raise the benchmark for tier 1 to y. It makes things even more complicated, but we do want to make sure we're always raising the bar and that we're not satisfied with schools that are mediocre at best.
Expectations
Weather we should be doing a better job as a society educating the high need population is a larger topic, and I think we can all agree that we should be doing a better job with it.
However, the quality measures here are being used as justification for spending millions of dollars a year busing people around town and creating a generally stressful and crappy environment for everyone. So, I think it is important that we make sure that our measures a realistic for the populations that we have. After all, why have a vast busing program if you are just moving kids a long way to go to a similar school.
Actually, on that topic you got me interested in looking at a few numbers and I thought of something else…
Why are the schools not ranked according to their performance with the high need population rather then the whole school? If the point of this lotto is to give opportunity to high needs students that is that not the more important number?
I looked at a few schools around me and I think it is fair to ask how much of the ranking has to do with the population of the students and not with the outcomes (at least from an MCAS standpoint) for the high needs student. For instance the ENG/MATH CPI for one of my tier 1 schools (Manning) is 63/63 and for one of my tier 4 schools (Mendell) it is 59/63. Now at Manning high needs students are 49/70 rather then 76/78 at Mendell so the school average is a lot higher. However, from the standpoint of creating a good outcome for a high needs student they are equivalent from an MCAS prospective.
I guess this really goes back to weather your believe that the test scores of a school are more affected by the quality of the school vs. the population that they are working with. I think it is a mix, but to me the highest performing schools should probably be the ones doing the most with a difficult population rather then the ones that have more favorable MCAS demographics due to the walk zone and more importantly the selection bias created by the opportunity of people to opt out of the system.
This also gets back to something that I talked about a while ago. The idea of tiers (especially if there always have to be 1/4 in each tier) seems like a self-perpetuating cycle. I do not know Mendell very well, but it probably has almost 100% high needs kids because people who are currently getting it in the lotto and have the means to move out of town or go to private school do so. Since Manning is a tier 1 school people who get it in the lotto will go to it, thus is will continue to have better demographics and better test scores overall, even if as now it is not getting higher test scores for high needs kids.
Transportation is used only to get access to quality
Just to address one part of what you wrote - I think you've misunderstood something about the plans. (Maybe more proof of how poorly the plans have been communicated.)
"the quality measures here are being used as justification for spending millions of dollars a year busing people around town"
Well, sort of - it's to give kids and families who are willing to travel in order to get a better education a chance to do that.
Keep in mind that under the Home Based plans a family only has access to their closest 2 or 3 schools of each type. So you shouldn't have shouldn't have kids in Dorchester going to school in Allston. That is, it only uses transportation if it helps people get to quality.
And especially so for lower-quality schools. People will get their nearest schools for those choices or as "capacity" schools. No more long bus rides for kids to schools that are no better than their nearby schools.
Also, once the changes kick in, it's estimated that distances traveled will go down by 40%, even with the effects of some families choosing farther away schools to get higher quality. (Including that behavior is why the demand model was created.) Forty percent is markedly better than the current system, even if it takes time to realize the savings.
And again, it's money well spent to balance close to home and access to quality.
Responses
My point was that weather a school measures as quality or not has a lot to do with the demographics of who goes there. I understand that the point is to give people access to better schools that are further away, but at least in the instance of Manning vs Mendell looking at the MCAS it seems like "better" has a lot to do with who the average student is. They both have similar test scores for similar populations.
As far as distance, that is great that the transportation budget goes down, but things are still very random and far away. I am currently somewhat central located in the West zone, so my furthest schools in Home B are basically as far as the furthest schools now (Hale and Hanley).
But most of all, remember that positive effects of the lotto (balance) do not come for free. In my opinion it has a highly corrosive effect on the system because of uncertainly and opt out. If the effects on opportunity of the busing are low, ie where the difference is schools is due to demographics, then the return on your busing investment is also very low, at a very high cost.
Self-perpetuating cycle
I share a lot of the concerns you lay out in your last paragraph. The Mendell is actually an interesting case, because there was a year when a bunch of middle class parents got together and selected that school - you can read about it here: http://www.wbur.org/2010/07/19/jp-school-2. The kids in that story are now in 3rd grade, I believe, so you won't have seen their scores in the MCAS results yet. This article shows that it may be possible to break the cycle, in this case by a kind of informal parent compacting.
It's interesting to me that BPS and the EAC have decided not to pursue parent compacting, which was included in Councilor Connolly's plan. I'm somewhat curious to see what their reasoning was, although I haven't tried too hard to track it down. I'm assuming there are downsides to parent compacting that I'm simply not aware of, or that there isn't any demonstrable benefit. I'd also be curious to hear if there are any other proposals that are specifically designed to make underchosen schools more desirable to parents (beyond just promises to improve quality). It would be a huge step forward if there weren't schools that were populated primarily with kids whose parents didn't love the school but simply didn't have any better options.
Compacting
I wondered what happened to compacting as well. I had heard about the Mendell thing as well back in the day, but not much else recently. It did seem like a pretty small thing to let people organize to take unpopular seats if they want them. I would suspect that it was probably killed off by the BPS because they do not want to deal with the capacity management issues it would bring up.
However, I suspect that the real reason that people are not interested in it is because of the risk that it would work. I think for some of the interests at work here keeping middle class kids in the system is a bug rather then a feature of a better system. After all, having a school get taken over and then having higher test scores would create inequality that would need to be fixed.
Also, an importent part of the belief system of hard core lotto advocates is that middle class people will not help the system. Reference this video (http://vimeo.com/59188000) where Kim Janey has a nice chuckle at the idea that kids being able to play with friends from school is too middle class or suburban or whatever for Boston.
Kim Janey's remarks
I assume you're talking about the remarks that start at around 7:05. If so, my interpretation of those remarks is very different from yours. I don't think Ms. Janey is suggesting that "middle class people will not help the system" at all. I think she's just saying that middle class assumptions about what steps we should take to build a school community might not translate well to poorer neighborhoods. For middle class neighborhoods, moving to neighborhood schools may well be enough to push families over the edge from being dissatisfied with the feeling of community at their schools to being satisfied. In poorer neighborhoods, it might take more than that if it's achievable at all. I doubt that she thinks that playdates are "too middle class or suburban or whatever" for these neighborhoods, as if we should completely reject them as something to strive for. I think it's more likely that she's just saying that poorer families have other barriers that make it difficult for them to set up playdates or attend more school events and meetings, and that a measure that makes those aspects of the school community a reality in a middle class neighborhood might not cut it elsewhere. I didn't hear her make any kind of judgment on the value of including middle class families in the system or not.
Video
I'll have to watch the video again, its possible my impression of it was incorrect. I remember not being very impressed with the sort of out of hand dismissal of the idea that we should do with our system what basically everyplace else does, but I may have been reading too much into that remark.
Better Plan
Sorry to keep beating this drum, but since we are on the subject of demographics I have to pitch one more time a plan that I think would give everyone a much better deal then these lotto systems.
Let's call it Neighborhood Plus Choice.
Start with neighborhood schools, but don't just dumbly draw blobs on a map, keep the school areas contiguous but craft them so they draw in a demographic area of the city similar to city averages. I think under a system such as this one of three things would happen to each school.
I would guess about 25% of schools would end up with basically suburban demographics, and thus most likely suburban MCAS scores. The students in these schools were either going to lotto into good schools anyway or opt out and move to a place with better schools, so you are not removing them from other schools in the city, they were not going there anyway.
I think about 50% the schools would end up improved, but would still retain a more representative demographic mix. I think these schools would most likely be the ones that had a zone that included a mix of gentrified and low income areas or were just in stable working class areas of the city. These schools would benefit because for the ones with a mixed population they would have a more socio-economically diverse student body then most BPS schools today, and the infusion of middle class people would provide the ability to raise money and organize more parent involvement. This is the mix that you see today in a lot of the quality schools that are in the higher income areas of the city where walk zone people take some amount of the seats. I think a lot of the Mendells and other schools in a place like JP would end up like this because you could incorporate the gentrified areas of JP with sections of Mission Hill and Roxbury. The schools in working class areas of the city would benefit from stability and neighborhood cohesion, as well as less opting out by people going to church schools and moving to other towns.
So, for this 50% of schools you have improved the situation of the at least 50% of their population that is high need by improving demographics and giving them a guarantee of going to a socio-economically integrated school that they now need to be bused to West Roxbury to go to if they want. Or we are giving them a stable school for their neighborhood. Some of these schools may be better then others, but there should at least be no demographic excuse for poor MCAS scores.
Now there is the difficult part of the 25% of schools that would draw from challenging areas. 25% is about 14k kids based on the enrollment of the BPS. However, we have a lot of tools in our arsenal now that we could now exclusively focus on these students:
- 8k students currently go to charter schools and METCO, how about giving these kids priority for these programs?
- There are a couple of city wide schools, maybe we could add a couple more so there would be 3k or so seats in city wide schools that would have high support and special programming.
- We could have a internal version of METCO that would allow these kids to be bused to one of the 25% of BPS schools that will end up with suburban type demographics.
- What about indexing school funding to the per-capita income of their area? The other 75% of schools should not need these resources as much, focus them here.
- Now that other BPS schools are going to be easier to run, more organizational energy can be focused on these schools.
I think when you think about this if you were in the area of the 25% of schools with difficult demographics, would your rather be given priority to all of the options above or have a slight chance of getting one of the few quality seats in the city in a lotto?
So in summary.
- For the 25% of kids in suburban type areas, there is no big change in quality of school they will go to as they would have left anyway if they had not gotten what they want in the lotto.
- For the middle class kids who live in an area with a mixed socio-economic school, they get the chance to go to one school with the people they know and has a mixed population.
- For the high needs kids who live in a mixed area, they get the chance to go to a mixed school, which is going to have higher test scores then where they go now and more fund raising resources.
- For the working class areas they get a stable school system and their kids get to go to school with their neighbors.
- For the high needs kids in the cluster of schools that will not have a mixed population, they get priority access to all the programs that we currently have to provide opportunity for these kids. Because we do not need these programs as much for the rest of the population their odds of getting into METCO are higher, and BPS can focus resources (including all of the savings from busing) on those schools.
It seems to me that everyone's deal is significantly better in this system. Yes inequality will increase because there will be some suburban type schools in the city, but remember, all those kids were getting those seats or leaving anyway. You are not creating equality be driving wealthy kids to live in Needham, you are just moving them off the BPS books and into Needham's, what is the point?
The biggest thing is that every kid has a better chance then they have now, especially high needs kids. This should be priority number one, not keeping equality high if that means that quality is low.
I would love to have someone poke some holes in this plan so at least I did not have to always sit here thinking that we are creating a bad situation for everyone in the city, just because we don't want to risk some far flung area of the city having a school with high MCAS scores that not everyone can go to. I don't mean concerns about inequality, but concerns about how this is not an improvement for each and every child in the city.
Live here, go there will increase segregation
My objection to any plan that guarantees access to a certain school based on where you live is the effect it would have on both school and neighborhood segregation. The problem is that once you know that having a certain address gets you into a certain school, families with means can buy access to good schools by buying or renting in certain areas. That forces housing prices up in those areas and down in areas that are assigned to poor schools.
This starts a vicious cycle that affects both schools and neighborhoods. As poor neighborhoods get poorer, their schools get worse and their housing gets less desirable. The opposite happens in wealthier neighborhoods.
Segregation
Segregation under the lotto is already about as complete as it can be. The district is 85% minority right now and if you take out BLS then it is around 90% I believe. If it was not for the walk zone it would probably be close to 100% (notice the locations of schools with integrated populations).
Of course under a neighborhood plan there are going to be schools that are majority white and some that are majority minority, but there would be quite a few that would have significant populations of all groups. Keep in mind right now most schools are near 100% minority. Under a neighborhood plan there would be a lot more desegregated schools then there are now.
The lotto just moves the segregation lines to the borders of municipalities so they are not visible if you are only looking at BPS, but they are still there and more extreme.
As far as property values, basically the quality of the BPS adds nothing to property values anywhere is Boston at the current time. It may be true that in already high income and gentrified areas there would be property price increases due to there being a better school, but the people for whom this is a problem have already been priced out a long time ago. I guess the biggest risk might be to an area like Egleston square which might share a school with a higher income area, but I think for the people in these areas the guaranteed access to a better school would be a good tradeoff for maybe slightly speeding up gentrification over the next 20 years.
For areas that would not have good schools, not having good schools is already priced in now, it cannot get any worse. Also, having priority access to METCO and charters may actually be more valuable then the very small odds of lottoing into a good school that they have today as far as property values go.