EAC Meeting Review - 1/14/2013
Addthis
The meeting started with a presentation by Dr. Tayfun Sönmez, an economics professor from BC on the effects of walk zone priority. A study by professors at BC and MIT and others shows that walk zone only results in slightly more students ending up in walk zone schools than an algorithm that doesn't give any walk zone priority. Their research was on K1 and K2 seats assigned in the first assignment round from 2009 through 2012. For K2, they find that 48.5% of kids assigned to schools live in the walk zone. If BPS eliminated walk zone priority, 47.3% would come from the walk zone. This is a suprisingly small difference. K1 also shows a very small difference.
This phenomenon is due to the order in which walk zone and open seats are assigned. After assigning siblings, BPS first assigns the rest of the available walk zone only seats to the students in the walk zone with the best lottery numbers. They then assign the rest of the seats to the remaining students with the best lottery numbers regardless of where they live. Because the walk zone students with the best lottery numbers have already been assigned, few of them have good enough numbers to get one of the open seats. Also, with the current large zones, there will often be more non-walk zone students than walk zone students applying for seats in a given school. This makes it particularly unlikely that walk zone students will get any of the open seats.
This is an important point because it means that any plan that increases the percentage of walk zone students competing for seats at schools will likely increase the percentage of walk zone kids attending schools regardless of any walk zone priority. I spoke to Dr. Sönmez after the meeting and he confirmed this. They are looking at how this would play out in various proposed plans.
Dr. Sönmez proposed three possible changes to the way walk zone priority is currently implemented. One would be to assign all the open seats first, then the 50% allocated to walk zone students. This would significantly increase the percentage of walk zone students getting seats because kids with walk zone preference would better compete for the open seats. The second option would be to just alternate assignments between walk zone and open seats. This would result in a little over 50% of seats going to walk zone kids. The third method would be to assign 25% of seats to walk zone kids, then 50% to all kids, then the last 25% to walk zone kids again. This would result in something very close to 50/50 allocation of seats under the current plan.
BPS said that unless the EAC makes a specific recommendation in this area, they will go with the 25-50-25 approach because it gets closest to a 50/50 allocation. I'm disappointed that BPS is once again putting the cart before the horse on walk zone priority. We don't know yet how walk zone priority will impact the final plan. In fact, I think it's likely that more than 50% of seats will go to walk zone kids in a new plan, even without any walk zone priority.
I have another concern about this presentation, which is that the numbers just don't add up for me. I did a quick check of the data provided by BPS for school year 2011-2012. According to this data set, around 55% of elementary grade students (K1-5) are walk zone kids. Also, BPS data shows that in round 1, 65% of parents put a walk zone school first. This should mean that in most schools at least 50% of students are from the walk zone. I will research this further and post more on this issue soon.
The committee discussed their meeting schedule over the next few weeks. The only firm date is next Wednesday, 1/23 from 6:00 to 8:30 at Suffolk, 73 Tremont St., when MIT will present their demand model analysis of the proposed plans. I suspect that they will also propose some modifications to what's been proposed based on what they've learned. They will then meet the first and/or second week of February for further discussion and a final vote. Apparently 2/14 has been ruled out as a meeting date, but nothing has been settled. There may also be meetings on 1/28 and/or 1/30. The committee does still need to finish their recommendation memo and these dates could be used for that purpose. There is a plan for a single meeting to get community feedback on the proposed plans between the 1/23 meeting and the EAC's final vote. The fact that there would be only one community meeting received strong negative reaction from members of the public in the audience. It really doesn't seem sufficient especially since most people have never seen these plans and there will likely be significant changes to them on or after 1/23.
The committee spent a little time at the end of the meeting to continue their discussion on their memo of recommendations to the school committee. There was some discussion of the BPS quadrant analysis that they're currently using to evaluate school quality. Hardin Coleman suggested using a different method to represent this data. Other committee members asked him to submit his proposal in writing at the next meeting. There was also some discussion on a set of recommendations by EAC member Kelly Bates.
All materials from this meeting have now been posted on bostonschoolchoice.com.
Comments
Looking at averages
Thanks very much for the summary.
If past experience is any indication, it would take a most of a meeting just to explain the demand model, to understand it in a meaningful way; and then at least couple of meetings to go through the evaluation of a few proposed plans.
I also wonder about the practice of looking at a single, hypothetical average school. It would be far more definitive to show actual or projected results for all the specific schools where the issue matters. I'd hope that by the time they get to actual recommendations, they can use more specific analysis for the full set of high-demand schools.
They need way more time
I agree, Bruce. There's really still a lot of work they have to do. In addition to everything else, they still haven't covered the overlays for ELL and SPED students or the whole Middle School Pathway proposal which is a huge topic that has received almost no coverage.
Assignment algorithm presentation
The assignment algorithm presentation really was interesting, not least because I realized that I didn't fully understood the way that sibling and walk zone priorities work, despite having had the algorithm explained to me in great detail during the West Zone Parents Group meetings a few years ago. I had been told that 50% of the seats were "priority" seats, and the other 50% were assigned without giving any priorities. So I was under the impression that a school could theoretically fill up all of its "priority" seats with siblings, and be left with no walk zone priority seats at all. I see now that walk zone siblings get priority in the walk zone seats, and all siblings get priority in the remaining seats. This year I'm entering my younger daughter in the K1 lottery, hoping to land her a seat at my older daughter's non-walk zone school, so this is timely information for me! As I understand it, my daughter's chances of landing a K1 seat are better than I had realized: she still has priority for half the seats, but walk zone siblings will take up slots that she doesn't have priority for rather than competing with her for the seats where she does have priority.
I'm not sure if there's a larger point to be made here or not. I consider myself well-informed and I'm not intimidated by relatively complex algorithms, and yet I still didn't understand the algorithm fully. It seems like some of the changes under consideration (such as the 25-50-25 plan) may make the algorithm harder for parents to understand rather than easier. I expect that parents want the process to be easier to understand, from the standpoint of both complexity and transparency. This is a secondary goal of the EAC at best - it's not explicitly part of their definition of equitable access - and I wonder how well they're addressing it.
The larger point
There is a larger point, as an implementation issue. While the BPS does try to communicate, they need to do a better job. Eg, look at Josh's post about the new K2 assignment option. It doesn't just describe what the policy is; it talks about what it means, for various family needs. BPS may be reluctant to do that, but they need to get over that. That's what people need, don't make parents guess what something means. Explain, give examples.
Not only does the complexity present a challenge, but as you probably know there is a lot of old info and folklore circulating. (While the parents at the playground are overall a great source of info, sometimes they pass on info that's just wrong.)
By the way, remember when the EAC was going to try to identify implementation issues? It seems sort of quaint at this point. Still, the EAC is doing a good job in my opinion. Maybe they need a little more time to finish the job.
Two Concerns
Having been at the presentation last night, I had two major concerns, one of which Bruce describes above:
#1. I also wish Professor Sönmez had described how the algorithm plays out in the more extreme circumstances, like for example in high-demand schools. By using an average, it's not clear what happens in these cases, though intuition suggests and anecdotal experience shows that some schools have walk zone percentages in the high 80's and 90's. As one of the professors from MIT suggested, BPS should release information about the numbers of children from the walk zones that attend specific schools, especially schools that are high-performing, or high demand.
#2. I was also really concerned that the time to digest this information was so limited. The information presented was really challenging and thank goodness a couple of EAC members said they needed more time. Without time to consider the analysis, many could easily have been swayed by the language of the presentation, words like "balanced" and "compromise" to describe the alternative methods laid out above. These are words that could easily have been mistaken for "fair," though it's not clear that this is actually the case. They are only "balanced" and a "compromise" if your intention is to get move towards more seats for walk zone students. If your intention is to have schools be 50% walk zone and 50% non-walk zone (which may not actually be the case), then either of these methods is not actually "balanced" or "fair" at all.
Finally, I was especially intrigued when an EAC member asked Carleton Jones about whether any of this would even be necessary if we went to a paired school model, because essentially this would highly favor students in the walk zone of a school. The response seemed to imply that it would still be in effect, though really the bottom line is that the EAC could easily recommend otherwise.
I'll look forward to hearing more about what you discover, as well as seeing what kinds of questions come up when the EAC has more time to consider the implications of what was explained.
Clarifications on BPS Algorithm Presentation
Josh:
First of all I would like to thank you for this blog post as well as for your questions at the end of the EAC meeting. Based on the discussions at the meeting and this blog post, there are a few points which could benefit from a clarification. Let me do my best for a clarification as follows:
1- The main purpose of our presentation was showing that, while there is walk-zone priority for
50% of the seats at each school, the actual BPS outcome is very close to the alternative scenario where none of the seats have walk-zone priority. We have tied this surprising outcome to the order in which walk-zone seats and open seats are allocated, and reading your post which clearly articulates this issue I feel we were able to explain this phenomenon to Boston community.
2- As I indicated during the presentation, we only presented the numbers for walk+sibling students and
walk (without sibling) students in our graphs, and the numbers on walk-zone students do not include students who are assigned a seat at their walk-zone via other ways. These include guaranteed students and administrative assignments. This point is also indicated as Note 1 at page 6 of presentation which is available at the following link at BPS website: http://bostonschoolchoice.files.wordpress.com/2013/01/bpsalgorithm_prese…
We tried to make it clear that 47% of the seats assigned to K1 students (for years 2009-2012) is the fraction of the students who receive a walk zone seat either via walk+sibling or via walk (but no sibling) and excludes guarantees plus administrative assignment. Here the role of administrative assignment is particularly important. These are students who are unmatched under the algorithm and later received an administrative assignment. This is about 24% of the K1 students for years 2009-2012. Finally, the fraction of non-walk students who received seats at schools in that period is about 28%. So for grade K1 and for years 2009-2012:
(a) 47% of the students are assigned a seat via walk+sibling or walk alone priority,
(b) 28% of the students are assigned a seat at a school for which they have no walk zone priority, and
(c) 24% of the students were unmatched by the algorithm and they later received an administrative assignment.
3- An important implication of the above point is, there are more students who are assigned a seat by the BPS algorithm to a school where they have walk zone priority (47%) than those who do not (28%).
Thus, if we exclude students who are unmatched by the BPS algorithm and thus received administrative assignments, about 62% of the students receive an assignment with walk zone priority.
This is the fraction of walk zone assignments by the algorithm, and the final actual number at schools depends on how administrative assignment was carried out, and in particular what fraction of administrative assignments were made to schools where students have walk zone priority.
4- The main message of the presentation which was showing that the outcome of BPS algorithm giving a result very close to one where there is no walk zone priority is not affected by any of the above discussions.
There is also one technical point I shall emphasize:
5- One of the alternatives we introduced in the presentation rotated between walk zone slots and open slots as you mentioned in your blog post. (This is what we called balanced treatment). To avoid the issue of walk zone students having unfavorable lottery numbers when they "compete" for open slots, this treatment utilizes two separate lottery numbers: One that is used for walk-zone seats and the other to be use with open seats.
We are in the process of revising the paper to clarify these points. In the revision we will also account for some of the students who do not submit choices but receive very high priority at their present schools. This list of students is call "final pass" students. When the first draft of the paper was written these students were excluded because they do not submit any preferences. We will make the revision available as soon as it is ready.
Hope this message is helpful.
Thank you for the clarification
Dr. Sonmez,
Thank you for the clarification, it's very helpful. This is pretty consistent with what I'm seeing in the data for K1 in the 2011-2012 school year. It looks like 59.6% of K1 students are walk zone kids. This would mean that the students you looked at and the students who were either guaranteed or were not assigned in the first round broke down in pretty similar ways.
The important point I want to make is that while walk zone priority, as currently implemented, may not significantly change the allocation of seats between walk zone and non-walk zone students, a significant majority of all seats are going to walk zone students. This is an important consideration as the EAC and BPS consider changes to this implementation, especially since any plan that reduces the geographic area with access to a given school is likely to increase the percentage of walk zone students attending that school.