sig: the sequel

Did Obama’s federal school turnaround program really fail?

Education Secretary Arne Duncan and HHS Secretary Kathleen Sebelius read to students enrolled in a Head Start program at Rolling Terrace Elementary School March 1, 2013 in Takoma Park, Maryland. (Photo by Win McNamee/Getty Images)

It has become a talking point for Betsy DeVos and a powerful example of the challenges of turning around struggling schools: a national study, released by the federal government, showing that its multibillion-dollar turnaround program failed.

“The previous administration spent seven billion of your dollars on ‘School Improvement Grants,’ thinking they could demonstrate that money alone would solve the problem,” DeVos said soon after becoming education secretary. “They tested their model, and it failed miserably.”

A new report says, not so fast. It points to studies of places like San Francisco, where the approach seemed to help students, and the limitations of the government’s study to conclude that the federal report painted too grim a picture.

“The autopsy on the grant program is flawed and its core conclusion faulty,” says the analysis, released by FutureEd, a Georgetown-based think tank generally supportive of Obama-era education policies.

It’s a year-old debate that remains relevant, as the study has become a touchstone for the idea that the federal government is unable to help long-struggling schools improve. And it comes as states, now with more freedom, are grappling with how to intervene in their lowest performing schools.

The federal turnaround program, known as School Improvement Grants or SIG, was a signature initiative of the Obama administration. In exchange for federal money, schools had to make changes, but had to use one of four approaches. About three-quarters chose the least disruptive option: firing the principal and making adjustments like lengthening the school day or toughening their teacher evaluations. Others replaced the principal and half the teaching staff. Few chose the other options, turning a school over to charter school operator or closing it altogether.

The initiative was evaluated by the external research firm Mathematica and the American Institutes for Research and released by the education department’s research arm, the Institute of Education Sciences. The results, released in January 2017, weren’t pretty.

“There was also no evidence that SIG had significant impacts on math or reading test scores, high school graduation, or college enrollment,” the study said.

The FutureEd report, written by two former Department of Education officials, suggests those conclusions are flawed for a few reasons, some of which were noted when the study was first released.

For one, it points out that even if School Improvement Grants successfully improved students’ academic outcomes, that would have been hard for the study to detect because the study’s bar for “statistical significance” was a very high one to clear.

The study actually estimated that the grants led to modest boosts in reading test scores and small declines in high school graduation rates — but neither impact was statistically significant, which is what the researchers mean when they say they found “no evidence.”

One of the report’s researchers, Lisa Dragoset of Mathematica, defended its approach. Setting a high bar for significance is reasonable for a program, like SIG, that was quite costly, she said. And even ignoring statistical significance, the estimated gains in reading were small and essentially zero in math, she noted.

“It’s unlikely that there were substantive or large impacts that were undetected by our study,” she said.

Second, FutureEd points out that the subset of schools in the federal study were not representative of all schools receiving federal grants.

The study compared schools receiving SIG grants to those schools near the eligibility cutoff that didn’t get grants. This is a widely used approach, but the FutureEd authors point out that the results then don’t say much about the effects of the grants on the lowest-performing schools. The studied schools were also disproportionately urban.

The new report also highlights a number of studies that focus on specific states and cities which paint a much more positive picture of the initiative. Most, though not all, of these studies find that the grants had positive effects on test scores.

“There are legitimate questions of whether the SIG program represented the best way to use federal funding to improve struggling schools,” the FutureEd authors conclude. “But it is wrong to suggest that there was no return on the SIG investment.”

Dragoset acknowledges that national results might not apply to all SIG schools, but says there’s nothing inconsistent with seeing no clear national effect but positive results in certain states.

“Ours, to my knowledge, is the only large-scale, rigorous study of the SIG program nationwide,” she said.

Dan Goldhaber, a University of Washington professor and vice president at the American Institutes for Research who reviewed the FutureEd report, said the limits of the federal study suggest policymakers shouldn’t “jump to the conclusion that the SIG program didn’t work.” But the study “was competently done,” he said.

The FutureEd analysis concludes that the U.S. Department of Education should spearhead a review of the research on turning around struggling schools. Tom Dee, a Stanford professor who found that the federal grants led to higher test scores in California schools, echoed this.

“I believe the question we should be asking is the following: why do federal reform catalysts seem to generate positive change in some states and communities but mere cosmetic regulatory compliance in others?” he said. “It seems to me that knowing more about the answer to that question is critical to efforts to drive meaningful change at scale.”

Yeshiva probe

As Yeshiva probe heats up, state issues guidance for reviewing nonpublic schools

PHOTO: Jackie Schechter
A school bus sits outside an all girls Jewish school in Williamsburg.

The state education department released long-awaited guidance Tuesday on the process that local authorities must follow to determine whether nonpublic and religious schools, including yeshivas, are meeting standards equivalent to those governing New York’s public schools.

The guidance arrives in New York City on the heels of a long-running probe into whether city yeshivas are providing an adequate secular education. The state direction also comes after the recent firing of former city investigator Mark Peters, whose office was scrutinizing City Hall’s involvement in the yeshiva investigation.

Will Mantell, spokesman for the city education department, said its officials will “work aggressively to implement” the state’s instructions.

Under the guidelines, local school districts must perform a review of each religious and independent school within their boundaries. But Tuesday’s guidance also folds in an amendment lawmakers passed this spring that largely applies to yeshivas: after an initial review by the local school district, the state education commissioner makes the final determination over schools that are nonprofit corporations, have a bilingual program, and operate during a certain time frame.

The new guidance comes after a three-year city Department of Education probe that found troubling lapses in secular education at the city’s yeshivas and asked for direction from the state, which recently granted oversight of the schools to the state education commissioner. Controversy heated up again last week as city education officials admitted they still haven’t visited many of the schools, whose students often come from the city’s ultra-Orthodox community, which is seen as a powerful voting bloc.

And last week, Mayor Bill de Blasio fired Peters, who quickly warned that de Blasio’s decision could reflect an effort to quash his office’s yeshiva probe. At a press conference on Monday, de Blasio denied that City Hall tried to interfere in any of Peters’ investigations.

The guidance, which stresses that oversight of nonpublic schools be “a collaborative effort,” sets out the procedure the city should follow and provides for a new round of training for investigators and a timeline of three years, up to December 2021, that districts can have to complete their reviews. Thereafter, districts will revisit the reviews every five years and maintain an open dialogue with nonpublic school leaders.

A preliminary city probe found that in many yeshivas instruction in English and math lasted only 90 minutes, didn’t take place every day, and was sometimes voluntary. Lessons in math didn’t go beyond basic division and fractions, science instruction was almost nonexistent, and teaching often occurred in languages other than English.

Naftuli Moster, the founder and executive director of Young Advocates For Fair Education, or YAFFED, an advocacy group that has pushed for more oversight of religious schools, thinks this timeline could stymie needed change. He notes that the city’s earlier review “may now have to be revisited in light of the new guidelines, dragging this investigation on for even longer while students in ultra-Orthodox schools continue to be deprived of a substantially equivalent education,” he said.We don’t believe that the yeshivas that have been stonewalling should be rewarded with even more time.”

In addition to core coursework, schools must abide by other requirements, including conducting “fire and emergency drills” and meeting “immunization requirements for their grades.” (A few Orthodox Jewish communities, which sometimes have low vaccination rates, have recently suffered outbreaks of measles in New York and New Jersey.)

New York City officials have reviewed many yeshivas already, and Elia said the city “should take the guidance that we have provided” and apply it to what they’ve found.

“The State has given the DOE clear authority to visit and evaluate all non-public schools, and we immediately requested the earliest possible staff training on the new guidance and will begin visits, evaluations, and recommendations and findings of substantial equivalency as soon as we’ve completed the training,” Mantell said.

The department will give priority to the “the six schools that have denied us access” and move “forward with the 24 schools that are part of our inquiry, which may include additional visits or gathering additional documents,” he said.

The state’s actions come as the number of students attending Jewish day schools and yeshivas in grades K-12 is growing rapidly, reaching a record 110,000, nearly rivaling the size of the city’s charter sector, which serves roughly 114,000 students. (Approximately 148,000 students attend parochial or independent day schools in the city.)

When asked about certain yeshivas denying the city access, Elia said, “If someone does not allow anyone in from the local school district to review and look at what’s happening there, there obviously would be consequences.”

The first remedy would be compelling schools to comply. But continued non-compliance could mean a loss of funding for certain services, like textbook and transportation, which Elia said is a rare occurrence. Parents at the schools would be notified, usually within six weeks to two months, that their children must be transferred to an appropriate school. If those students stay at the school past the established deadline, they could be marked as truant.

Christina Veiga and Alex Zimmerman contributed to this report.

Fixing Special Education

How will Chicago repair the harm from special-education neglect?

PHOTO: Adeshina Emmanuel / Chalkbeat
Laurel Henson, at the podium, spoke at a press conference in Chicago on Nov. 12, 2018, about her 2-year struggle to get a school nurse on staff to help her son, who suffers from seizures.

Illinois may be forcing Chicago Public Schools to repair its broken special education program, but the ambitious effort still begs a critical question: What happens to hundreds of Chicago children who were harmed by the district refusing them services that would  help them learn?

Neither the state nor the school district is saying yet, even as advocates for students in special education have pressed for answers.

Those children include an unnamed third-grader trapped by a tactic the district apparently used to avoid offering services required by federal law.

The child can’t read the word cat or dog, health-legal advocate Barbara Cohen said, but his teacher didn’t believe in giving low grades. So the third grader received a B in English. Then, she told the State Board of Education on Friday, when the child’s mother sought an evaluation for special education services, school officials denied the request based on his having a good grade.

Laura Boedeker, the state’s monitor overseeing special-education reforms, acknowledged that schools vary in understanding the laws and best practices. Her job, she said, “is to have those discussions and explain what good practices look like.”

That’s not likely to satisfy parents and advocates pushing for quicker action that would help families like the third-grader’s. On Friday, they pressed authorities like Boedeker, who previously served as the district’s in-house attorney.

But with a staff of just three, including herself, it’s not clear how fast Boedeker can move. In four-and-a-half months on the job, she’s only visited 10 of the district’s 600-plus schools.

“Do you have enough boots on the ground, enough help to do this work at the rate you need to do it?” asked Illinois State Board of Education member Susie Morrison.

“We could have an army and not have enough boots on the ground,” said Stephanie Jones, the board’s general counsel. “What we need more than anything is eyes and ears that tell us what is going on so we can take action. Unless we can put an ISBE employee in every school, which is unrealistic, we need parents and teachers and staff members to tell us what is going on.”

Recognizing the lag in responding to parents, the state board is weighing whether to extend the one-year deadline for filing complaints about denied or improper services.

It’s possible, Jones said, that “we can wave this until we have a system of corrective action in place.”

Neither the state nor district have answered questions like: How many students could be eligible? When exactly will the system go into effect? And what roles should advocates and schools play?

Boedeker said that federal officials have insisted that teams who put together students’ individualized education programs be involved in the remedy, because “they’re the ones on the front lines with these students.” 

But lawyer Matt Cohen said he and other advocates want a process that involves more people than the IEP team.

A child who, for example, went without a one-on-one aide for many months or who didn’t get placed in therapeutic day school when needed “might have had a profound loss,” Cohen said.

How the district will compensate that family is the question.  

“They may need more than just a few hours of tutoring to make up for that, they may need months and months of additional services and a specialized process to help them catch up,” Cohen said. “We’re encouraging families whose kids were hurt to bring their complaints to the state, and to seek action to get their individual child’s needs met.”

Jones said that board officials and the school district, federal government and special education advocates are discussing school guidelines for identifying students harmed, notifying their families, assessing damages and offering remedies.

About half a year has passed since a state probe found the school district violated students’ rights by routinely delaying and denying services — like aides, therapy, outside placement and busing — to students in what the district calls its Diverse Learners Program.

The state board’s Jones and Boedeker tried to placate critics by preaching patience.

“From the outside looking in it looks really slow,” Jones said, “but I think we’ve accomplished a great deal in the time we have had.”

Patience doesn’t sit well with parents desperately worried about their children.

Laurel Henson, whose son suffers from seizures, said she’s been pushing to get a nurse on staff at Smyser Elementary for two years, but has encountered “delays and excuses.” On Nov. 1, the school finally granted a meeting to discuss an IEP, she said.

“In that time, he’s had a significant increase in seizures at his school causing fatigue, aggression and bed wetting during the night,” she said. Despite her hopes for the monitor, “nothing has improved for my son and it now feels like neither CPS nor the state are accountable for ensuring students like my son have a free and appropriate education.”