9 ms·

How to Analyze False Claims about Charter Schools

41 points by ph0rque 12y ago

squozzer 12y ago
Not sure why the author would bother analyzing claims she has already decided were false.
- dsjoerg 12y ago
  Even if the article were instead titled with the more accurate "How to refute..." it would nevertheless be interesting and worthy of further discussion.
- msabalau 12y ago
  It seems straightforward enough to understand that if you believed that there was a pattern of zombie memes recurring, that it would be helpful to have a structure to efficiently evaluate poor journalism.
- shoover 12y ago
  The more accurate title is in the article: How to Read News Stories about Charter Schools. And then the justification for why bother: It grows tiresome to dispute every tendentious article written on charter schools. But let's see how the ICBS strategy would help us evaluate a sample story.
- crusso 12y ago
  Because it's a political hit piece, designed to help nip this nasty threat to the established public education power structure in the bud. At the end of the day, school choice doesn't matter. The fact that Charter schools are relatively new and just finding their way doesn't matter. I'm all for apples to apples comparisons. But when you have a huge basket of fruit you're trying to normalize to apples, it's really telling which ones are picked.
  - pdabbadabba 12y ago
    Your comment would be more persuasive is it were accompanied by evidence that the errors described in the article were not, in fact, pervasive and that there were real data to indicate that charter schools outperform ordinary public schools. As it is, your comment reads like a similar attempt to nip a nasty threat to charter schools in the bud.
    - crusso 12y ago
      Your comment indicates that you didn't really even read or attempt to think about mine. You're asking me to nitpick statistics that frame the exact worldview the article was pushing. I'm saying that in the big picture, those probably aren't the specific statistics that many people even care much about at this point. It's like an article comparing a Tesla to a Ford pickup truck that focuses solely on 0-60 speed and fuel costs but never bothers talking about towing capacity, load capacity, ability to refuel anywhere, etc.
      - pdabbadabba 12y ago
        I still don't get it. The article only points out that, in determining whether a Charter School provides a superior education to ordinary public schools (which is the claim made by the Economist and many other sources in the media) we need to know more than raw test scores. We need to control for a number of variables, such as the resources of attendees, etc. If Charter Schools are attended by (e.g.) wealthier students than public schools, or students with more involved parents, then a charter school's superior aggregate performance on a standardized test tells us precisely zero about the educational quality of the school unless one can somehow compensate for this confounding variable. Yet this is routinely overlooked in the media. You seem to be saying that we shouldn't care about controlling for variables like this and should, instead, look at other ones. OK. But you don't tell us why, or what statistics would be preferable, or give any evidence to suggest that publications like the Economist actually consider your preferred measures in lieu of those suggested by the article (remember: the article is about how to evaluate the claims made in the media about charter schools, not about the merits of charter schools themselves -- though I readily grant that the author clearly has a view on the latter topic as well).
- pdabbadabba 12y ago
  I think you have this backwards. > It was, as is typical among news stories, incredibly naive. The writer didn’t ask the right questions. Maybe he already believed in the charter “miracle” story and didn’t ask any questions. The author concluded that the claims in the story were probably false (or at least not supported by evidence) on the basis of the framework described in the article. That's what is meant by "didn't ask the right questions." (Of course, given that the author is well versed in this area, it did not take much effort to spot the mistakes.)
simon_ 12y ago
If we stipulate that the charter schools in question have strong selection biases in their student bodies and better resources relative to public schools, then: a) It's true that statistics are frequently misused by journalists and advocates, BUT: b) It hardly impugns the schools themselves. Taking a subset of more promising students without behavioral issues out of failing public schools and giving them better resources at charters is not necessarily a bad idea.
- scott_s 12y ago
  Point b) is a problem if charter schools are held up as a general solution to the problems with our schools - because the solution obviously doesn't scale. (The hard-to-teach students need to go somewhere.) Independent of that, it's also problematic because you're essentially giving up on the "failing schools public schools", by taking resources away from them.
  - humanrebar 12y ago
    Call me crazy, but I have no emotional attachment to buildings and organizations (whichever you mean by schools). As long as the students in failing public schools have at least replacement-level alternatives, there won't be a net negative here. > The hard-to-teach students need to go somewhere. That's obvious. The criticism is that, in the current system, laws and tradition have dictated that "somewhere" is Oswald Cobblepot High School down the street, and if that school is a hole, sucks to be you. Blame the voters that didn't pass the bond issue five years ago. Or get rich and move to a nicer neighborhood. And that last point is especially important. The rich have places to send their problem students. And they can afford to move to overpriced neighborhoods that effectively weed out at-risk students from the student body. The current system (unintentionally) creates public schools full of hard-to-teach students.
    - vonmoltke 12y ago
      A potential solution, if the district is large enough, to that is to create an open enrollment district. Stop forcing kids to go to a specific school just because they happen to live at a certain address, or change schools because they need to move.
simeonf 12y ago
Sentence two of the expert advice is propaganda. >The majority of these stories praise charters, while often demeaning public schools. Charter schools are public schools - they are just less regulated ones.
- jack-r-abbit 12y ago
  They also (typically) get less money per student than non-charter public schools. So even if they are only equally as good, they still win for spending less of the state budget.
- phkahler 12y ago
  >> Charter schools are public schools - they are just less regulated ones. No, they are not. They are run by private corporations. That's like saying McDonalds is a public restaurant because they are open to the public. Please don't mix common understanding of what words mean with some plausible other use of those words, it does not actually help your cause.
  - Joky 12y ago
    What is the difference between a charter school and a private school then? Why do a charter school gets the same amount of public money from the district as the other school in the district?
  - wil421 12y ago
    Not sure where you live but in Georgia charter schools are not run by private corporation like McDonalds. They take public funding just like every school but have more freedom in their curriculum.[1] They are not governed by a board of education but by a non profit board with less rules but higher accountability by the state. [1]http://www.gadoe.org/external-affairs-and-policy/charter-schools/pages/general-frequently-asked-questions.aspx http://www.gadoe.org/external-affairs-and-policy/charter-sch...
    - depingus 12y ago
      Charter schools are run by for-profit management companies. http://www.miamiherald.com/news/special-reports/cashing-in-on-kids/article1939199.html http://www.miamiherald.com/news/special-reports/cashing-in-o...
crazy1van 12y ago
The public school system isn't exactly stellar. I don't claim to know why, but I don't see how competition does anything other than make the whole system stronger. Why so much fear if you are confident you have a superior product?
- GabrielF00 12y ago
  It's not clear to me that competition for students does make the whole system better. You end up with a phenomenon where some schools get used as dumping grounds for the kids that the other schools don't want to deal with or whose parents don't care or don't know any better. I witnessed this when I had a part-time teaching job in the Boston Public Schools. I spent a year at Jeremiah Burke High School, which has the reputation as one of the worst in the state. A huge proportion of kids there either did not speak much English (Cape Verdean Creole was very common), or had severe learning or behavioral issues. The problems that exist in public education are incredibly complex, and it's not clear to me that any simple, ideologically-tinged solution is going to solve them.
  - crazy1van 12y ago
    > The problems that exist in public education are incredibly complex, and it's not clear to me that any simple, ideologically-tinged solution is going to solve them. This is exactly why I want competition. A few experts sitting around a conference table for a year are not going to just deduce the solution, just like the best VCs invest in many companies instead of just picking only the Facebooks. These problems are far too complicated for a centrally architected solution. In the end the best solution might be one that most people never even imagined.
xname 12y ago
The author is silly. Studies have been conducted to account for selection bias. Because, by law, students have to be randomly selected to get into charter schools with lotteries, it is not hard to control for the selective effects. Check this: http://ies.ed.gov/ncee/pubs/20104029/ http://ies.ed.gov/ncee/pubs/20104029/ and this http://seii.mit.edu/wp-content/uploads/2013/10/Charter-School-Demand-and-Effectiveness.pdf http://seii.mit.edu/wp-content/uploads/2013/10/Charter-Schoo... Also, I don't understand why so many leftists hate charter schools so much. First, charter schools are public schools, not private schools. Second, charter schools are created to help the poor, not the rich. What's wrong with those leftists? I just can't understand. Do you really love teachers union so much that poor kids mean nothing to you?
- japhyr 12y ago
  If you randomly select students for enrollment through a lottery, but then weed out difficult students, you've introduced a selection bias for graduation. This is not a very good approach to "helping the poor".
  - jaynos 12y ago
    Also, the lottery is used to choose from those that apply (not all students in a district). Students with parents that are more involved are the ones that apply for the lottery.
    - jack-r-abbit 12y ago
      You say that like it is a bad thing. But that is exactly how a lottery works. If you don't have a Powerball ticket, you can't sit at home and watch the drawing on TV and then complain you didn't win. Parents should be involved in the education of their children. If they take the (tiny, tiny) extra effort to fill out the application to a charter school in addition to enrolling in their designated neighborhood school then those are the ones that have a chance to win. You can't blame the school for the parents not involved enough to make an effort. (and really... the effort is not that great)
clavalle 12y ago
Basically the author is claiming that we have to hold all variables equal to compare charter schools and traditional public schools in order to 'be wise' and 'think critically' about charter schools. Why is that necessarily the case? What if the expulsion rate (dropping disruptive students) and parent engagement is part of the 'secret sauce' of charter school success? Why should data that is based on these factors be considered 'bogus statistics'? There is some good, common sense advice in this article but it considers too many fundamental factors 'bogus'. This is a bit of a tangent but my fundamental problem with the traditional public school system is this: why should my child's education be primarily determined by where our domicile happens to be located?
- scott_s 12y ago
  To answer your first questions: because in the US, we typically assume our mandate is to educate everyone. If the "secret sauce" is to marginalize the students who are in the most need to help, then that does not achieve our goal.
  - mattmcknight 12y ago
    It does if the problem students are denying a quality education to students who can't afford to move away from them. It only takes a small percentage to disrupt the effectiveness of a traditional classroom.
  - clavalle 12y ago
    Why is it assumed that to educate everyone to the best of their ability that the traditional public school system, where everyone is thrown together based on where they live, is the best to achieve that goal and anything else leads to marginalization? I used to work at educating blind children. Most of those I worked with also had other disabilities as well. The school I worked at was a boarding school and first rate. It concentrated experts into one place and offered continuity of instruction beyond anything a public district could provide. Student progress was, on the whole, amazing compared to their progress at a public school. This is an extreme example, granted, but why wouldn't a similar approach work with other kids who need extra help and attention? Why not have behavior specialists at one charter school working with disruptive kids and engineers teaching kids that are interested in STEM?
tokenadult 12y ago
From the article: "So my correspondent–who requires anonymity– decided that it would be helpful to reporters and members of the public to explain how to read stories about charter schools." So we don't know who is advancing these comments. It is plainly not Diane Ravitch herself, and we have no idea who the person is, so we have no idea if the person suggesting this checklist is a person with a conflict of interest in education policy. I'll go down the checklist item by item. "Does the story compare the demographics of the student population served by charter schools to the demographics of local public schools?" What aspect of "demographics" is most relevant here? There are already charter school programs (not many, but a few) that do better with mostly pupils who come from poor homes and parents without higher education than many other schools do with pupils who come from high-income homes.[1] A lot of public schools in the United States serve up mediocre results while serving well off pupils. "Does it include data on the charter school attrition rate?" This is a fair item to put on the checklist, and should be reported more in stories about schools in general. In other words, a story about public schools should also report on their attrition rate, and on what rate at which pupils move in and out of the school district, and in general how stable or unstable enrollments are at all schools being compared. (The United States has a highly geographically mobile population, and it would be a very unusual school that has the same group of pupils enrolled by sixth grade as was enrolled in first grade five years earlier.) "Does it include data on how the students who leave the charters compare to students who leave public schools?" Again, this is a question well worth asking, but we shouldn't guess, until we have seen the answer to questions like this, whether charter schools or public schools would come out better in a comparison of this kind. We should be checking the facts in each individual case all around the country. "Does it include numbers of students expelled?" Since the early 1990s, I have been aware of homeschooling advocates who talk about "push outs" (not "drop outs") from the public school system, kids who are basically told to like it or lump it in the public school system. People leaving the public school system should always be looked at closely by researchers to figure out why they left, and whether they are later able to find a more fitting school situation in which they can learn better. "Does it include numbers of students suspended?" As above, this is a question to ask about any kind of school. Moreover, at the extremes, a school that never suspends any pupil is possibly a school that is not trying hard enough to maintain a learning environment for all other pupils enrolled, so a researcher would want to look at grounds for suspension, procedures surrounding suspension, what corrective steps are taken when a pupil is suspended, and so on. I've certainly seen suspension abused in public school settings. "Does the story focus exclusively on test scores?" This is a general defect of reporting on schools. At the broad statistical level, and especially for international comparisons of school systems, test scores[2] are mostly what we have to look at. What I also look at, as someone who has lived in more than one country and who reads more than one language, is the actual item content of textbooks and the attitudes toward learning shown by teachers and pupils. I think some school systems in other countries compare very favorably to those in the United States (not least because pupils in those other countries begin foreign language study much earlier than pupils in the United States, a fact not usually reflected in international testing programs). Educational researchers should be looking more closely at the actual item content of school textbooks and at the specific teacher practices in each classroom.[3] "If so, has someone, with educational expertise, visited the school to determine if the school focuses on test prep at the expense of a rich curriculum?" As above, we should be looking in detail at what actually goes on in the classroom. Indeed, we should be looking at whether persons employed as teachers are actually teaching anything, which we shouldn't assume sight unseen.[4] "Are the test scores reported outside of school assessments such as the SAT/ACT or does the story only report test scores of exams that are proctored in-house?" This is also important, and also a question that should be applied to news stories about schools in general, not just about charter schools. So far many of the worst cheating scandals in school testing programs have originated in regular public schools.[5] "Does the story account for the fact that, due to the need to apply to the charter school, parents of the students at charters are, on average, likely to be more engaged in education than the parents of students at public schools?" The policy response to this is to make school choices much more widely available to many more families. I have seen some examples of helpful reforms where I live. Minnesota, where I now live and where I grew up, has had largely equal per-capita funding for public school pupils statewide since the 1970s. The state law change that made most school funding come from general state appropriations rather than from local property taxes was called the "Minnesota miracle."[6] Today most funding for schools is distributed by the state government on a per-pupil enrollment basis.[7] The funding reform in the 1970s was followed up by two further reforms in the 1980s. First, the former compulsory instruction statute in Minnesota was ruled unconstitutional in a court case involving a homeschooling family, and a new compulsory instruction statute explicitly allows more nonpublic school alternatives for families who seek those. Second, the Legislature, pushed by the then Governor, set up statewide open enrollment[8] and the opportunity for advanced learners to attend up to two years of college while still high school students on the state's dime.[9] And Minnesota also has the oldest charter school statute in the United States.[10] We haven't had big problems with charter school performance here. Parents in Minnesota now have more power to shop than parents in most states. That gets closer to the ideal of detecting the optimum education environment for each student (by parents observing what works for each of their differing children) and giving it to them by open-enrolling in another school district (my school district has inbound open-enrollment students from forty-one other school districts of residence) or by homeschooling, or by postsecondary study at high school age, or by exercising other choices. The educational results of Minnesota schools are well above the meager results of most United States schools, and almost competitive (but not fully competitive) with the better schools in the newly industrialized countries of east Asia and southeast Asia. It's a start. More choices would be even better. "Does it exclusively or primarily cite reports funded by pro-charter or conservative think tanks?" Here it's regrettable that we don't know the identity of the author of this checklist. Do we know whether or not the author is funded by pro-public-school interest groups? Disclosure of that would be helpful here. "Does it include quotes from academic scholars or does it just cite charter school advocates?" I can recommend a website full of the writings of actual academic scholar to Hacker News participants interested in education reform. That is the Education Next website,[11] which is better supplied than most English-language websites about education policy with information about education policy outside the United States, which is important for a reality-check on issues discussed in this country. "Does it identify advocates or simply call them 'experts' or 'researchers'?" This is a funny question for a commentator who insisted on anonymity to ask. "Does it compare the resources available to charter schools to those available to public schools?" In Minnesota, and in all states I have checked, the resources available to charter schools (funding per student, and availability of buildings) are strictly less than the resources available to public school disticts, by law. Yes, lets's check that detail carefully. It's commendable to receive a suggestion that we should check news stories carefully. We should always do that here on Hacker News. Let's check stories on education policy especially carefully, as public school spending is a huge part of state budgets[12] and a lot of interest groups are lined up behind the public trough. [1] http://educationnext.org/when-the-best-is-mediocre/ http://educationnext.org/when-the-best-is-mediocre/ [2] http://www.oecd.org/pisa/ http://www.oecd.org/pisa/ http://timssandpirls.bc.edu/ http://timssandpirls.bc.edu/ [3] http://www.amazon.com/Knowing-Teaching-Elementary-Mathematics-Understanding/dp/0415873843 http://www.amazon.com/Knowing-Teaching-Elementary-Mathematic... http://www.timssvideo.com/timss-video-study http://www.timssvideo.com/timss-video-study [4] http://www.pbs.org/newshour/bb/students-take-a-stand-to-topple-teacher-tenure/ http://www.pbs.org/newshour/bb/students-take-a-stand-to-topp... [5] http://www.ajc.com/s/news/school-test-scores/#top-stories http://www.ajc.com/s/news/school-test-scores/#top-stories http://online.wsj.com/articles/for-school-tests-measures-to-detect-cheating-proliferate-1411752291 http://online.wsj.com/articles/for-school-tests-measures-to-... http://www.newyorker.com/magazine/2014/07/21/wrong-answer http://www.newyorker.com/magazine/2014/07/21/wrong-answer http://www.washingtonpost.com/blogs/answer-sheet/wp/2014/08/11/50-ways-schools-cheat-on-high-stakes-standardized-tests/ http://www.washingtonpost.com/blogs/answer-sheet/wp/2014/08/... [6] http://www.mnhs.org/library/tips/history_topics/18public.html http://www.mnhs.org/library/tips/history_topics/18public.htm... [7] http://www.house.leg.state.mn.us/hrd/pubs/mnschfin.pdf http://www.house.leg.state.mn.us/hrd/pubs/mnschfin.pdf http://education.state.mn.us/MDE/SchSup/SchFin/index.html http://education.state.mn.us/MDE/SchSup/SchFin/index.html [8] http://education.state.mn.us/MDE/StuSuc/EnrollChoice/index.html http://education.state.mn.us/MDE/StuSuc/EnrollChoice/index.h... [9] http://education.state.mn.us/MDE/StuSuc/CollReadi/PSEO/index.html http://education.state.mn.us/MDE/StuSuc/CollReadi/PSEO/index... [10] http://www.amazon.com/Zero-Chance-Passage-Pioneering-Charter/dp/1592984762/ http://www.amazon.com/Zero-Chance-Passage-Pioneering-Charter... [11] http://educationnext.org/ http://educationnext.org/ [12] http://nces.ed.gov/fastfacts/display.asp?id=66 http://nces.ed.gov/fastfacts/display.asp?id=66 http://www2.census.gov/govs/school/12f33pub.pdf http://www2.census.gov/govs/school/12f33pub.pdf
japhyr 12y ago
Much of the conversation in this comment thread assumes a public vs. charter school split. If my understanding is correct, these charter schools are public schools. The distinction is that the charter schools have selection criteria. A public school with selection criteria is not, by itself, a bad thing. I love that schools like Stuyvesant HS and Bronx School of Science exist, but that's because we all know what their selection criteria are. I don't like charter schools pretending that the only thing it takes to be part of the school is winnning an entrance lottery. That might be the main way to get in, but it's not all it takes to stay enrolled. Many charter school advocates seem to be ignoring, or ignorant of, the fact that these schools actively weed out students who are difficult to reach. It's okay to build some schools that require a more serious commitment to learning; it's not okay to build schools like this and pretend they serve everyone, or do a better job than schools who have to keep every eligible student enrolled.
- tokenadult 12y ago
  Charter schools by law do not have selection criteria. The example you mention, the Bronx High School of science, is a public school with an entrance examination, usually called an "exam school" for short. No charter school anywhere in the country can operate on that basis, as all charter school statutes everywhere explicitly ban examination requirements for entrance.
  - japhyr 12y ago
    Thanks for the clarification, I'm glad I qualified that comment!
snake_plissken 12y ago
>Reports and stories about charter schools are in the media every day. The majority of these stories praise charters, while often demeaning public schools. Interesting. My experience with how the media presents charter schools is quite the opposite. It seems like every few weeks there is a story about a corrupt charter school management team embezzling funds or over-enrolling of students or suspect academic results. Of course with stories on actual events, such as an investigation into missing funds, it's difficult if not downright impossible, if a media outlet so desired, to spin the story into something positive. But I don't recollect reading anything within the past few years that actually praised charter schools, and that's not to mean the traditional public school system is squeaky clean and a model of best practices. I suspect there is an equivalent level of corruption there. I've wondered if it's because I live in Philadelphia where this a strong teachers union and a traditional public school system that is, for all intents and purposes, bankrupt. However, I would expect that these conditions would lead to the opposite in how charter schools are portrayed, if only for the sake that they are an alternative to the current system that is failing miserably. The charter school debate is a sub-segment of a larger issue, which is this push towards the privatization of everything that is consumed by the public and paid for by tax dollars, from public school systems to prisons to health care programs, such as Medicaid. Personally, I disagree with this push. It creates perverse incentives and from a political economy point of view, I think there are some goods which can't and should not be privatized, but that discussion is for another time.
mbubb 12y ago
This is really interesting - to be honest I am at work and am resisting being pulled into reading this until tonight... I have a good friend who I have had since college days. Has been an NYC public school teacher for a while. Loves to teach - onel of those folks that is constantly formulating ideas, reads voraciously, argues passionately, etc. I see him as caught between the attempt to corporatize education and a bloated, syphillic (sp) union... My ideas not his. On my side I had kids in public school until 2nd grade where I saw disturbing things that made me jump ship to a charter where my boys thrive. This is anecdotal - I do not want to try and prove/ disprove anything by this. I wish we had a viable pub school option locally but we don't at the moment. Our school survived a scummy attack on charters locally in which false claims were made about segregation. This was disproven but we were made aware of the fact that the other side has more resources and deeper legal pockets then the charter school does. As I said I will sit down and read this later. I am interested in something that comes up when my buddy and I talk about this stuff. Somewhere I picked up a concept from French which I will anglicize as "Deformation Professionelle" (my apologies for butchering it - I will properly google it later). ed. http://en.wikipedia.org/wiki/D%C3%A9formation_professionnelle http://en.wikipedia.org/wiki/D%C3%A9formation_professionnell... I see this when tech and teachers discuss education. I sit and talk to him about tech issues - 'net neutrality', privacy issue, (going back a decade) attacks on open source software, etc. And he doesn't know about these things. And I (at times) wax enthusiastic about things like Khan Academy or charter schools without seeing possible drawbacks. I look forward to giving this a proper read later - am glad to see this on HN.

41 points by ph0rque 12y ago

squozzer 12y ago

Not sure why the author would bother analyzing claims she has already decided were false.

dsjoerg 12y ago
Even if the article were instead titled with the more accurate "How to refute..." it would nevertheless be interesting and worthy of further discussion.
msabalau 12y ago
It seems straightforward enough to understand that if you believed that there was a pattern of zombie memes recurring, that it would be helpful to have a structure to efficiently evaluate poor journalism.
shoover 12y ago
The more accurate title is in the article: How to Read News Stories about Charter Schools. And then the justification for why bother: It grows tiresome to dispute every tendentious article written on charter schools. But let's see how the ICBS strategy would help us evaluate a sample story.
crusso 12y ago
Because it's a political hit piece, designed to help nip this nasty threat to the established public education power structure in the bud. At the end of the day, school choice doesn't matter. The fact that Charter schools are relatively new and just finding their way doesn't matter. I'm all for apples to apples comparisons. But when you have a huge basket of fruit you're trying to normalize to apples, it's really telling which ones are picked.
- pdabbadabba 12y ago
  Your comment would be more persuasive is it were accompanied by evidence that the errors described in the article were not, in fact, pervasive and that there were real data to indicate that charter schools outperform ordinary public schools. As it is, your comment reads like a similar attempt to nip a nasty threat to charter schools in the bud.
  - crusso 12y ago
    Your comment indicates that you didn't really even read or attempt to think about mine. You're asking me to nitpick statistics that frame the exact worldview the article was pushing. I'm saying that in the big picture, those probably aren't the specific statistics that many people even care much about at this point. It's like an article comparing a Tesla to a Ford pickup truck that focuses solely on 0-60 speed and fuel costs but never bothers talking about towing capacity, load capacity, ability to refuel anywhere, etc.
    - pdabbadabba 12y ago
      I still don't get it. The article only points out that, in determining whether a Charter School provides a superior education to ordinary public schools (which is the claim made by the Economist and many other sources in the media) we need to know more than raw test scores. We need to control for a number of variables, such as the resources of attendees, etc. If Charter Schools are attended by (e.g.) wealthier students than public schools, or students with more involved parents, then a charter school's superior aggregate performance on a standardized test tells us precisely zero about the educational quality of the school unless one can somehow compensate for this confounding variable. Yet this is routinely overlooked in the media. You seem to be saying that we shouldn't care about controlling for variables like this and should, instead, look at other ones. OK. But you don't tell us why, or what statistics would be preferable, or give any evidence to suggest that publications like the Economist actually consider your preferred measures in lieu of those suggested by the article (remember: the article is about how to evaluate the claims made in the media about charter schools, not about the merits of charter schools themselves -- though I readily grant that the author clearly has a view on the latter topic as well).
pdabbadabba 12y ago
I think you have this backwards. > It was, as is typical among news stories, incredibly naive. The writer didn’t ask the right questions. Maybe he already believed in the charter “miracle” story and didn’t ask any questions. The author concluded that the claims in the story were probably false (or at least not supported by evidence) on the basis of the framework described in the article. That's what is meant by "didn't ask the right questions." (Of course, given that the author is well versed in this area, it did not take much effort to spot the mistakes.)

simon_ 12y ago

If we stipulate that the charter schools in question have strong selection biases in their student bodies and better resources relative to public schools, then: a) It's true that statistics are frequently misused by journalists and advocates, BUT: b) It hardly impugns the schools themselves. Taking a subset of more promising students without behavioral issues out of failing public schools and giving them better resources at charters is not necessarily a bad idea.

scott_s 12y ago
Point b) is a problem if charter schools are held up as a general solution to the problems with our schools - because the solution obviously doesn't scale. (The hard-to-teach students need to go somewhere.) Independent of that, it's also problematic because you're essentially giving up on the "failing schools public schools", by taking resources away from them.
- humanrebar 12y ago
  Call me crazy, but I have no emotional attachment to buildings and organizations (whichever you mean by schools). As long as the students in failing public schools have at least replacement-level alternatives, there won't be a net negative here. > The hard-to-teach students need to go somewhere. That's obvious. The criticism is that, in the current system, laws and tradition have dictated that "somewhere" is Oswald Cobblepot High School down the street, and if that school is a hole, sucks to be you. Blame the voters that didn't pass the bond issue five years ago. Or get rich and move to a nicer neighborhood. And that last point is especially important. The rich have places to send their problem students. And they can afford to move to overpriced neighborhoods that effectively weed out at-risk students from the student body. The current system (unintentionally) creates public schools full of hard-to-teach students.
  - vonmoltke 12y ago
    A potential solution, if the district is large enough, to that is to create an open enrollment district. Stop forcing kids to go to a specific school just because they happen to live at a certain address, or change schools because they need to move.

simeonf 12y ago

Sentence two of the expert advice is propaganda. >The majority of these stories praise charters, while often demeaning public schools. Charter schools are public schools - they are just less regulated ones.

jack-r-abbit 12y ago
They also (typically) get less money per student than non-charter public schools. So even if they are only equally as good, they still win for spending less of the state budget.
phkahler 12y ago
>> Charter schools are public schools - they are just less regulated ones. No, they are not. They are run by private corporations. That's like saying McDonalds is a public restaurant because they are open to the public. Please don't mix common understanding of what words mean with some plausible other use of those words, it does not actually help your cause.
- Joky 12y ago
  What is the difference between a charter school and a private school then? Why do a charter school gets the same amount of public money from the district as the other school in the district?
- wil421 12y ago
  Not sure where you live but in Georgia charter schools are not run by private corporation like McDonalds. They take public funding just like every school but have more freedom in their curriculum.[1] They are not governed by a board of education but by a non profit board with less rules but higher accountability by the state. [1]http://www.gadoe.org/external-affairs-and-policy/charter-schools/pages/general-frequently-asked-questions.aspx http://www.gadoe.org/external-affairs-and-policy/charter-sch...
  - depingus 12y ago
    Charter schools are run by for-profit management companies. http://www.miamiherald.com/news/special-reports/cashing-in-on-kids/article1939199.html http://www.miamiherald.com/news/special-reports/cashing-in-o...

crazy1van 12y ago

The public school system isn't exactly stellar. I don't claim to know why, but I don't see how competition does anything other than make the whole system stronger. Why so much fear if you are confident you have a superior product?

GabrielF00 12y ago
It's not clear to me that competition for students does make the whole system better. You end up with a phenomenon where some schools get used as dumping grounds for the kids that the other schools don't want to deal with or whose parents don't care or don't know any better. I witnessed this when I had a part-time teaching job in the Boston Public Schools. I spent a year at Jeremiah Burke High School, which has the reputation as one of the worst in the state. A huge proportion of kids there either did not speak much English (Cape Verdean Creole was very common), or had severe learning or behavioral issues. The problems that exist in public education are incredibly complex, and it's not clear to me that any simple, ideologically-tinged solution is going to solve them.
- crazy1van 12y ago
  > The problems that exist in public education are incredibly complex, and it's not clear to me that any simple, ideologically-tinged solution is going to solve them. This is exactly why I want competition. A few experts sitting around a conference table for a year are not going to just deduce the solution, just like the best VCs invest in many companies instead of just picking only the Facebooks. These problems are far too complicated for a centrally architected solution. In the end the best solution might be one that most people never even imagined.

xname 12y ago

The author is silly. Studies have been conducted to account for selection bias. Because, by law, students have to be randomly selected to get into charter schools with lotteries, it is not hard to control for the selective effects. Check this: http://ies.ed.gov/ncee/pubs/20104029/ http://ies.ed.gov/ncee/pubs/20104029/ and this http://seii.mit.edu/wp-content/uploads/2013/10/Charter-School-Demand-and-Effectiveness.pdf http://seii.mit.edu/wp-content/uploads/2013/10/Charter-Schoo... Also, I don't understand why so many leftists hate charter schools so much. First, charter schools are public schools, not private schools. Second, charter schools are created to help the poor, not the rich. What's wrong with those leftists? I just can't understand. Do you really love teachers union so much that poor kids mean nothing to you?

japhyr 12y ago
If you randomly select students for enrollment through a lottery, but then weed out difficult students, you've introduced a selection bias for graduation. This is not a very good approach to "helping the poor".
- jaynos 12y ago
  Also, the lottery is used to choose from those that apply (not all students in a district). Students with parents that are more involved are the ones that apply for the lottery.
  - jack-r-abbit 12y ago
    You say that like it is a bad thing. But that is exactly how a lottery works. If you don't have a Powerball ticket, you can't sit at home and watch the drawing on TV and then complain you didn't win. Parents should be involved in the education of their children. If they take the (tiny, tiny) extra effort to fill out the application to a charter school in addition to enrolling in their designated neighborhood school then those are the ones that have a chance to win. You can't blame the school for the parents not involved enough to make an effort. (and really... the effort is not that great)

clavalle 12y ago

Basically the author is claiming that we have to hold all variables equal to compare charter schools and traditional public schools in order to 'be wise' and 'think critically' about charter schools. Why is that necessarily the case? What if the expulsion rate (dropping disruptive students) and parent engagement is part of the 'secret sauce' of charter school success? Why should data that is based on these factors be considered 'bogus statistics'? There is some good, common sense advice in this article but it considers too many fundamental factors 'bogus'. This is a bit of a tangent but my fundamental problem with the traditional public school system is this: why should my child's education be primarily determined by where our domicile happens to be located?

scott_s 12y ago
To answer your first questions: because in the US, we typically assume our mandate is to educate everyone. If the "secret sauce" is to marginalize the students who are in the most need to help, then that does not achieve our goal.
- mattmcknight 12y ago
  It does if the problem students are denying a quality education to students who can't afford to move away from them. It only takes a small percentage to disrupt the effectiveness of a traditional classroom.
- clavalle 12y ago
  Why is it assumed that to educate everyone to the best of their ability that the traditional public school system, where everyone is thrown together based on where they live, is the best to achieve that goal and anything else leads to marginalization? I used to work at educating blind children. Most of those I worked with also had other disabilities as well. The school I worked at was a boarding school and first rate. It concentrated experts into one place and offered continuity of instruction beyond anything a public district could provide. Student progress was, on the whole, amazing compared to their progress at a public school. This is an extreme example, granted, but why wouldn't a similar approach work with other kids who need extra help and attention? Why not have behavior specialists at one charter school working with disruptive kids and engineers teaching kids that are interested in STEM?

tokenadult 12y ago

From the article: "So my correspondent–who requires anonymity– decided that it would be helpful to reporters and members of the public to explain how to read stories about charter schools." So we don't know who is advancing these comments. It is plainly not Diane Ravitch herself, and we have no idea who the person is, so we have no idea if the person suggesting this checklist is a person with a conflict of interest in education policy. I'll go down the checklist item by item. "Does the story compare the demographics of the student population served by charter schools to the demographics of local public schools?" What aspect of "demographics" is most relevant here? There are already charter school programs (not many, but a few) that do better with mostly pupils who come from poor homes and parents without higher education than many other schools do with pupils who come from high-income homes.[1] A lot of public schools in the United States serve up mediocre results while serving well off pupils. "Does it include data on the charter school attrition rate?" This is a fair item to put on the checklist, and should be reported more in stories about schools in general. In other words, a story about public schools should also report on their attrition rate, and on what rate at which pupils move in and out of the school district, and in general how stable or unstable enrollments are at all schools being compared. (The United States has a highly geographically mobile population, and it would be a very unusual school that has the same group of pupils enrolled by sixth grade as was enrolled in first grade five years earlier.) "Does it include data on how the students who leave the charters compare to students who leave public schools?" Again, this is a question well worth asking, but we shouldn't guess, until we have seen the answer to questions like this, whether charter schools or public schools would come out better in a comparison of this kind. We should be checking the facts in each individual case all around the country. "Does it include numbers of students expelled?" Since the early 1990s, I have been aware of homeschooling advocates who talk about "push outs" (not "drop outs") from the public school system, kids who are basically told to like it or lump it in the public school system. People leaving the public school system should always be looked at closely by researchers to figure out why they left, and whether they are later able to find a more fitting school situation in which they can learn better. "Does it include numbers of students suspended?" As above, this is a question to ask about any kind of school. Moreover, at the extremes, a school that never suspends any pupil is possibly a school that is not trying hard enough to maintain a learning environment for all other pupils enrolled, so a researcher would want to look at grounds for suspension, procedures surrounding suspension, what corrective steps are taken when a pupil is suspended, and so on. I've certainly seen suspension abused in public school settings. "Does the story focus exclusively on test scores?" This is a general defect of reporting on schools. At the broad statistical level, and especially for international comparisons of school systems, test scores[2] are mostly what we have to look at. What I also look at, as someone who has lived in more than one country and who reads more than one language, is the actual item content of textbooks and the attitudes toward learning shown by teachers and pupils. I think some school systems in other countries compare very favorably to those in the United States (not least because pupils in those other countries begin foreign language study much earlier than pupils in the United States, a fact not usually reflected in international testing programs). Educational researchers should be looking more closely at the actual item content of school textbooks and at the specific teacher practices in each classroom.[3] "If so, has someone, with educational expertise, visited the school to determine if the school focuses on test prep at the expense of a rich curriculum?" As above, we should be looking in detail at what actually goes on in the classroom. Indeed, we should be looking at whether persons employed as teachers are actually teaching anything, which we shouldn't assume sight unseen.[4] "Are the test scores reported outside of school assessments such as the SAT/ACT or does the story only report test scores of exams that are proctored in-house?" This is also important, and also a question that should be applied to news stories about schools in general, not just about charter schools. So far many of the worst cheating scandals in school testing programs have originated in regular public schools.[5] "Does the story account for the fact that, due to the need to apply to the charter school, parents of the students at charters are, on average, likely to be more engaged in education than the parents of students at public schools?" The policy response to this is to make school choices much more widely available to many more families. I have seen some examples of helpful reforms where I live. Minnesota, where I now live and where I grew up, has had largely equal per-capita funding for public school pupils statewide since the 1970s. The state law change that made most school funding come from general state appropriations rather than from local property taxes was called the "Minnesota miracle."[6] Today most funding for schools is distributed by the state government on a per-pupil enrollment basis.[7] The funding reform in the 1970s was followed up by two further reforms in the 1980s. First, the former compulsory instruction statute in Minnesota was ruled unconstitutional in a court case involving a homeschooling family, and a new compulsory instruction statute explicitly allows more nonpublic school alternatives for families who seek those. Second, the Legislature, pushed by the then Governor, set up statewide open enrollment[8] and the opportunity for advanced learners to attend up to two years of college while still high school students on the state's dime.[9] And Minnesota also has the oldest charter school statute in the United States.[10] We haven't had big problems with charter school performance here. Parents in Minnesota now have more power to shop than parents in most states. That gets closer to the ideal of detecting the optimum education environment for each student (by parents observing what works for each of their differing children) and giving it to them by open-enrolling in another school district (my school district has inbound open-enrollment students from forty-one other school districts of residence) or by homeschooling, or by postsecondary study at high school age, or by exercising other choices. The educational results of Minnesota schools are well above the meager results of most United States schools, and almost competitive (but not fully competitive) with the better schools in the newly industrialized countries of east Asia and southeast Asia. It's a start. More choices would be even better. "Does it exclusively or primarily cite reports funded by pro-charter or conservative think tanks?" Here it's regrettable that we don't know the identity of the author of this checklist. Do we know whether or not the author is funded by pro-public-school interest groups? Disclosure of that would be helpful here. "Does it include quotes from academic scholars or does it just cite charter school advocates?" I can recommend a website full of the writings of actual academic scholar to Hacker News participants interested in education reform. That is the Education Next website,[11] which is better supplied than most English-language websites about education policy with information about education policy outside the United States, which is important for a reality-check on issues discussed in this country. "Does it identify advocates or simply call them 'experts' or 'researchers'?" This is a funny question for a commentator who insisted on anonymity to ask. "Does it compare the resources available to charter schools to those available to public schools?" In Minnesota, and in all states I have checked, the resources available to charter schools (funding per student, and availability of buildings) are strictly less than the resources available to public school disticts, by law. Yes, lets's check that detail carefully. It's commendable to receive a suggestion that we should check news stories carefully. We should always do that here on Hacker News. Let's check stories on education policy especially carefully, as public school spending is a huge part of state budgets[12] and a lot of interest groups are lined up behind the public trough. [1] http://educationnext.org/when-the-best-is-mediocre/ http://educationnext.org/when-the-best-is-mediocre/ [2] http://www.oecd.org/pisa/ http://www.oecd.org/pisa/ http://timssandpirls.bc.edu/ http://timssandpirls.bc.edu/ [3] http://www.amazon.com/Knowing-Teaching-Elementary-Mathematics-Understanding/dp/0415873843 http://www.amazon.com/Knowing-Teaching-Elementary-Mathematic... http://www.timssvideo.com/timss-video-study http://www.timssvideo.com/timss-video-study [4] http://www.pbs.org/newshour/bb/students-take-a-stand-to-topple-teacher-tenure/ http://www.pbs.org/newshour/bb/students-take-a-stand-to-topp... [5] http://www.ajc.com/s/news/school-test-scores/#top-stories http://www.ajc.com/s/news/school-test-scores/#top-stories http://online.wsj.com/articles/for-school-tests-measures-to-detect-cheating-proliferate-1411752291 http://online.wsj.com/articles/for-school-tests-measures-to-... http://www.newyorker.com/magazine/2014/07/21/wrong-answer http://www.newyorker.com/magazine/2014/07/21/wrong-answer http://www.washingtonpost.com/blogs/answer-sheet/wp/2014/08/11/50-ways-schools-cheat-on-high-stakes-standardized-tests/ http://www.washingtonpost.com/blogs/answer-sheet/wp/2014/08/... [6] http://www.mnhs.org/library/tips/history_topics/18public.html http://www.mnhs.org/library/tips/history_topics/18public.htm... [7] http://www.house.leg.state.mn.us/hrd/pubs/mnschfin.pdf http://www.house.leg.state.mn.us/hrd/pubs/mnschfin.pdf http://education.state.mn.us/MDE/SchSup/SchFin/index.html http://education.state.mn.us/MDE/SchSup/SchFin/index.html [8] http://education.state.mn.us/MDE/StuSuc/EnrollChoice/index.html http://education.state.mn.us/MDE/StuSuc/EnrollChoice/index.h... [9] http://education.state.mn.us/MDE/StuSuc/CollReadi/PSEO/index.html http://education.state.mn.us/MDE/StuSuc/CollReadi/PSEO/index... [10] http://www.amazon.com/Zero-Chance-Passage-Pioneering-Charter/dp/1592984762/ http://www.amazon.com/Zero-Chance-Passage-Pioneering-Charter... [11] http://educationnext.org/ http://educationnext.org/ [12] http://nces.ed.gov/fastfacts/display.asp?id=66 http://nces.ed.gov/fastfacts/display.asp?id=66 http://www2.census.gov/govs/school/12f33pub.pdf http://www2.census.gov/govs/school/12f33pub.pdf

japhyr 12y ago

Much of the conversation in this comment thread assumes a public vs. charter school split. If my understanding is correct, these charter schools are public schools. The distinction is that the charter schools have selection criteria. A public school with selection criteria is not, by itself, a bad thing. I love that schools like Stuyvesant HS and Bronx School of Science exist, but that's because we all know what their selection criteria are. I don't like charter schools pretending that the only thing it takes to be part of the school is winnning an entrance lottery. That might be the main way to get in, but it's not all it takes to stay enrolled. Many charter school advocates seem to be ignoring, or ignorant of, the fact that these schools actively weed out students who are difficult to reach. It's okay to build some schools that require a more serious commitment to learning; it's not okay to build schools like this and pretend they serve everyone, or do a better job than schools who have to keep every eligible student enrolled.

tokenadult 12y ago
Charter schools by law do not have selection criteria. The example you mention, the Bronx High School of science, is a public school with an entrance examination, usually called an "exam school" for short. No charter school anywhere in the country can operate on that basis, as all charter school statutes everywhere explicitly ban examination requirements for entrance.
- japhyr 12y ago
  Thanks for the clarification, I'm glad I qualified that comment!

snake_plissken 12y ago

>Reports and stories about charter schools are in the media every day. The majority of these stories praise charters, while often demeaning public schools. Interesting. My experience with how the media presents charter schools is quite the opposite. It seems like every few weeks there is a story about a corrupt charter school management team embezzling funds or over-enrolling of students or suspect academic results. Of course with stories on actual events, such as an investigation into missing funds, it's difficult if not downright impossible, if a media outlet so desired, to spin the story into something positive. But I don't recollect reading anything within the past few years that actually praised charter schools, and that's not to mean the traditional public school system is squeaky clean and a model of best practices. I suspect there is an equivalent level of corruption there. I've wondered if it's because I live in Philadelphia where this a strong teachers union and a traditional public school system that is, for all intents and purposes, bankrupt. However, I would expect that these conditions would lead to the opposite in how charter schools are portrayed, if only for the sake that they are an alternative to the current system that is failing miserably. The charter school debate is a sub-segment of a larger issue, which is this push towards the privatization of everything that is consumed by the public and paid for by tax dollars, from public school systems to prisons to health care programs, such as Medicaid. Personally, I disagree with this push. It creates perverse incentives and from a political economy point of view, I think there are some goods which can't and should not be privatized, but that discussion is for another time.

mbubb 12y ago

This is really interesting - to be honest I am at work and am resisting being pulled into reading this until tonight... I have a good friend who I have had since college days. Has been an NYC public school teacher for a while. Loves to teach - onel of those folks that is constantly formulating ideas, reads voraciously, argues passionately, etc. I see him as caught between the attempt to corporatize education and a bloated, syphillic (sp) union... My ideas not his. On my side I had kids in public school until 2nd grade where I saw disturbing things that made me jump ship to a charter where my boys thrive. This is anecdotal - I do not want to try and prove/ disprove anything by this. I wish we had a viable pub school option locally but we don't at the moment. Our school survived a scummy attack on charters locally in which false claims were made about segregation. This was disproven but we were made aware of the fact that the other side has more resources and deeper legal pockets then the charter school does. As I said I will sit down and read this later. I am interested in something that comes up when my buddy and I talk about this stuff. Somewhere I picked up a concept from French which I will anglicize as "Deformation Professionelle" (my apologies for butchering it - I will properly google it later). ed. http://en.wikipedia.org/wiki/D%C3%A9formation_professionnelle http://en.wikipedia.org/wiki/D%C3%A9formation_professionnell... I see this when tech and teachers discuss education. I sit and talk to him about tech issues - 'net neutrality', privacy issue, (going back a decade) attacks on open source software, etc. And he doesn't know about these things. And I (at times) wax enthusiastic about things like Khan Academy or charter schools without seeing possible drawbacks. I look forward to giving this a proper read later - am glad to see this on HN.