5 ms·

Measuring (and funding) schools based on student outcome is fraught because a student's performance / preparedness for the "next level" is not entirely a functi

Measuring (and funding) schools based on student outcome is fraught because a student's performance / preparedness for the "next level" is not entirely a function of the school. There are other significant parameters, including parental upbringing, home life stability, neighborhood safety, friends, hunger/nutrition, various trauma and abuse, the list goes on. I'm sure it's been studied, but I'd bet "school quality" is not even close to number 1 on the list of predictors of educational outcome.

hedora 5mo ago
This is true. There are safeguards (that are currently failing) that my program would engage: - The state is legally required to provide those kids with an education. - There is funding allocated to help those districts. If "we will not pay you if the kids do not learn" means there are zero schools in those districts then (1) the state government will get sued for not doing its job (because closing 100% of the schools makes the failure objective and obvious) and (2) it will have to update those funding formulas so that it is possible for some school (state run, or private) to break even while providing an education in those areas.
- gausswho 5mo ago
  With sympathy to your appeal that 100% closures will force us to reckon with the problem, I suspect it'd only lead to missing the forest for the trees. This would come with substantial pains to the community. Potentially ones that knock-on to other pains. You're at the root of why this is a tricky problem to solve. In fact there is no solution, just a wide basket of expensive things we should aspire to do to improve affairs.
  - hedora 5mo ago
    What pain, exactly? - The local public school goes from 80 kids per grade to 40, and a new school opens across the street or just rents an existing building from the existing school district. - Funding stays flat, and academic performance goes up. - Administrators get to decide which teachers to lay off, and they will be de facto fired if they get rid of the high performers while keeping the low performers. - If the union contracts make it impossible to retain the high-performers, then the school eventually shuts down, and teachers that are competitive on the job market get hired by the new school for similar pay / benefits. - Teachers at the new school get evaluated on whether they do their job, and the new administrators have a strong financial incentive to use performance-based evaluation instead of seniority / nepotism / whatever. I see no downside whatsoever.
    - gausswho 5mo ago
      The pains I was thinking of largely occupy the transitionary period of a school closing before alternatives are open. When does the deficient school close? After this new school is opened? If not, what happens to students and families that depend on an education in the interim? Who pays for this new school? Must they immediately show improvement or do they get some years to show that their approach is working better? Will the metrics even be accurate in the new school? Will there be a self-selecting bias in the newly formed student body?
      - hedora 5mo ago
        I don't think these details are particularly hard to work out: - You can shrink the deficient school to zero by reducing teacher count starting in the lower grades and moving up, and by allowing parents to opt for transfers in higher grades. - The building still exists, so you could reuse it. Or, investors could build a new school. Obviously, there's some lag in the measurement, since it requires a few years of student data. I'd say look at the first and second derivative of the test scores. Note that the claw-back model deeply screws over investors that fund substandard schools. This is likely to create stranded real-estate for the next round of investors to buy at a discount. - The metrics are produced downstream, so there shouldn't be measurement bias. There probably will be self-selection bias. There are existing funding mechanisms to deal with challenging student bodies. If those are working, then the per-student funding of the old school with skyrocket. If the old school still fails, then that produces a high-revenue group of students for some other new school to take on. If those funding mechanisms are not working, then it creates an externally detectable signal to the outside world that the problem is one level up (no schools in certain areas), making it easy for voters / courts to intervene (currently, those funding mechanisms are failing, and no one is held accountable).
        convolvatron 5mo ago
        investors? you're going to raise the cost of primary education to accommodate enough of a margin to attract investors? I thought we were talking about public education so that people in our society can at least read - a task that we're doing pretty bad at. The private school system for the 1% is doing just fine already.
M3L0NM4N 5mo ago
The number 1 predictor of educational outcome is IQ by a long shot, which is hardly affected by any of the factors you listed. Yes, high IQ kids usually have high IQ parents who are likely to prevent those things, partly because they are likely high income, but none of those are as important as how smart the child is.
- HelloMcFly 5mo ago
  [flagged]
- thewebguyd 5mo ago
  The heritability of IQ actually changes based on wealth, so its the other way around. A child from a wealthy family will reach their potential, where one from a poorer family will not. (https://pubmed.ncbi.nlm.nih.gov/14629696/ https://pubmed.ncbi.nlm.nih.gov/14629696/) A child may have the genetic potential but never reach their potential because of outside factors. One's environment shapes one's brain development. That's why equity is just as important as equality in education. Equity is understanding that children start from different circumstances and may need specific support to actually reach their potential. Although the biggest factor here would just be for society to make sure no child has an upbringing where food, shelter, other lack of resources are a problem.
  - scarmig 5mo ago
    That mistakes the point of education. Schools do not exist to fix every social problem, and demanding they treat fixing every social problem as their number one priority is how we got into this mess of "teach nothing but make sure everyone passes" in the first place.
    - hedora 5mo ago
      Yes, but back when California was poorer, it had some of the best schools in the nation. Now that it's richer, the schools are collapsing, so it's really hard to argue that systematic social problems are the root cause.
jazzkingrt 5mo ago
I have many concerns with this kind of funding model, but I don't think the measurement problem is so serious. Performance incentives in education typically reward improvement of the student cohort relative to how it was performing the previous year, or even use value-added models that use multiple past years to predict the student trajectory.
amluto 5mo ago
It’s also fraught because schools will spend increasingly large fractions of the time preparing kids for tests instead of teaching them anything.
- pelagicAustral 5mo ago
  Wasn't this the plot of the Wire season 3 or something?
  - Novosell 5mo ago
    It is one part of the plot that focuses on inner city schooling in season 4 :)
- SpicyLemonZest 5mo ago
  Doesn't this whole story suggest that the aversion to "preparing kids for tests" was wrong? The UC system changed its admissions policies to help kids who weren't prepared for tests, and now they have a bunch of students who don't seem to have been taught anything despite their high grades.

hedora 5mo ago

This is true. There are safeguards (that are currently failing) that my program would engage: - The state is legally required to provide those kids with an education. - There is funding allocated to help those districts. If "we will not pay you if the kids do not learn" means there are zero schools in those districts then (1) the state government will get sued for not doing its job (because closing 100% of the schools makes the failure objective and obvious) and (2) it will have to update those funding formulas so that it is possible for some school (state run, or private) to break even while providing an education in those areas.

gausswho 5mo ago
With sympathy to your appeal that 100% closures will force us to reckon with the problem, I suspect it'd only lead to missing the forest for the trees. This would come with substantial pains to the community. Potentially ones that knock-on to other pains. You're at the root of why this is a tricky problem to solve. In fact there is no solution, just a wide basket of expensive things we should aspire to do to improve affairs.
- hedora 5mo ago
  What pain, exactly? - The local public school goes from 80 kids per grade to 40, and a new school opens across the street or just rents an existing building from the existing school district. - Funding stays flat, and academic performance goes up. - Administrators get to decide which teachers to lay off, and they will be de facto fired if they get rid of the high performers while keeping the low performers. - If the union contracts make it impossible to retain the high-performers, then the school eventually shuts down, and teachers that are competitive on the job market get hired by the new school for similar pay / benefits. - Teachers at the new school get evaluated on whether they do their job, and the new administrators have a strong financial incentive to use performance-based evaluation instead of seniority / nepotism / whatever. I see no downside whatsoever.
  - gausswho 5mo ago
    The pains I was thinking of largely occupy the transitionary period of a school closing before alternatives are open. When does the deficient school close? After this new school is opened? If not, what happens to students and families that depend on an education in the interim? Who pays for this new school? Must they immediately show improvement or do they get some years to show that their approach is working better? Will the metrics even be accurate in the new school? Will there be a self-selecting bias in the newly formed student body?
    - hedora 5mo ago
      I don't think these details are particularly hard to work out: - You can shrink the deficient school to zero by reducing teacher count starting in the lower grades and moving up, and by allowing parents to opt for transfers in higher grades. - The building still exists, so you could reuse it. Or, investors could build a new school. Obviously, there's some lag in the measurement, since it requires a few years of student data. I'd say look at the first and second derivative of the test scores. Note that the claw-back model deeply screws over investors that fund substandard schools. This is likely to create stranded real-estate for the next round of investors to buy at a discount. - The metrics are produced downstream, so there shouldn't be measurement bias. There probably will be self-selection bias. There are existing funding mechanisms to deal with challenging student bodies. If those are working, then the per-student funding of the old school with skyrocket. If the old school still fails, then that produces a high-revenue group of students for some other new school to take on. If those funding mechanisms are not working, then it creates an externally detectable signal to the outside world that the problem is one level up (no schools in certain areas), making it easy for voters / courts to intervene (currently, those funding mechanisms are failing, and no one is held accountable).
      - convolvatron 5mo ago
        investors? you're going to raise the cost of primary education to accommodate enough of a margin to attract investors? I thought we were talking about public education so that people in our society can at least read - a task that we're doing pretty bad at. The private school system for the 1% is doing just fine already.

M3L0NM4N 5mo ago

The number 1 predictor of educational outcome is IQ by a long shot, which is hardly affected by any of the factors you listed. Yes, high IQ kids usually have high IQ parents who are likely to prevent those things, partly because they are likely high income, but none of those are as important as how smart the child is.

HelloMcFly 5mo ago
[flagged]
thewebguyd 5mo ago
The heritability of IQ actually changes based on wealth, so its the other way around. A child from a wealthy family will reach their potential, where one from a poorer family will not. (https://pubmed.ncbi.nlm.nih.gov/14629696/ https://pubmed.ncbi.nlm.nih.gov/14629696/) A child may have the genetic potential but never reach their potential because of outside factors. One's environment shapes one's brain development. That's why equity is just as important as equality in education. Equity is understanding that children start from different circumstances and may need specific support to actually reach their potential. Although the biggest factor here would just be for society to make sure no child has an upbringing where food, shelter, other lack of resources are a problem.
- scarmig 5mo ago
  That mistakes the point of education. Schools do not exist to fix every social problem, and demanding they treat fixing every social problem as their number one priority is how we got into this mess of "teach nothing but make sure everyone passes" in the first place.
  - hedora 5mo ago
    Yes, but back when California was poorer, it had some of the best schools in the nation. Now that it's richer, the schools are collapsing, so it's really hard to argue that systematic social problems are the root cause.

jazzkingrt 5mo ago

I have many concerns with this kind of funding model, but I don't think the measurement problem is so serious. Performance incentives in education typically reward improvement of the student cohort relative to how it was performing the previous year, or even use value-added models that use multiple past years to predict the student trajectory.

amluto 5mo ago

It’s also fraught because schools will spend increasingly large fractions of the time preparing kids for tests instead of teaching them anything.

pelagicAustral 5mo ago
Wasn't this the plot of the Wire season 3 or something?
- Novosell 5mo ago
  It is one part of the plot that focuses on inner city schooling in season 4 :)
SpicyLemonZest 5mo ago
Doesn't this whole story suggest that the aversion to "preparing kids for tests" was wrong? The UC system changed its admissions policies to help kids who weren't prepared for tests, and now they have a bunch of students who don't seem to have been taught anything despite their high grades.