9 ms·

What if we tested laws before passing them?

150 points by robg 16y ago

messel 16y ago
Why stop at new laws? Existing laws should be relentlessly reviewed for relevance. One issue I see with random sample law experimentation is blindness to effects which only crop up when everyone is subject to a law. Take for example gun control. If a small sample of folks aren't allowed to have guns there may be little negative fall out. But taken to the extreme if gun ownership is illegal than by definition only law breakers would have guns making the random burglary or home invasion far less dangerous for criminals and far more dangerous for law abiding citizens.
- randallsquared 16y ago
  I'm not sure it's clear that carrying a gun makes crime safer for the average criminal, though I think it is clear that armed homeowners make crime more dangerous for criminals. My impression is that when guns are hard to get, crime goes up, but the number of criminals carrying goes down.
- Zak 16y ago
  Gun control is a case where there's probably enough data to do interesting analysis if someone were to collect it and study it without trying to make it show a specific result. Noteworthy data points would be the first derivatives of gun crime, non-gun violent crime and burglary rates before and after specific gun control measures were passed or repealed.
  - Retric 16y ago
    There is a fair amount of evidence that gun owners are less safe than non gun owners, and societies with higher levels of gun ownership tend to be less safe overall. But, changing gun laws don't alter the prevalence of gun ownership vary quickly so it's hard to such analysis with idential populations.
    - Zak 16y ago
      There's also the fact that correlation does not imply causation. The possibility of a common cause is fairly obvious here: people who are at higher risk of being the targets of crime may be more likely to buy guns for protection, and people who are armed may be more likely to become involved if they see someone else being attacked.
      - Retric 16y ago
        Yep, while hiding and calling the cops is generally safer during a robbery, the attitude that causes people to defend their property is the same one that results in gun ownership. Add in people being shot with their own weapons and the home protection benefit becomes very complex. There is an idea that gun ownership deters criminals but at the same time it promotes criminals buying guns escalating the situation. PS: For a somewhat less contentious example, video games have been linked with a significant drop in crime. However, because that does not agree with their biases people focus on the direct effects to support their argument.
        Zak 16y ago
        I think you'll find there tends to be a correlation between using [protection against bad thing] and being at risk for [bad thing]. As an example, I'm fairly certain you'll find a correlation between wearing a helmet while driving a car and being injured in car crashes because most people who wear helmets while driving are racing or doing stunts. I also suspect people with burglar alarms lose more money to burglary than people without if no other variables are taken in to account. I don't think an unbiased study has been done that accurately identifies causes looking at gun ownership and crime. My suspicion is that it would be possible to do, as I described above.
        yummyfajitas 16y ago
        Add in people being shot with their own weapons... This is a pretty minor effect, unless you are including suicides. In 2007, there were only 613 accidental firearm deaths. For comparison, there were 1039 "transport, other" deaths (I believe this means plane/boat crashes).
  - yummyfajitas 16y ago
    At least one study purports to do exactly this. http://www.amazon.com/More-Guns-Less-Crime-Understanding/dp/0226493636 http://www.amazon.com/More-Guns-Less-Crime-Understanding/dp/... (I haven't read it myself, so I'm not endorsing it. It's not exactly uncontroversial.)
    - gjm11 16y ago
      In particular, its author got caught promoting his work with sockpuppets. See http://en.wikipedia.org/wiki/John_Lott http://en.wikipedia.org/wiki/John_Lott and (for a rather less staid take on the affair) http://www.whoismaryrosh.com/ http://www.whoismaryrosh.com/ .
      - yummyfajitas 16y ago
        Yeah, it looks like the author is a dickwad with a big ego. I don't plan to invite him to any parties. However, the scientific criticisms of his methodology hardly look damning (according to wikipedia). It doesn't look perfect, but it doesn't look terrible either. The criticisms wikipedia lists: he used cocaine prices rather than cocaine consumption as a variable, if you remove smaller counties +florida the drop in murders becomes statistically insignificant (drops in assault and robbery are still significant), and a couple of critics believe the model has bias and coding errors. As I said, I haven't read it in detail, so I won't comment on how significant these are.
- amelim 16y ago
  This is the reason why jury nullification exists. It's just a shame that most citizens have no idea that it either exists, or is a valid choice when serving on a jury.
  - Zaak 16y ago
    A problem with jury nullification is that the interactions between laws is complex and can easily have unintended consequences. I am overall in favor of jury nullification, but it requires an educated jury. The complexity of our system of laws makes that very difficult.
- G_Wen 16y ago
  An interesting issue. One way around it would be to enact the law in two states that have a similar environments. Continuing the example with gun control if there were two states with similar situations, both of them have an abnormally high gun crime rate, one of them could serve as a control state. If say 10 pairs of states were compared it would serve as a reasonable experiment on whether the law should be passed nation wide or not. Even better if evidence comes up that passing the law would only be beneficial in certain circumstances then it would help reveal the underlying causes of the problem in the first place.
  - messel 16y ago
    Sharp response G_wen. Appreciate the way you overcame the random sample issue with distributed geo-sampling.
- adolph 16y ago
  Texas operates along a similar proposition. It is called the Sunset Advisory Commission and applies to policies, regulations and agencies. http://www.sunset.state.tx.us/ http://www.sunset.state.tx.us/
pierrefar 16y ago
Surely this is a prime candidate for the Hawthorne effect? From Wikipedia: http://en.wikipedia.org/wiki/Hawthorne_effect http://en.wikipedia.org/wiki/Hawthorne_effect The Hawthorne effect is a form of reactivity whereby subjects improve or modify an aspect of their behavior being experimentally measured simply in response to the fact that they are being studied, not in response to any particular experimental manipulation.
- metageek 16y ago
  From the article: >But in other cases, participating knowingly in an experiment could distort the result (this phenomenon even has a name, the Hawthorne effect).
btilly 16y ago
It is an interesting idea. However I'd be concerned that the law would work differently with a subset than the whole population. For instance take a law banning cellphones while driving. If you try to apply this to half the population, then the police aren't going to try to apply it because at least half the time they would be pulling over people who are allowed to use cellphones. But if you pass the law for all of the people, then the police could enforce it. So you try to solve this by saying that it will be selectively enforced according to location. But now you've gotten rid of the random selection that is the heart of evaluating the statistics. For another example, look at the tax cut example they were discussing. The challenge here is that the touted economic benefit of tax cutting is that people with lower taxes spend their money, increasing circulation, and general prosperity. (And eventually improving tax revenue.) However there is no way to economically separate out the people you're giving the cut to from those you aren't. Therefore any economic change cannot be attributed to the tax cut, and you're unlikely to get a clear economic difference. (Other than that the less taxed have more disposable income.)
- Symmetry 16y ago
  You can't see the change in the overall economy, but you can see if the people who you gave the tax cut to are investing or spending more.
  - btilly 16y ago
    We already have that data. The debated question is how many times said money continues to circulate.
- alexandros 16y ago
  For your seatbelt experiment, you can always separate the population based on license plates. There, randomness restored.
  - tokenadult 16y ago
    That depends on whether or not license plate numbers are assigned by county, or by driver last name, or some other nonrandom criterion. And in some states, significant numbers of drivers choose their own customized license plate numbers.
tomjen3 16y ago
this sounds smart until you realize that would mean giving up the principle that the law treats everybody the same, which we always complain about the politicians doing.
- ugh 16y ago
  I see this as a possible problem that would have to be considered when such an idea is implemented but not something that stops the idea dead in its tracks because it is fundamentally opposed to the principle of equality. You would have to make sure that the tests are truly random (everyone has an equal chance of being selected), that they run only for a limited and pre-determined time which cannot be extended and that the differences to existing laws is not too big (e.g. people shouldn’t suddenly become multi-millionaires because some law was tested on them). Such tests would then no more or less contribute to inequality than, say, jury duty.
ph0rque 16y ago
I was hoping the proposal would be testing laws on a virtual US, something commented on before: http://news.ycombinator.com/item?id=1724778 http://news.ycombinator.com/item?id=1724778
- Benjo 16y ago
  How do you know we're not just part of a simulation to determine the outcome of Bush v. Gore?
Goladus 16y ago
I don't think you could test taxation, for a number of reasons. First, you don't have time to measure the effects of funding changes. How do people behave with the new services funded by the taxes, when you aren't collecting enough to provide them anyway? Does the government go bankrupt? No, because revenue was not changed significantly due to the small size of the test. Second, sample populations are unlikely to exhibit the same behavioral traits of large-scale populations. For example, if you applied a tax increase to a limited segment of the population the first behavior change you would see is that people would try to escape the test group. For something like a sales tax this is even more obvious. The overall rate of taxation matters much less than the rate relative to a competitor. It's a free competitive advantage and of course behavior will change-- the untaxed businesses will be more likely to thrive and grow while driving their competitors bankrupt. When everyone has the same tax rate they have to find real market advantages. The whole point of taxes is that they are applied universally as much as possible. Everyone contributes a little bit so that the society can have things that benefit everyone but the market is unlikely to ever provide. NPR just discussed autopsies as an example of this. Understanding why people die is good for the public, yet only the most altruistic individuals are inclined to pay for an autopsy of their own loved ones. While I suppose it is possible to design test runs of tax laws, I doubt it will ever be as simple as picking a "representative sample of 10,000." There are probably only certain kinds of laws that can be tested, and tax laws probably aren't among them.
CWuestefeld 16y ago
What is the purpose of passing a law? I submit that its primary purpose may not be to bring about a change in the behavior of those subject to it, but rather, as a signaling mechanism by which a politician can say to those in his party, or his constituents, "I'm playing on your team". We see time and time again regulation that is known to be worthless or even destructive even before it's implemented (e.g., CAN-SPAM, airport porn scanners), yet bureaucrats stand behind them, and even trumpet their "success", just the same. Recent threads here have discussed whether voting behavior is really based on weighing issues, as opposed to signaling one's identity as a member of the "club" (party, special interest, etc.). If this is true, then it's only logical that the politician's job is to make it know that he is the representative of that "club", and actual goodness of his work isn't particularly relevant.
- anamax 16y ago
  > I submit that its primary purpose may not be to bring about a change in the behavior of those subject to it, but rather, as a signaling mechanism by which a politician can say to those in his party, or his constituents, "I'm playing on your team". Another important purpose is encouraging support. For example, one of the best ways to get bankers to contribute to your campaign is to propose banking regulations.
- DanielBMarkham 16y ago
  It would follow that the ultimate law would be one that sent a strong signal to your base and had very few people who could appear on TV and say they were directly harmed by it. For some reason the tax code comes immediately to mind.
jokermatt999 16y ago
Hmm..Hacker News' love of A/B testing also applies to laws, apparently. Also, isn't one of the arguments for stronger state laws and weaker federal laws very similar to this proposal? That is, if you give states greater autonomy, it's easy to test the effects of laws and give citizens choice over which laws they'd like.
DougWebb 16y ago
Or, we could pay attention to the 10th Ammendment: "The powers not delegated to the United States by the Constitution, nor prohibited by it to the States, are reserved to the States respectively, or to the people." Instead of randomly applying variations of a law, don't federalize it. Let the states pass their own variations, and see how that works out. Sure there are other variables at play, but they mostly revolve around how suitable the law is for the people it applies to, and that's a legitimate reason for the law to vary and part of the purpose of the 10th Ammendment: local self-governance. I believe most federal laws should be repealed and replaced with guidelines for the states to follow when designing their own laws. These would be "Best Practices" for governments. Most of the world treated the US Constituion this way as they wrote their own constitutions, and I think that worked pretty well. It should work for State governments too.
- DanielBMarkham 16y ago
  That won't work. Somehow we've gotten into Lord of the Rings Mode -- "one ring to rule them all" We have effectively turned a distributed system into a monolithic piece of spaghetti. And now folks are suggesting that what we need -- wait for it -- is more testing. We're effectively having all the same conversations large organizations have when creating billion-dollar POS centralized software development systems. Should we focus on requirements (listen to the voters). Should we be more clear on the impact of the change (environmental, financial, etc)? Maybe we just need some super-smart people put in the right spots. That'll fix it. Many times I wonder: are people doomed to repeat the same mistakes over and over again just in slightly different settings? (sigh)
  - jhancock 16y ago
    "are people doomed to repeat the same mistakes over and over again" This is a common view in mainland China. Particularly, the last few years, in regards to the U.S.: "They've had a good 200-250 year run, now this arc/dynasty is coming to its end and another arc will replace it. Hopefully the turmoil while bootstrapping the next won't be too painful." 200-300 years is the generally accepted time of a Chinese dynasty. As near I as can tell, this is due to people "forgetting" their history. Written history may not be enough. You need to be able to feel it similar to those that put their lives on the line to create it.
tibbon 16y ago
YES! We should approach this from a two-tiered testing platform. The first tier is rather like how you'd do it in Rails or any other TDD environment. We write a list of tests that we know should pass as true at the end. Let's say, start with the US and State constitutions as the basis. def must_not_prohibit_exercise_of_religion() { People.can_exercise_religion? } We then write in the test laws and see what side effects occur. When those side effects occur, we must make note of them and consider the potential exceptions. def freedom_of_speech() { unless(crowded_theatre && People.speak == fire) People.can_speak? end } As a second layer then we must test it on smaller populations to determine the effects. This has worked wonderfully in some situations. "Gay Marriage will destroy marriage and society" is a claim I've heard prior. So test it in Massachusetts. Several years later, no marriages on record have been destroyed due to gay marriage, and the metrics of society's stability have not moved significantly in any direction that we can attribute to same-sex marriage. So it (should) pass and become law nationally because we've observed its effects. We can observe similar for the long term effects of required health insurance in Massachusetts as we debate how things should work out nationally. Just as we have 'human-readable code' why shouldn't our laws be machine readable? Computers can remember/understand 200+ years of case law probably better than humans. Again, this is something that we could test. One of my friends is working on a law firm run by computers (http://www.robotandhwang.com/ http://www.robotandhwang.com/). Of course the machines can't show up in court or legally offer advice, but they can advise the attorney. (Law.com and ABA Journal have cited him doing this, so maybe it isn't as crazy as it sounds).
- jgfoot 16y ago
  I was about to respond until I realized your post was a Swift-ian parody so sophisticated as to be almost over my head. Well done. But if we were going to try to reduce the complicated, inherently human, and uncertain concept of law to computer code, surely you'd agree that a declarative programming language would be a better choice. Something like: prohibits_free_exercise_of_religion(X) :- unconstitutional(X). respects_establishment_of_religion(X) :- unconstitutional(X). unconstitutional(X) :- not_law(X). Now, all you have to do is write Prolog code to test for when a law prohibits the exercise of religion or respects the establishment of religion!
stuaxo 16y ago
I've been talking to people about exactly this sort of thing, I'd make the trial groups bigger though.
hugh3 16y ago
Note how the example is lower taxes: why don't we give lower taxes to a hundred thousand people and see what they do? But now suppose the proposal is higher taxes for a hundred thousand people. How do you think the hundred thousand people in the test group are going to react? Not too well, and probably not nearly the same way as if you raised taxes on everybody. Isn't this one of the supposed advantages of having states, though? So that different laws can be tried out in different places under the same general societal conditions and we can learn what works and what doesn't work? And yet doesn't the Federal Government keep trying to subsume more state powers into itself?
- nickpinkston 16y ago
  I'm guessing behavioral economists would be lapping this one up if they were given the cash to do so, but then we'd actually have some empirical proof about policy - politicians prefer to just rant for votes - far cheaper and easier to manipulate... Yea, power always concentrates up - seemingly because coordination costs get cheaper as the pool of people decreases in size.
  - jbooth 16y ago
    Well, for the record, we have tried the whole supply-side economics thing in the real world twice now, with less-than-stellar results.. I mean, there's no control group but you'd need an alternate universe for that. EDIT: sigh. Downvote away for not liking what I said, but you can't even disagree with it, it's factual. That particular idea has been tried twice in the real world.
    - CWuestefeld 16y ago
      It's been tried much more than twice. And it's been done with success. For example, Kennedy in 1964 was very successful. See http://www.heritage.org/Research/Reports/2003/08/The-Historical-Lessons-of-Lower-Tax-Rates http://www.heritage.org/Research/Reports/2003/08/The-Histori... And also http://www.house.gov/jec/fiscal/tx-grwth/reagtxct/reagtxct.htm http://www.house.gov/jec/fiscal/tx-grwth/reagtxct/reagtxct.h... : High marginal tax rates discourage work effort, saving, and investment, and promote tax avoidance and tax evasion. A reduction in high marginal tax rates would boost long term economic growth, and reduce the attractiveness of tax shelters and other forms of tax avoidance. The economic benefits of ERTA were summarized by President Clinton's Council of Economic Advisers in 1994: "It is undeniable that the sharp reduction in taxes in the early 1980s was a strong impetus to economic growth." Unfortunately, the Council could not bring itself to acknowledge the counterproductive effects high marginal tax rates can have upon taxpayer behavior and tax avoidance activities. Since 1984 the JEC has provided factual information about the impact of the tax cuts of the 1980s. For example, for many years the JEC has published IRS data on federal tax payments of the top 1 percent, top 5 percent, top 10 percent, and other taxpayers. These data show that after the high marginal tax rates of 1981 were cut, tax payments and the share of the tax burden borne by the top 1 percent climbed sharply. For example, in 1981 the top 1 percent paid 17.6 percent of all personal income taxes, but by 1988 their share had jumped to 27.5 percent, a 10 percentage point increase. The graph below illustrates changes in the tax burden during this period. Indeed, I'd like to see your evidence that it's bad. That evidence should be clear, in that it shows that the problem is not actually government over-spending or other issues.
albemuth 16y ago
I thought that's what California was all about
qjz 16y ago
I'd be happy if they developed a protocol for testing the constitutionality of a law. That would have to be part of any experiment, since the effectiveness of a law is irrelevant if it's unconstitutional.
wazoox 16y ago
Problem : most people aren't interested in reality and truth. See the the "teach creationism" debate. It would be great, however.
gibsonf1 16y ago
The idea that raising taxes increases revenue has been shown to be untrue with actual taxation data time and again. A good resource for cause and effect with taxation is "Hauser's Law": http://en.wikipedia.org/wiki/Hausers_Law http://en.wikipedia.org/wiki/Hausers_Law
JimboOmega 16y ago
I've advocated this for a while in a somewhat obvious field, traffic law. However, there's a flaw; science in the political arena is subject to interpretation. In VA, we've gone through this with red light cameras. For a while they were authorized at a few intersections to see if they reduced accidents. They didn't (http://www.thenewspaper.com/news/18/1844.asp http://www.thenewspaper.com/news/18/1844.asp). But a few years later, red light cameras are back, the whole study thing having been an annoying thorn in the side of politicians, who like red light cameras (because they are perceived as good for safety, for money, whatever - who knows?) So an article comes out like this: http://www.washingtonpost.com/wp-dyn/content/article/2009/02/18/AR2009021801108.html http://www.washingtonpost.com/wp-dyn/content/article/2009/02... which contradicts the link I posted before. How can the same study come to opposite conclusions? That's politics. In any case, the politicians decide what they want to do, and then change the science to suit. After all, can you really imagine a politician saying "I supported X fully, until the results of the study came out, and clearly, X is not effective, so I no longer support X?" Only if the study is poll numbers, I imagine.
Aaronontheweb 16y ago
We have a system for doing this - it's called Federalism.
- jimmyk 16y ago
  Unfortunately, the federal aspect of the US government has almost disappeared as a result of the 16th amendment and the 17th amendment and a few other factors such as mandates that the states must follow in order to receive federal funding. The US government is now plainly a national one.
brudgers 16y ago
IANAL, but I don't think this meets the Equal Protection Clause of the 14th Amendment.
vhackish 16y ago
This would be a great way to test out new jails too - when you break a test law, straight to the test jail with you!
jessriedel 16y ago
The RAND study is famous for trying this on a pretty large scale with health insurance: http://en.wikipedia.org/wiki/RAND_Health_Insurance_Experiment http://en.wikipedia.org/wiki/RAND_Health_Insurance_Experimen... Robin Hanson has been a big proponent of resuming similar experiments: http://www.overcomingbias.com/2007/05/rand_health_ins.html http://www.overcomingbias.com/2007/05/rand_health_ins.html http://www.overcomingbias.com/2007/05/rand_health_ins_1.html http://www.overcomingbias.com/2007/05/rand_health_ins_1.html http://www.overcomingbias.com/2007/05/rand_experiment.html http://www.overcomingbias.com/2007/05/rand_experiment.html Worth reading even if you're of a different political persuasion.
ChristianMarks 16y ago
Effective methods to elevate political and social discourse above the level of ideological debate would be revolutionary. Empirical testing of proposed legislation should be implemented as soon as possible. In controversial cases where experimentation doesn't lead to satisfactory outcomes, thanks to ideological intransigence, the method should be supplemented with a proposal that I call Bimodal Politics. Technology could be used to manage controversial political issues, for which the distribution of voters is bimodal and for which there is essentially no middle ground. Such issues include abortion rights, stem cell research and gay marriage. Through the system of bimodal politics, voters would live in parallel legal and political worlds, with different rights and obligations to the state depending on how they voted. Where controversial legal and political matters are concerned, the distribution of opinion involves roughly equal and opposite numbers of equally-informed, impartial rational voters. (Some would contend that for issues such as abortion, a few vociferous opponents give the misleading impression of a much larger opposition, but until this is established, we will proceed as if public discourse reflects a genuine controversy.) Although the voters on each side of a controversy may not see the other side as rational or equally informed, one essential feature of controversies is that they cannot be decided by common morality, and they have to be transferred to the legal and political system for resolution. The legal and political system can attempt to resolve controversial issues one way or the other, but it is understood that the resolution is provisional and does not resolve any underlying moral issues. Deciding controversial issues one way or the other is inherently unstable. What if it were possible to decide issues in parallel, in such a way that each group in a controversial issue imposes rights and obligations only upon itself and no group imposes its vote on the members of the other group? Is there a procedure that would result in less political and social instability than deciding one way or the other for all voters? Bimodal politics attempts to provide a legal and political mechanism for resolving politically controversial issues, with the understanding that it provides a political decision procedure: it does not address the underlying moral issues. In outline, a database is maintained of voter preference on controversial issues that are designated bimodal issues. Your vote is recorded by the bimodal voter database. Your vote determines your rights and obligations to the state on that particular issue in parallel with those voters who voted oppositely, and who may have (and probably have) different rights and obligations under the state. Consider stem cell research. Under the proposed system, stem cell research would be designated a bimodal issue. During an election, your vote on stem cell research would be entered into the database. If you voted in favor of stem cell research, you may be taxed to support it, your embryos may be harvested for stem cells (these may be from embryos slated for destruction in any case) and if you develop a disease that requires stem cell derived therapy for its treatment, you will be eligible for it. If you voted against stem cell research, you will not be taxed to support it, your embryos will not be harvested for stem cells and if you develop a disease you will be prohibited from pursuing treatments derived from stem cell research. Abortion is another issue that would be designated a bimodal issue under the system of bimodal politics. If you vote against abortion, your tax dollars will not be used to support state-sponsored family planning programs or sex education, and you will be legally barred from having an abortion if you are female. If you are male and you impregnate a woman who has an abortion, and you voted against abortion, you will be held legally liable. In either case, if you voted against abortion and your fetus or your partner’s fetus is aborted, you will be prosecuted by the state. However, if you voted in favor of abortion, your fetus can be aborted, and your tax dollars may go to support state-sponsored family planning programs and sex education. These examples illustrate the slogan that under bimodal politics, you live in the world you voted for.
rwhitman 16y ago
I firmly believe that the next major system of government to emerge in the future will be heavily influenced from the lessons learned in governing social media communities. The idea of A/B testing laws seems like it would fit into this pretty well.
Natsu 16y ago
Maybe we could also put the laws into source control, be able to roll back to a "known good" state and have a "blame" command for who introduced changes that screwed everything up. Unfortunately, politics is largely in the realm of emotion. And people hate being proven wrong.
giardini 16y ago
At least we could do A/B testing with them! Isn't that what states are for?
gills 16y ago
Experimenting on humans with a system backed by the use of force would be despicable. What happened to basic respect for each other? <snark> Anyway, if we want to know how a group behaves when lawbreaking goes unpunished, we need look no further than the FIRE industries, Congress, and the current Cabinet. </snark>
j_baker 16y ago
I would imagine now is when the Haskellers start claiming that laws wouldn't need to be tested if they had a good enough type system. :-P
RyanMcGreal 16y ago
>Randomly assign a representative sample of the population - say, 10,000 taxpayers - a lower tax rate, and see what happens. This assumes the tax rate only affects the individual paying tax; but taxes are used at least in part to finance public goods - positive externalities that also affect individuals. If a random group of people personally pay lower taxes but still benefit from the available public services that other taxpayers are still funding, that doesn't tell us anything useful.

150 points by robg 16y ago

Why stop at new laws? Existing laws should be relentlessly reviewed for relevance. One issue I see with random sample law experimentation is blindness to effects which only crop up when everyone is subject to a law. Take for example gun control. If a small sample of folks aren't allowed to have guns there may be little negative fall out. But taken to the extreme if gun ownership is illegal than by definition only law breakers would have guns making the random burglary or home invasion far less dangerous for criminals and far more dangerous for law abiding citizens.

randallsquared 16y ago
I'm not sure it's clear that carrying a gun makes crime safer for the average criminal, though I think it is clear that armed homeowners make crime more dangerous for criminals. My impression is that when guns are hard to get, crime goes up, but the number of criminals carrying goes down.
Zak 16y ago
Gun control is a case where there's probably enough data to do interesting analysis if someone were to collect it and study it without trying to make it show a specific result. Noteworthy data points would be the first derivatives of gun crime, non-gun violent crime and burglary rates before and after specific gun control measures were passed or repealed.
- Retric 16y ago
  There is a fair amount of evidence that gun owners are less safe than non gun owners, and societies with higher levels of gun ownership tend to be less safe overall. But, changing gun laws don't alter the prevalence of gun ownership vary quickly so it's hard to such analysis with idential populations.
  - Zak 16y ago
    There's also the fact that correlation does not imply causation. The possibility of a common cause is fairly obvious here: people who are at higher risk of being the targets of crime may be more likely to buy guns for protection, and people who are armed may be more likely to become involved if they see someone else being attacked.
    - Retric 16y ago
      Yep, while hiding and calling the cops is generally safer during a robbery, the attitude that causes people to defend their property is the same one that results in gun ownership. Add in people being shot with their own weapons and the home protection benefit becomes very complex. There is an idea that gun ownership deters criminals but at the same time it promotes criminals buying guns escalating the situation. PS: For a somewhat less contentious example, video games have been linked with a significant drop in crime. However, because that does not agree with their biases people focus on the direct effects to support their argument.
      - Zak 16y ago
        I think you'll find there tends to be a correlation between using [protection against bad thing] and being at risk for [bad thing]. As an example, I'm fairly certain you'll find a correlation between wearing a helmet while driving a car and being injured in car crashes because most people who wear helmets while driving are racing or doing stunts. I also suspect people with burglar alarms lose more money to burglary than people without if no other variables are taken in to account. I don't think an unbiased study has been done that accurately identifies causes looking at gun ownership and crime. My suspicion is that it would be possible to do, as I described above.
      - yummyfajitas 16y ago
        Add in people being shot with their own weapons... This is a pretty minor effect, unless you are including suicides. In 2007, there were only 613 accidental firearm deaths. For comparison, there were 1039 "transport, other" deaths (I believe this means plane/boat crashes).
- yummyfajitas 16y ago
  At least one study purports to do exactly this. http://www.amazon.com/More-Guns-Less-Crime-Understanding/dp/0226493636 http://www.amazon.com/More-Guns-Less-Crime-Understanding/dp/... (I haven't read it myself, so I'm not endorsing it. It's not exactly uncontroversial.)
  - gjm11 16y ago
    In particular, its author got caught promoting his work with sockpuppets. See http://en.wikipedia.org/wiki/John_Lott http://en.wikipedia.org/wiki/John_Lott and (for a rather less staid take on the affair) http://www.whoismaryrosh.com/ http://www.whoismaryrosh.com/ .
    - yummyfajitas 16y ago
      Yeah, it looks like the author is a dickwad with a big ego. I don't plan to invite him to any parties. However, the scientific criticisms of his methodology hardly look damning (according to wikipedia). It doesn't look perfect, but it doesn't look terrible either. The criticisms wikipedia lists: he used cocaine prices rather than cocaine consumption as a variable, if you remove smaller counties +florida the drop in murders becomes statistically insignificant (drops in assault and robbery are still significant), and a couple of critics believe the model has bias and coding errors. As I said, I haven't read it in detail, so I won't comment on how significant these are.
amelim 16y ago
This is the reason why jury nullification exists. It's just a shame that most citizens have no idea that it either exists, or is a valid choice when serving on a jury.
- Zaak 16y ago
  A problem with jury nullification is that the interactions between laws is complex and can easily have unintended consequences. I am overall in favor of jury nullification, but it requires an educated jury. The complexity of our system of laws makes that very difficult.
G_Wen 16y ago
An interesting issue. One way around it would be to enact the law in two states that have a similar environments. Continuing the example with gun control if there were two states with similar situations, both of them have an abnormally high gun crime rate, one of them could serve as a control state. If say 10 pairs of states were compared it would serve as a reasonable experiment on whether the law should be passed nation wide or not. Even better if evidence comes up that passing the law would only be beneficial in certain circumstances then it would help reveal the underlying causes of the problem in the first place.
- messel 16y ago
  Sharp response G_wen. Appreciate the way you overcame the random sample issue with distributed geo-sampling.
adolph 16y ago
Texas operates along a similar proposition. It is called the Sunset Advisory Commission and applies to policies, regulations and agencies. http://www.sunset.state.tx.us/ http://www.sunset.state.tx.us/

pierrefar 16y ago

Surely this is a prime candidate for the Hawthorne effect? From Wikipedia: http://en.wikipedia.org/wiki/Hawthorne_effect http://en.wikipedia.org/wiki/Hawthorne_effect The Hawthorne effect is a form of reactivity whereby subjects improve or modify an aspect of their behavior being experimentally measured simply in response to the fact that they are being studied, not in response to any particular experimental manipulation.

metageek 16y ago
From the article: >But in other cases, participating knowingly in an experiment could distort the result (this phenomenon even has a name, the Hawthorne effect).

btilly 16y ago

It is an interesting idea. However I'd be concerned that the law would work differently with a subset than the whole population. For instance take a law banning cellphones while driving. If you try to apply this to half the population, then the police aren't going to try to apply it because at least half the time they would be pulling over people who are allowed to use cellphones. But if you pass the law for all of the people, then the police could enforce it. So you try to solve this by saying that it will be selectively enforced according to location. But now you've gotten rid of the random selection that is the heart of evaluating the statistics. For another example, look at the tax cut example they were discussing. The challenge here is that the touted economic benefit of tax cutting is that people with lower taxes spend their money, increasing circulation, and general prosperity. (And eventually improving tax revenue.) However there is no way to economically separate out the people you're giving the cut to from those you aren't. Therefore any economic change cannot be attributed to the tax cut, and you're unlikely to get a clear economic difference. (Other than that the less taxed have more disposable income.)

Symmetry 16y ago
You can't see the change in the overall economy, but you can see if the people who you gave the tax cut to are investing or spending more.
- btilly 16y ago
  We already have that data. The debated question is how many times said money continues to circulate.
alexandros 16y ago
For your seatbelt experiment, you can always separate the population based on license plates. There, randomness restored.
- tokenadult 16y ago
  That depends on whether or not license plate numbers are assigned by county, or by driver last name, or some other nonrandom criterion. And in some states, significant numbers of drivers choose their own customized license plate numbers.

tomjen3 16y ago

this sounds smart until you realize that would mean giving up the principle that the law treats everybody the same, which we always complain about the politicians doing.

ugh 16y ago
I see this as a possible problem that would have to be considered when such an idea is implemented but not something that stops the idea dead in its tracks because it is fundamentally opposed to the principle of equality. You would have to make sure that the tests are truly random (everyone has an equal chance of being selected), that they run only for a limited and pre-determined time which cannot be extended and that the differences to existing laws is not too big (e.g. people shouldn’t suddenly become multi-millionaires because some law was tested on them). Such tests would then no more or less contribute to inequality than, say, jury duty.

ph0rque 16y ago

I was hoping the proposal would be testing laws on a virtual US, something commented on before: http://news.ycombinator.com/item?id=1724778 http://news.ycombinator.com/item?id=1724778

Benjo 16y ago
How do you know we're not just part of a simulation to determine the outcome of Bush v. Gore?

Goladus 16y ago

I don't think you could test taxation, for a number of reasons. First, you don't have time to measure the effects of funding changes. How do people behave with the new services funded by the taxes, when you aren't collecting enough to provide them anyway? Does the government go bankrupt? No, because revenue was not changed significantly due to the small size of the test. Second, sample populations are unlikely to exhibit the same behavioral traits of large-scale populations. For example, if you applied a tax increase to a limited segment of the population the first behavior change you would see is that people would try to escape the test group. For something like a sales tax this is even more obvious. The overall rate of taxation matters much less than the rate relative to a competitor. It's a free competitive advantage and of course behavior will change-- the untaxed businesses will be more likely to thrive and grow while driving their competitors bankrupt. When everyone has the same tax rate they have to find real market advantages. The whole point of taxes is that they are applied universally as much as possible. Everyone contributes a little bit so that the society can have things that benefit everyone but the market is unlikely to ever provide. NPR just discussed autopsies as an example of this. Understanding why people die is good for the public, yet only the most altruistic individuals are inclined to pay for an autopsy of their own loved ones. While I suppose it is possible to design test runs of tax laws, I doubt it will ever be as simple as picking a "representative sample of 10,000." There are probably only certain kinds of laws that can be tested, and tax laws probably aren't among them.

CWuestefeld 16y ago

What is the purpose of passing a law? I submit that its primary purpose may not be to bring about a change in the behavior of those subject to it, but rather, as a signaling mechanism by which a politician can say to those in his party, or his constituents, "I'm playing on your team". We see time and time again regulation that is known to be worthless or even destructive even before it's implemented (e.g., CAN-SPAM, airport porn scanners), yet bureaucrats stand behind them, and even trumpet their "success", just the same. Recent threads here have discussed whether voting behavior is really based on weighing issues, as opposed to signaling one's identity as a member of the "club" (party, special interest, etc.). If this is true, then it's only logical that the politician's job is to make it know that he is the representative of that "club", and actual goodness of his work isn't particularly relevant.

anamax 16y ago
> I submit that its primary purpose may not be to bring about a change in the behavior of those subject to it, but rather, as a signaling mechanism by which a politician can say to those in his party, or his constituents, "I'm playing on your team". Another important purpose is encouraging support. For example, one of the best ways to get bankers to contribute to your campaign is to propose banking regulations.
DanielBMarkham 16y ago
It would follow that the ultimate law would be one that sent a strong signal to your base and had very few people who could appear on TV and say they were directly harmed by it. For some reason the tax code comes immediately to mind.

jokermatt999 16y ago

Hmm..Hacker News' love of A/B testing also applies to laws, apparently. Also, isn't one of the arguments for stronger state laws and weaker federal laws very similar to this proposal? That is, if you give states greater autonomy, it's easy to test the effects of laws and give citizens choice over which laws they'd like.

DougWebb 16y ago

Or, we could pay attention to the 10th Ammendment: "The powers not delegated to the United States by the Constitution, nor prohibited by it to the States, are reserved to the States respectively, or to the people." Instead of randomly applying variations of a law, don't federalize it. Let the states pass their own variations, and see how that works out. Sure there are other variables at play, but they mostly revolve around how suitable the law is for the people it applies to, and that's a legitimate reason for the law to vary and part of the purpose of the 10th Ammendment: local self-governance. I believe most federal laws should be repealed and replaced with guidelines for the states to follow when designing their own laws. These would be "Best Practices" for governments. Most of the world treated the US Constituion this way as they wrote their own constitutions, and I think that worked pretty well. It should work for State governments too.

DanielBMarkham 16y ago
That won't work. Somehow we've gotten into Lord of the Rings Mode -- "one ring to rule them all" We have effectively turned a distributed system into a monolithic piece of spaghetti. And now folks are suggesting that what we need -- wait for it -- is more testing. We're effectively having all the same conversations large organizations have when creating billion-dollar POS centralized software development systems. Should we focus on requirements (listen to the voters). Should we be more clear on the impact of the change (environmental, financial, etc)? Maybe we just need some super-smart people put in the right spots. That'll fix it. Many times I wonder: are people doomed to repeat the same mistakes over and over again just in slightly different settings? (sigh)
- jhancock 16y ago
  "are people doomed to repeat the same mistakes over and over again" This is a common view in mainland China. Particularly, the last few years, in regards to the U.S.: "They've had a good 200-250 year run, now this arc/dynasty is coming to its end and another arc will replace it. Hopefully the turmoil while bootstrapping the next won't be too painful." 200-300 years is the generally accepted time of a Chinese dynasty. As near I as can tell, this is due to people "forgetting" their history. Written history may not be enough. You need to be able to feel it similar to those that put their lives on the line to create it.

tibbon 16y ago

YES! We should approach this from a two-tiered testing platform. The first tier is rather like how you'd do it in Rails or any other TDD environment. We write a list of tests that we know should pass as true at the end. Let's say, start with the US and State constitutions as the basis. def must_not_prohibit_exercise_of_religion() { People.can_exercise_religion? } We then write in the test laws and see what side effects occur. When those side effects occur, we must make note of them and consider the potential exceptions. def freedom_of_speech() { unless(crowded_theatre && People.speak == fire) People.can_speak? end } As a second layer then we must test it on smaller populations to determine the effects. This has worked wonderfully in some situations. "Gay Marriage will destroy marriage and society" is a claim I've heard prior. So test it in Massachusetts. Several years later, no marriages on record have been destroyed due to gay marriage, and the metrics of society's stability have not moved significantly in any direction that we can attribute to same-sex marriage. So it (should) pass and become law nationally because we've observed its effects. We can observe similar for the long term effects of required health insurance in Massachusetts as we debate how things should work out nationally. Just as we have 'human-readable code' why shouldn't our laws be machine readable? Computers can remember/understand 200+ years of case law probably better than humans. Again, this is something that we could test. One of my friends is working on a law firm run by computers (http://www.robotandhwang.com/ http://www.robotandhwang.com/). Of course the machines can't show up in court or legally offer advice, but they can advise the attorney. (Law.com and ABA Journal have cited him doing this, so maybe it isn't as crazy as it sounds).

jgfoot 16y ago
I was about to respond until I realized your post was a Swift-ian parody so sophisticated as to be almost over my head. Well done. But if we were going to try to reduce the complicated, inherently human, and uncertain concept of law to computer code, surely you'd agree that a declarative programming language would be a better choice. Something like: prohibits_free_exercise_of_religion(X) :- unconstitutional(X). respects_establishment_of_religion(X) :- unconstitutional(X). unconstitutional(X) :- not_law(X). Now, all you have to do is write Prolog code to test for when a law prohibits the exercise of religion or respects the establishment of religion!

stuaxo 16y ago

I've been talking to people about exactly this sort of thing, I'd make the trial groups bigger though.

hugh3 16y ago

Note how the example is lower taxes: why don't we give lower taxes to a hundred thousand people and see what they do? But now suppose the proposal is higher taxes for a hundred thousand people. How do you think the hundred thousand people in the test group are going to react? Not too well, and probably not nearly the same way as if you raised taxes on everybody. Isn't this one of the supposed advantages of having states, though? So that different laws can be tried out in different places under the same general societal conditions and we can learn what works and what doesn't work? And yet doesn't the Federal Government keep trying to subsume more state powers into itself?

nickpinkston 16y ago
I'm guessing behavioral economists would be lapping this one up if they were given the cash to do so, but then we'd actually have some empirical proof about policy - politicians prefer to just rant for votes - far cheaper and easier to manipulate... Yea, power always concentrates up - seemingly because coordination costs get cheaper as the pool of people decreases in size.
- jbooth 16y ago
  Well, for the record, we have tried the whole supply-side economics thing in the real world twice now, with less-than-stellar results.. I mean, there's no control group but you'd need an alternate universe for that. EDIT: sigh. Downvote away for not liking what I said, but you can't even disagree with it, it's factual. That particular idea has been tried twice in the real world.
  - CWuestefeld 16y ago
    It's been tried much more than twice. And it's been done with success. For example, Kennedy in 1964 was very successful. See http://www.heritage.org/Research/Reports/2003/08/The-Historical-Lessons-of-Lower-Tax-Rates http://www.heritage.org/Research/Reports/2003/08/The-Histori... And also http://www.house.gov/jec/fiscal/tx-grwth/reagtxct/reagtxct.htm http://www.house.gov/jec/fiscal/tx-grwth/reagtxct/reagtxct.h... : High marginal tax rates discourage work effort, saving, and investment, and promote tax avoidance and tax evasion. A reduction in high marginal tax rates would boost long term economic growth, and reduce the attractiveness of tax shelters and other forms of tax avoidance. The economic benefits of ERTA were summarized by President Clinton's Council of Economic Advisers in 1994: "It is undeniable that the sharp reduction in taxes in the early 1980s was a strong impetus to economic growth." Unfortunately, the Council could not bring itself to acknowledge the counterproductive effects high marginal tax rates can have upon taxpayer behavior and tax avoidance activities. Since 1984 the JEC has provided factual information about the impact of the tax cuts of the 1980s. For example, for many years the JEC has published IRS data on federal tax payments of the top 1 percent, top 5 percent, top 10 percent, and other taxpayers. These data show that after the high marginal tax rates of 1981 were cut, tax payments and the share of the tax burden borne by the top 1 percent climbed sharply. For example, in 1981 the top 1 percent paid 17.6 percent of all personal income taxes, but by 1988 their share had jumped to 27.5 percent, a 10 percentage point increase. The graph below illustrates changes in the tax burden during this period. Indeed, I'd like to see your evidence that it's bad. That evidence should be clear, in that it shows that the problem is not actually government over-spending or other issues.

albemuth 16y ago

I thought that's what California was all about

qjz 16y ago

I'd be happy if they developed a protocol for testing the constitutionality of a law. That would have to be part of any experiment, since the effectiveness of a law is irrelevant if it's unconstitutional.

wazoox 16y ago

Problem : most people aren't interested in reality and truth. See the the "teach creationism" debate. It would be great, however.

gibsonf1 16y ago

The idea that raising taxes increases revenue has been shown to be untrue with actual taxation data time and again. A good resource for cause and effect with taxation is "Hauser's Law": http://en.wikipedia.org/wiki/Hausers_Law http://en.wikipedia.org/wiki/Hausers_Law

JimboOmega 16y ago

I've advocated this for a while in a somewhat obvious field, traffic law. However, there's a flaw; science in the political arena is subject to interpretation. In VA, we've gone through this with red light cameras. For a while they were authorized at a few intersections to see if they reduced accidents. They didn't (http://www.thenewspaper.com/news/18/1844.asp http://www.thenewspaper.com/news/18/1844.asp). But a few years later, red light cameras are back, the whole study thing having been an annoying thorn in the side of politicians, who like red light cameras (because they are perceived as good for safety, for money, whatever - who knows?) So an article comes out like this: http://www.washingtonpost.com/wp-dyn/content/article/2009/02/18/AR2009021801108.html http://www.washingtonpost.com/wp-dyn/content/article/2009/02... which contradicts the link I posted before. How can the same study come to opposite conclusions? That's politics. In any case, the politicians decide what they want to do, and then change the science to suit. After all, can you really imagine a politician saying "I supported X fully, until the results of the study came out, and clearly, X is not effective, so I no longer support X?" Only if the study is poll numbers, I imagine.

Aaronontheweb 16y ago

We have a system for doing this - it's called Federalism.

jimmyk 16y ago
Unfortunately, the federal aspect of the US government has almost disappeared as a result of the 16th amendment and the 17th amendment and a few other factors such as mandates that the states must follow in order to receive federal funding. The US government is now plainly a national one.

brudgers 16y ago

IANAL, but I don't think this meets the Equal Protection Clause of the 14th Amendment.

vhackish 16y ago

This would be a great way to test out new jails too - when you break a test law, straight to the test jail with you!

jessriedel 16y ago

The RAND study is famous for trying this on a pretty large scale with health insurance: http://en.wikipedia.org/wiki/RAND_Health_Insurance_Experiment http://en.wikipedia.org/wiki/RAND_Health_Insurance_Experimen... Robin Hanson has been a big proponent of resuming similar experiments: http://www.overcomingbias.com/2007/05/rand_health_ins.html http://www.overcomingbias.com/2007/05/rand_health_ins.html http://www.overcomingbias.com/2007/05/rand_health_ins_1.html http://www.overcomingbias.com/2007/05/rand_health_ins_1.html http://www.overcomingbias.com/2007/05/rand_experiment.html http://www.overcomingbias.com/2007/05/rand_experiment.html Worth reading even if you're of a different political persuasion.

ChristianMarks 16y ago

Effective methods to elevate political and social discourse above the level of ideological debate would be revolutionary. Empirical testing of proposed legislation should be implemented as soon as possible. In controversial cases where experimentation doesn't lead to satisfactory outcomes, thanks to ideological intransigence, the method should be supplemented with a proposal that I call Bimodal Politics. Technology could be used to manage controversial political issues, for which the distribution of voters is bimodal and for which there is essentially no middle ground. Such issues include abortion rights, stem cell research and gay marriage. Through the system of bimodal politics, voters would live in parallel legal and political worlds, with different rights and obligations to the state depending on how they voted. Where controversial legal and political matters are concerned, the distribution of opinion involves roughly equal and opposite numbers of equally-informed, impartial rational voters. (Some would contend that for issues such as abortion, a few vociferous opponents give the misleading impression of a much larger opposition, but until this is established, we will proceed as if public discourse reflects a genuine controversy.) Although the voters on each side of a controversy may not see the other side as rational or equally informed, one essential feature of controversies is that they cannot be decided by common morality, and they have to be transferred to the legal and political system for resolution. The legal and political system can attempt to resolve controversial issues one way or the other, but it is understood that the resolution is provisional and does not resolve any underlying moral issues. Deciding controversial issues one way or the other is inherently unstable. What if it were possible to decide issues in parallel, in such a way that each group in a controversial issue imposes rights and obligations only upon itself and no group imposes its vote on the members of the other group? Is there a procedure that would result in less political and social instability than deciding one way or the other for all voters? Bimodal politics attempts to provide a legal and political mechanism for resolving politically controversial issues, with the understanding that it provides a political decision procedure: it does not address the underlying moral issues. In outline, a database is maintained of voter preference on controversial issues that are designated bimodal issues. Your vote is recorded by the bimodal voter database. Your vote determines your rights and obligations to the state on that particular issue in parallel with those voters who voted oppositely, and who may have (and probably have) different rights and obligations under the state. Consider stem cell research. Under the proposed system, stem cell research would be designated a bimodal issue. During an election, your vote on stem cell research would be entered into the database. If you voted in favor of stem cell research, you may be taxed to support it, your embryos may be harvested for stem cells (these may be from embryos slated for destruction in any case) and if you develop a disease that requires stem cell derived therapy for its treatment, you will be eligible for it. If you voted against stem cell research, you will not be taxed to support it, your embryos will not be harvested for stem cells and if you develop a disease you will be prohibited from pursuing treatments derived from stem cell research. Abortion is another issue that would be designated a bimodal issue under the system of bimodal politics. If you vote against abortion, your tax dollars will not be used to support state-sponsored family planning programs or sex education, and you will be legally barred from having an abortion if you are female. If you are male and you impregnate a woman who has an abortion, and you voted against abortion, you will be held legally liable. In either case, if you voted against abortion and your fetus or your partner’s fetus is aborted, you will be prosecuted by the state. However, if you voted in favor of abortion, your fetus can be aborted, and your tax dollars may go to support state-sponsored family planning programs and sex education. These examples illustrate the slogan that under bimodal politics, you live in the world you voted for.

rwhitman 16y ago

I firmly believe that the next major system of government to emerge in the future will be heavily influenced from the lessons learned in governing social media communities. The idea of A/B testing laws seems like it would fit into this pretty well.

Natsu 16y ago

Maybe we could also put the laws into source control, be able to roll back to a "known good" state and have a "blame" command for who introduced changes that screwed everything up. Unfortunately, politics is largely in the realm of emotion. And people hate being proven wrong.

giardini 16y ago

At least we could do A/B testing with them! Isn't that what states are for?

gills 16y ago

Experimenting on humans with a system backed by the use of force would be despicable. What happened to basic respect for each other? <snark> Anyway, if we want to know how a group behaves when lawbreaking goes unpunished, we need look no further than the FIRE industries, Congress, and the current Cabinet. </snark>

j_baker 16y ago

I would imagine now is when the Haskellers start claiming that laws wouldn't need to be tested if they had a good enough type system. :-P

RyanMcGreal 16y ago

>Randomly assign a representative sample of the population - say, 10,000 taxpayers - a lower tax rate, and see what happens. This assumes the tax rate only affects the individual paying tax; but taxes are used at least in part to finance public goods - positive externalities that also affect individuals. If a random group of people personally pay lower taxes but still benefit from the available public services that other taxpayers are still funding, that doesn't tell us anything useful.