12 ms·

Dutch scandal serves as a warning for Europe over risks of using algorithms

244 points by Kiala 4y ago

jacquesm 4y ago
This scandal and the issues at the dutch tax office runs much deeper than just this one set of issues, they also maintained illegal lists of 'potential fraudulent people' on which 100's of thousands of people that had done nothing wrong, such as having the 'wrong' surname. It's rotten to the core and to date there have been zero consequences for the perps and the damage continues to pile up. Our government has fallen over this and - surprise - the exact same players were re-elected without taking any responsibility for any of this.
- StreamBright 4y ago
  It seems like bodies of government without accountability have something in common: corruption.
  - Freak_NL 4y ago
    This was not a case of corruption. Incompetence, discrimination, and lack of accountability, but not corruption. Most civil servants involved believed they were doing the right thing (despite ruining many lives in reality), and none profited personally, provided favours, or pocketed money.
    - StreamBright 4y ago
      "Corruption is a form of dishonesty or a criminal offense which is undertaken by a person or an organization which is entrusted with a position of authority, in order to acquire illicit benefits or abuse power for one's personal gain." I use corruption in the dictionary definition sense. I guess you think that corruption only means bribery.
      - evandijk70 4y ago
        GP used the same definition - employees of the tax-office received no illicit benefits or personal gain.
      - Freak_NL 4y ago
        > […] in order to acquire illicit benefits or abuse power for one's personal gain. There was no personal gain nor illicit benefits gained by the civil servants responsible. For something to be corruption, the perpetrator has to gain something.
        edejong 4y ago
        Would a promotion (or not losing your job) count as a gain?
        zambal 4y ago
        It might be a gain, but that doesn't make it corruption.
        rocqua 4y ago
        I'd say no to keeping your job, and yes to a promotion only if it is accelerated.
        StreamBright 4y ago
        You think that they did this because they are purely evil? I think they had to gain something. Maybe the gain was less work.
- yunohn 4y ago
  Meanwhile, the irony amidst all this is that the Netherlands is the largest tax haven for companies globally. Literally every brand/company you can think of is legally headquartered in NL for tax purposes. Eg: FAANG, Disney, IKEA, Uber, Mars Chocolate, etc etc
  - chekibreki 4y ago
    Are you sure you’re not mixing up the Netherlands with Ireland?
    - Freak_NL 4y ago
      Both are tax havens.
    - yunohn 4y ago
      No, I’m not. Most of these companies also have incorporated in Ireland alongside the Netherlands to exploit this loophole: https://en.m.wikipedia.org/wiki/Dutch_Sandwich https://en.m.wikipedia.org/wiki/Dutch_Sandwich > Ireland has the BEPS tools to enable US IP-heavy multinationals to reroute global profits into Ireland, tax-free. The Netherlands then enables these Irish profits to get to a classical tax haven (e.g. the Cayman Islands or Jersey) without incurring EU withholding tax. I empathise with your the confusion - corruption has enabled these companies to construct confusing and elaborate ways of avoiding their fair share of taxes.
      - disgruntledphd2 4y ago
        > corruption has enabled these companies to construct confusing and elaborate ways of avoiding their fair share of taxes. I'm not sure one can consider it corruption, per se, rather it's tax competition on the part of smaller EU states. To my mind, it's not that the countries are being bribed by companies, rather that the laws attract a (far too small proportion of) global/EU revenue to be taxed in their country.
        yunohn 4y ago
- eesmith 4y ago
  It's shocking to read, and reminds me of the British Post Office scandal. A €3.7 million + €2.75 million fines are a slap-on-the wrist for what happened. Even €30,000/case seems a pittance compared to its effects on people. It mentions there were "secret blacklists of people for two decades" and that "authorities [were] hiding information or misleading the parliament about the facts". I don't see how regulating algorithms and AI could affect those institutional injustices.
  - jacquesm 4y ago
    Yes, and besides it is the one government institution fining another, so there is really no consequence for any of this.
    - justin_oaks 4y ago
      It kills me when government institutions get fined as a punishment. It's just the tax payers that are being punished. We as a society need to do better to incentivize good behavior for those in trusted government positions, and hold the individuals responsible when they betray that trust.
      - iostream24 4y ago
        Yes!!! Exactly and precisely this. The actual individuals should be accountable for decisions they made that ruined other peoples lives for wrongful cause. Too often political decisions have been predicated on the notion of cost-free bullying of lower status humans to serve as grist for the mills of polemical political opposition and discourse, to feed careers. The bullfighter needs a bull and ideally it should look scary before the predetermined outcome of slaying the scapegoat takes place. While corporations and governments only punish the people with unilateral action of malice or ignorance, one can never truly punish an institution, one seeks individuals who are responsible in the hopes of reducing the willingness of individuals to commit such acts given the possibility of individual punishment. Nothing says impunity like impunity. Fining the institution does nothing to discourage career hit-and-ruin experts who keep climbing the career ladder upon the backs of others.
- Kaotique 4y ago
  On the other hand one of the main triggers for this whole operation was the mass fraud by people from Bulgaria who netted a lot of benefits by gaming the system. They traveled to the Netherlands, applied for benefits and then went back to Bulgaria and used Dutch bank accounts and debit cards to get the cash. The tax office never wanted that to happen again and started this program to find fraudsters but ended up classifying a lot of people wrongly. To make it it even more Kafkaesque some of the Bulgarian fraudsters ALSO received the 30k compensation! One of the recommendations is to stop pumping around so much money. If you let less money circulate the system trough taxes and benefits less can go wrong. Why do people need to receive hundreds of thousands of Euros for children daycare? It makes no sense whatsoever. Let the people keep the money, pay less taxes and pay the daycare centers from the government directly for each child. The bureaucracy is the real problem here. When parents requested their paperwork they received hundreds of papers with all the real information blacked out. Imagine completely black papers. It was absolutely disgusting and the Dutch government even stepped down because of the whole affair. Amazingly the same political parties got elected again and are now in office in the same coalition as before...
  - jacquesm 4y ago
    This has been trotted out many times before in defense of the tax office's behavior but it doesn't hold water: far more people were affected than Bulgarian fraudsters and besides that compared to the fraud committed by the well-off portion of NL the fraud committed by the poorer part is negligible. But guess who is better at defending themselves against unfair behavior by the authorities.
    - rocqua 4y ago
      This is not a defense of the tax office, but it is the cause. The 'bulgaren-fraude' was the cause for the government to change policy (and I believe law) to be much stricter on fraud and be much more vigilant when trying to detect fraud. The current scandal is one of the end results here. An interesting intermediary result is that the culture at the tax-office changed to be much more about 'getting the fraudsters' and much less about 'helping people'. Throw on a bit of "bureaucrats trying to cover their own and their bosses' ass" when people accidentally get marked for fraud, and you start getting the results we have here. I fear that this culture (both the citizen as adversary 'getting fraudsters' culture and the ass covering) is still in place, and much more widespread throughout the government. Similar things seem to be happening at the National Mining Institute trying to discredit earthquake damage in Groningen for one example. <sarcasm> luckily when the government stepped down because of the last scandal, the new government was a big change and a big step towards fixing these attitudes </sarcasm>
- sAbakumoff 4y ago
  D66 who I would argue was the true winner of the election does not consist of "exact same players". The next election, given that we don't have the new pandemic or war in Europe, will be the last one for Rutte, I believe.
  - jacquesm 4y ago
    D66 also wasn't the major problem here, that was clearly the VVD. As for the last of Rutte, I thought that we had seen that one election before the last and yet he's still there.
- vanderZwan 4y ago
  > This scandal and the issues at the dutch tax office runs much deeper than just this one set of issues The article doesn't even list all of the issues. For example, on top of the things you rightfully pointed out, this article also fails to mention is that (at least) 1115 children were taken away from their parents as a result of this, and despite this being known for more than half a year, not a single child has been returned to their parents[0]. There just seems to be zero interest by the authorities to help the people who went through life-destroying tragedies as a result of all this. To paraphrase one comedian from the Dutch version of Have I Got News For You[1], "if one child would go missing tomorrow the entire country would be upside down, there would be search parties, the media would go crazy with the story. And yet at least 1115 children have been wrongfully taken from their families, we've known this for half a year, and nothing seems to happen." [0] https://www.dutchnews.nl/news/2021/10/over-1100-children-of-childcare-benefit-victims-taken-into-care/ https://www.dutchnews.nl/news/2021/10/over-1100-children-of-... [1] https://www.youtube.com/watch?v=5JIejxvgjz0 https://www.youtube.com/watch?v=5JIejxvgjz0
sidewndr46 4y ago
> The Dutch tax authorities now face a new €3.7 million fine from the country's privacy regulator. How does this work? Wouldn't the government be fining the government and paying itself with the citizens tax revenues? Also, this doesn't seem to serve as a warning at all. Unless all the politicians and civil servants were removed from their job, barred from public positions and given hefty fines then this is actually a clear sign encouraging more of this behavior from governments. Until there are clear consequences enacted by the populace, governments will continue this kind of behavior.
- StreamBright 4y ago
  It is part of the theatre, unelected government officials waste taxpayers money on something illegal so they have to pay with taxpayer money for it. The only real punishment would have been is jail time for the people involved and explaining to the rest of the unelected government officials that they can end up in jail. Just like how the rest of society works.
  - sidewndr46 4y ago
    This sounds like playing monopoly with taxpayer dollars.
  - rjmunro 4y ago
    It doesn't have to be jail time for the people involved, it could just be fines, salary cuts, demotions etc. Only the very worst offenders should go to jail.
- yunohn 4y ago
  > Until there are clear consequences enacted by the populace, governments will continue this kind of behavior. In fact, the government staged a token dissolution right before scheduled general elections, and proceeded to win the elections and come right back into power. Nothing changed. Truly magical how accountability works.
  - rocqua 4y ago
wallaBBB 4y ago
By reading, seems that the algorithm is just a scapegoat used in the title. This is racial discriminations dating way back, purposely introduced in the algorithm, and supported for 6 years. > Having dual nationality was marked as a big risk indicator, as was a low income. >In 2020, Trouw and another Dutch news outlet, RTL Nieuws revealed that the tax authorities also kept secret blacklists of people for two decades, which tracked both credible and unsubstantiated “signals” of potential fraud. Citizens had no way of finding out why they were on the list or defending themselves. >An audit showed that the tax authorities focused on people with “a non-Western appearance,” while having Turkish or Moroccan nationality was a particular focus. Being on the blacklist also led to a higher risk score in the child care benefits system.
- ClumsyPilot 4y ago
  No fly lists too, no algorythms involved
  - droopyEyelids 4y ago
    The only reason it isn't an "algorithm" is because no one thought to use that language when it was created. All "algorithm" means is "a process or set of rules to be followed in calculations"- computers don't need to be involved, nor does "AI". The no-fly list is a process of looking up people in a list of names when calculating whether they can fly. That fits the definition as much as anything else the media calls an algorithm.
    - samhw 4y ago
      Honestly, as someone most of whose life is spent writing algorithms, when I hear any non-technical person say the word "algorithm" that's my cue to tune out. "Algorithms" seems to have a meaning that's approximately "super-intelligent runaway AI robots ... oh, and they're also racist". Meanwhile, neural nets are accomplishing the great, hitherto-unreachable task of [checks notes] grade school math: https://www.tensorflow.org/datasets/catalog/math_dataset https://www.tensorflow.org/datasets/catalog/math_dataset
    - 4y ago
fallingknife 4y ago
> Authorities penalized families over a mere suspicion of fraud based on the system’s risk indicators This is the real story here. The algorithm is incidental.
- Sharlin 4y ago
  Not sure about that in the real world. The temptation of "algorithms" is in reducing human workload which, realistically, means saving on salary costs by having fewer humans in the loop. The attraction of cost savings – and the externalization of repercussions – biases decisionmakers into believing that algorithms are a great solution. Yes, the real problem is abuse of algorithmic decisionmaking, and a lack of oversight, but in the real world you can't separate a technology from the people who use it.
- orangepurple 4y ago
  The families were guilty of breaking the community guidelines /s
- cool_dude85 4y ago
  Can't agree with this. Even if it was just feeding a list to human investigators, who fairly investigated each case, there's still the significant issue of how the list is determined. For example, the tax authority in your country right now catches a small portion of overall fraud. I give them a list of "hot leads" of fraudsters that's really just a list of all the ethnic minorities I hate. Obviously, even if they fairly investigate everyone, and nobody who's not defrauding loses out, this practice would be deeply racist and disproportionately punish the minority. In short, we can't human-wash this problem at the end. Even if everyone punished was a fraudster, strong bias in the initial list is still clearly an issue.
  - rocqua 4y ago
    Whilst you are right in general that biased algorithms are bad even when people using them are good, the problem in _this_ scandal was the the people using the algorithm were trigger-happy, and partially so in a biased way.
raxxorraxor 4y ago
> That’s not good enough, argues Renske Leijten, a Socialist member of the Dutch parliament Exactly, and especially in the public sector on the EU level this could be even vastly more damaging. The warning were loud enough and people didn't listen or care, so it is probably necessary that many will have to feel it before anything will change. This is probably too small scale, even if thousands are affected.
liftm 4y ago
> risks of using algorithms Bubblesort so dangerous. Not exactly the best headline, is it?
- rocqua 4y ago
  Presuming a sorted list gives an 'objective view' which you can act on directly, without needing to check context, interpretation, bias in your input, and unintended consequences of treating this sorted list as truth, is quite dangerous. Whether that is done with bubblesort or not indeed does not matter. But thinking that acting based purely on data and data processing is a good idea, that sure as shit matters. In fact, it is a big mistake.
- iostream24 4y ago
  You clearly have not encountered the term “human sorting” eh?
smcl 4y ago
Nadine Dorries (UK Secretary of State for Digital, Culture, Media and Sport) was right, we should simply get rid of algorithms. They are clearly harmful /s [0] - https://www.huffingtonpost.co.uk/entry/nadine-dorries-microsoft-algorithm-twitter_uk_62331aaee4b0d39357c37f9c https://www.huffingtonpost.co.uk/entry/nadine-dorries-micros...
BlueTemplar 4y ago
Around here we have a law where any decision taken by the public sector with the help of a computer program must (upon request) be explained in plain language, the calculations detailed step by step. Of course, this indirectly prohibits the use of the kind of programs that seem to have been used in this case, since instead of an algorithm, they use a neural network which is a black box even to his trainer (or architect). Sadly, I hear that our own tax authorities have started to use them anyway, but then they have long had this tendency of consider themselves "special", and that laws didn't apply as much to them...
- rjmunro 4y ago
  Where is "around here"?
  - BlueTemplar 4y ago
    A EU country.
chekibreki 4y ago
I hate how the correlation between income/country of origin and fraud probability is discarded as „racist“ by at least Amnesty International. Maybe we should first do a correlation and causality analysis and see what it yields instead of closing our eyes because of PC?
- jacquesm 4y ago
  That analysis has been done and the conclusions are devastating for the tax office. https://www.consultancy.nl/nieuws/39000/pwc-belastingdienst-plaatste-mensen-op-basis-van-uiterlijk-en-nationaliteit-op-zwarte-lijst https://www.consultancy.nl/nieuws/39000/pwc-belastingdienst-... (dutch)
  - orangepurple 4y ago
    Devastating? I don't see anything devastating in the article you linked. Perhaps you can point it out? The language is very weak. > PwC considers it plausible that the people who ended up on the blacklist on the basis of appearance or nationality were disadvantaged as a result. Like others on the list, they presumably did not qualify for smooth debt restructuring, debt forgiveness, or a personal payment arrangement. Unlike in the allowance affair, according to PwC, the list played no role in the assessment of income tax returns. aannemelijk vermoedelijk They can't point to a single case in which this happened?
    - jacquesm 4y ago
      You could of course start with the first paragraph: "Bij het inschatten van frauderisico’s beoordeelde de Belastingdienst zeker tientallen mensen op hun nationaliteit en uiterlijk, zo blijkt uit onderzoek van PwC. Zo kwamen de mensen terecht op de Fraude Signaleringsvoorziening (FSV), beter bekend als de zwarte lijst."
- croes 4y ago
  So what could cause a causality between income/country of origin and fraud probability? Is fraud genetic? I don't think so, so it's only a correlation at most.
deleted 4y ago
[deleted]
RcouF1uZ4gsC 4y ago
> Chermaine Leysner’s life changed in 2012, when she received a letter from the Dutch tax authority demanding she pay back her child care allowance going back to 2008. Leysner, then a student studying social work, had three children under the age of 6. The tax bill was over €100,000. A student can get €100,000 for child care? That is amazing. For me, that is the most surprising thing in the entire article.
- jacquesm 4y ago
  That may well have been including punitive components. They're really good at that.
- Loughla 4y ago
  €100,000 = ~$108,000 So if that's the credit for 3 children, for 4 years, that's ~$750/month. I have zero idea about how much it would cost for the Dutch, but in the closest city center to me in the US, the average cost for a licensed daycare center is ~$1300/month. Not unrealistic, really, unless I missed something.
  - pacifika 4y ago
    210 euro a month for 2 days a week supposedly https://www.xpat.nl/living-netherlands/kids/day-care/ https://www.xpat.nl/living-netherlands/kids/day-care/
- Freak_NL 4y ago
  She was a student at the time she received the letters claiming 'missing information' that led to the (unjust) claim. This does not mean she didn't have a job before that or some other occupation. She was around 30 at that time, and study was just part of her career path. If you have several children, the child care allowance adds up — I receive over €1000 a month for one preschool child. Multiply this for more than one child and several years… It is a system that is increasingly seen as hostile to citizens, because you need to update your registration with every change in income. It's silly too, because the allowance you receive goes to a child day care centre or kindergarten anyway, so you are just shovelling money around which leads to almost all parents being burdened with something completely avoidable. The answer is to just make child day care free for all children and not bother citizens with subsidies to pay it. It's probably cheaper to do it that way (we would be following in the footsteps of other progressive countries).
nicgrev103 4y ago
Reminds me of this: https://www.bbc.co.uk/news/business-56718036 https://www.bbc.co.uk/news/business-56718036 700 Post office franchisees made criminals by faulty accounting software.
mahesh_rm 4y ago
Once upon a time I made the mistake of incorporating a startup in the Netherlands. Never. Again.
- orangepurple 4y ago
  What went wrong?
  - vanderZwan 4y ago
    Based on their username I would guess "being not-white in a country that is totally convinced it isn't racist but is actually seriously racist". Also, it's not discussed much but the Netherlands has a very serious "old boy network" problem where white douchebags who didn't achieve much during their studies except boozing with other white douchebags keep failing upwards. It's the reason why me and my mixed ethnicity background do not regret emigrating either, despite the superior bike infrastructure and hagelslag.
    - BlargMcLarg 4y ago
      Mind you, literally anyone who doesn't fit this mentality in some way (ethnicity, sex, character, etc.) is quickly considered an outsider with problems getting in. You can be a white heterosexual native guy and still struggle because the Dutch mentality doesn't mesh with you personally. Very much because "boozing with other white douchebags" concerns all descriptors. It's why the Dutch "kan niet"-mentality is so prominent.
      - mtts 4y ago
        Very much this. The Netherlands is essentially a frat house kept afloat by dangerously unregulated immigrant labor, a dwindling number of middle class professionals and tax laws that would be an embarrassment to all but the most unrepentant of tax havens.
    - ChuckNorris89 4y ago
      >but the Netherlands has a very serious "old boy network" problem where white douchebags who didn't achieve much during their studies except boozing with other white douchebags keep failing upwards. This isn't some Dutch thing. I've seen the exact same thing in Germany, Austria and Switzerland at the big-name companies in those countries (like BMW, etc.). Basically, to succeeded, be a white local from a mid-upper/upper class family, go to a famous uni there and party, and you'll fail yourself to the top management in a big company there quite nicely, while immigrants have glass ceilings to make sure they know their place.
tomputer 4y ago
In the documentary Alone Against the State, five mothers who became victims of the Dutch childcare benefits scandal share the chilling stories of how their lives and that of their children were destroyed by a state system deprived of humanity. It is a Dutch documentary, unfortunately without English subtitles: https://www.2doc.nl/documentaires/series/2doc/2021/alleen-tegen-de-staat.html https://www.2doc.nl/documentaires/series/2doc/2021/alleen-te...
ai_ja_nai 4y ago
> "An audit showed that the tax authorities focused on people with “a non-Western appearance,” while having Turkish or Moroccan nationality was a particular focus". This has nothing to do with algorithms, mates. It had to do with how it wasn't tested and monitored. It appears that it was just rolled out live. > "Fraud prediction and predictive policing based on profiling should just be banned" And here are the usual luddists pandering on the confused using wrong arguments. Sigh.
- convexfunction 4y ago
  It should've been (and almost surely was) some specific person's job to sign off on this thing being at least reasonable before deploying it, which they failed to do. Successfully redirecting attention from that specific person's failure to do their job (and their manager who failed to ensure they were doing their job, etc) to the abstract idea of Algorithms is some exceptional accountability judo.
  - rocqua 4y ago
    Our prime minister (Rutte) is known as 'teflon Rutte' because no real scandal seems to stick to him. The entire idea of dodging accountability seems to be present from the very top of government.
- raxxorraxor 4y ago
rowanG077 4y ago
I'm dutch and this really is disgusting. But you have to understand that the Dutch population has no real problem with this. The exact same political parties were elected a few months after this scandal. It's just democracy really.
- ChuckNorris89 4y ago
  Probably because most Dutch people assume shit like this can't/won't happen to them so it's no biggie that the lives of 5 families were ruined. Even less of an issues for them if some of those families affected were immigrants and not Dutch natives.
  - amelius 4y ago
    5 families? the article speaks of thousands of people ...
  - jb1991 4y ago
    Except it was a lot more than just five families.
turbo_bean 4y ago
Born and raised in EU South, I grew up looking up to Northern Europe with regards to their efficiency, values, etc. This is news I'd expect to hear from a backwater country I'd never heard off. But I'm afraid it is just yet one data point of how wrong I was and how bad things have gone in the heart of EU and maybe the whole western world.
gsliepen 4y ago
Another issue that contributed to this scandal is that the rules for benefits (not just social benefits) are often such that you have to request it up front at the beginning of the year, but if at the end of the year it turns out that your situation changed midway and you no longer had the (full) right to the benefit, you have to pay it back along with a fine. It's really easy to not be aware of this.
mint2 4y ago
This is utterly bonkers. The discrimination sounds completely blatant and is it actually saying any case flagged by the algorithm was considered and treated as proven fraud rather than just raised for investigation?!? That’s absurd.
einpoklum 4y ago
I don't understand the article. Suppose they had an algorithm for computing a risk score. Ok, so someone is determined to have a "high risk score". That doesn't mean that they've done anything wrong; at most, it means prioritizing them for closer scrutiny. That could still be discriminatory (e.g. focus on Turkish/Moroccan immigrants), but in itself - it's not supposed to lose anyone their benefits. Also, the article says: > Citizens had no way of ... defending themselves. Why could they not sue the tax authorities in court, asking the court to order the tax authorities to withdraw their payment demands? If there was no evidence of them having evaded taxes, I mean.
- mtts 4y ago
  High risk score means you’re under scrutiny. Once you’re under scrutiny every minor thing that goes wrong or is unclear is by default assumed to be fraud and you get slapped with (often outrageously high) fines, payable immediately. You can only appeal after the fact, so you’re drowning in debt even before the first judge looks at your case. That is assuming a judge gets anywhere near your case. This often didn’t happen because access to the law is made extremely difficult in the Netherlands (on purpose). The people this affected mostly did not have the skills to find their way in the Dutch legal system so they were ruined. All of this is by design, by the way: the system assumes ill will by default, spends an inordinate amount of resources on detecting it and punishes it severely all because being tough on fraud polls well.
logifail 4y ago
> Having dual nationality was marked as a big risk indicator Dual nationality indeed has its ups and downs. FWIW, our children have dual nationality, and on top of that, are all bilingual. On enrolling them in their first school the school insisted on tweaking the enrollment data we'd submitted to list the locally-spoken language second, and so put the non-local language first, as "mother tongue" (and yes, that's exactly how it's described round here <sigh>). This meant the school would get extra money from the government from a pot of money intended to support migrant kids with language issues. The first time this happened we pushed straight back and asked to get this corrected (all our kids were born locally and speak the local language and dialect like everyone else round here), but the school wasn't having it. They couldn't understand why we didn't want to support the school getting extra money. What did we learn from this? If the incentives and/or system are broken by design, it can get very hard for anyone wanting to get the system corrected later.
- DavideNL 4y ago
  > but the school wasn't having it That's rude. I would probably consider getting the school to communicate this in writing, and then report it for fraud at some point; when the kids leave the school, or when I'd be sure it couldn't cause any issues for the kids...
  - pessimizer 4y ago
    > I would probably consider getting the school to communicate this in writing You would only catch the stupidest, most pitiable people with this trap.
- hef19898 4y ago
  How would they now? By no means those dual nationalities are centrally managed. And therw is no obligation to state a second cotizenship on any papers. Disclaimer: Dual citizenship kids in Europe.
  - logifail 4y ago
    > And therw is no obligation to state a second cotizenship on any papers You're probably right re: citizenship but this is about language, how could one "hide" being bilingual, especially if we're talking about a young child? (Even in kindergarten all my daughter's friends are 100% aware that she and I speak a language when we're communicating solely with each other that isn't the language my daughter uses the rest of the time in kindergarten.)
drno123 4y ago
If I calculate correctly, Dutch child care allowance is around $2000 per month, which seems generous. What is the average child care allowance in other EU countries?
Tozen 4y ago
This really isn't about "algorithms", so much as it is about xenophobia and ethnic discrimination. The "algorithm", is just another plausible deniability cover for institutionalized discrimination against various minorities and the poor who are least able to fight back. Those in charge, know exactly what they are doing when they create their "blacklists" and "targets". They know the names, they have the bias and prejudices, and they want to target those minority groups. The poorer and less able to fight back, the better. They know the type of damage they are doing to those peoples lives, and sadistically enjoy inflicting it. It's only when they get caught, do they start running around and pulling out excuses from their orifices, and "it was the algorithm" is just one of them.
- vanderZwan 4y ago
  > The "algorithm", is just another plausible deniability cover for institutionalized discrimination against various minorities and the poor who are least able to fight back. What makes this extra infuriating is that even before these algorithms were implemented in the Netherlands there were already plenty of people warning us that this would happen if we ever would be so foolish to do so. I can only presume they feel a certain frustrated kinship with climate scientists right now
goatcode 4y ago
From what the article spelled out, using algorithms isn't precisely the risk, but using them in a completely horrendously unpracticed and uninformed way seems to be.
kragen 4y ago
"Risks of using algorithms"? Clearly the Enlightenment has ended, because this is a headline that would only make sense in the age of the Inquisition.
matheusmoreira 4y ago
The perils of false positives and making decisions based on probability. These governments are punishing innocents and hoping the guilty is among them. Horrifying.
kkfx 4y ago
That's not really an algorithm usage issue but an administrative ones. Really. The issue does not arise from an ineffective/wrong system but for the absence of quick, powerful and effective human backup. A small example: automated procedures, if well designed tend to be really effective *in means* witch means they are good. BUT they tend to have hard to spot (or even evident, but no one care up front) loopholes and corner cases that makes not them but their use a nightmare. In those case the solution is simple: a human that pick all issues not automatically treated. The *administrative* issue is this: those who choose automation must know it can fail and so must be prepared for a normal and regular and effective backup. Personally if a day my country tax agency ask me 100k euros for a clear error I feel no specific scandal, anyone can makes error, but I expect to have a phone number to call where someone pick the call quickly and in little time we can sort out the issue. It's not a life-or-death thing, and it's complex, issues normally happen and that's is. If I can't call because no one answer or they answer but they are powerless or there is no quick solution than it's not the erroneous claim the scandal, but the absence of means to sort it out rapidly.
jokoon 4y ago
I'm on welfare in france, and sometimes I wonder if the government is going to find some excuse to stop giving me money. Could be "not applying to enough job offers", while most employers are always complaining about candidates not being skilled enough, which is why unemployment is so high.
- BlueTemplar 4y ago
  This is happening to my half-sick mother, right now, 6 months before retirement (!), because Pôle Emploi is seemingly unable to both check its own database for the presence of a recent job offer she was rejected from and also to reply to (or even read ?) the (e-)mail pointing this out that has been sent to it. Going to have to seize the Ombudsperson now... Some related levity : Nistoire, professeur de chômage - Groland - CANAL+ : https://www.youtube.com/watch?v=n-Yck-aO4Ww https://www.youtube.com/watch?v=n-Yck-aO4Ww (fr)
a_bonobo 4y ago
Sounds a bit like Australia's illegal Robodebt scheme: https://en.wikipedia.org/wiki/Robodebt_scheme https://en.wikipedia.org/wiki/Robodebt_scheme The Australian system was simpler than the Dutch system, comparing social benefits paid with averaged tax-income from the ATO. It was a stupidly wrong calculation, the averaged income is much higher in people with inconsistent income than it actually is, so people were hounded by debt collectors for nonexisting debt. A$1.2 billion false debt for a nation of 25 million people. A good number of associated suicides. Of course, there was no real accountability, with the system having been introduced by the previous government who is now in opposition. Money has been paid back but everybody involved still has their job.
puffoflogic 4y ago
When are we going to bite the bullet and require governmental licensing for algorithms? I know HN hates the idea because we're all too busy using algorithms to ruin people's lives for profit, but it needs to happen and it will happen sooner or later.

244 points by Kiala 4y ago

This scandal and the issues at the dutch tax office runs much deeper than just this one set of issues, they also maintained illegal lists of 'potential fraudulent people' on which 100's of thousands of people that had done nothing wrong, such as having the 'wrong' surname. It's rotten to the core and to date there have been zero consequences for the perps and the damage continues to pile up. Our government has fallen over this and - surprise - the exact same players were re-elected without taking any responsibility for any of this.

StreamBright 4y ago
It seems like bodies of government without accountability have something in common: corruption.
- Freak_NL 4y ago
  This was not a case of corruption. Incompetence, discrimination, and lack of accountability, but not corruption. Most civil servants involved believed they were doing the right thing (despite ruining many lives in reality), and none profited personally, provided favours, or pocketed money.
  - StreamBright 4y ago
    "Corruption is a form of dishonesty or a criminal offense which is undertaken by a person or an organization which is entrusted with a position of authority, in order to acquire illicit benefits or abuse power for one's personal gain." I use corruption in the dictionary definition sense. I guess you think that corruption only means bribery.
    - evandijk70 4y ago
      GP used the same definition - employees of the tax-office received no illicit benefits or personal gain.
    - Freak_NL 4y ago
      > […] in order to acquire illicit benefits or abuse power for one's personal gain. There was no personal gain nor illicit benefits gained by the civil servants responsible. For something to be corruption, the perpetrator has to gain something.
      - edejong 4y ago
        Would a promotion (or not losing your job) count as a gain?
        zambal 4y ago
        It might be a gain, but that doesn't make it corruption.
        rocqua 4y ago
        I'd say no to keeping your job, and yes to a promotion only if it is accelerated.
      - StreamBright 4y ago
        You think that they did this because they are purely evil? I think they had to gain something. Maybe the gain was less work.
yunohn 4y ago
Meanwhile, the irony amidst all this is that the Netherlands is the largest tax haven for companies globally. Literally every brand/company you can think of is legally headquartered in NL for tax purposes. Eg: FAANG, Disney, IKEA, Uber, Mars Chocolate, etc etc
- chekibreki 4y ago
  Are you sure you’re not mixing up the Netherlands with Ireland?
  - Freak_NL 4y ago
    Both are tax havens.
  - yunohn 4y ago
    No, I’m not. Most of these companies also have incorporated in Ireland alongside the Netherlands to exploit this loophole: https://en.m.wikipedia.org/wiki/Dutch_Sandwich https://en.m.wikipedia.org/wiki/Dutch_Sandwich > Ireland has the BEPS tools to enable US IP-heavy multinationals to reroute global profits into Ireland, tax-free. The Netherlands then enables these Irish profits to get to a classical tax haven (e.g. the Cayman Islands or Jersey) without incurring EU withholding tax. I empathise with your the confusion - corruption has enabled these companies to construct confusing and elaborate ways of avoiding their fair share of taxes.
    - disgruntledphd2 4y ago
      > corruption has enabled these companies to construct confusing and elaborate ways of avoiding their fair share of taxes. I'm not sure one can consider it corruption, per se, rather it's tax competition on the part of smaller EU states. To my mind, it's not that the countries are being bribed by companies, rather that the laws attract a (far too small proportion of) global/EU revenue to be taxed in their country.
      - yunohn 4y ago
eesmith 4y ago
It's shocking to read, and reminds me of the British Post Office scandal. A €3.7 million + €2.75 million fines are a slap-on-the wrist for what happened. Even €30,000/case seems a pittance compared to its effects on people. It mentions there were "secret blacklists of people for two decades" and that "authorities [were] hiding information or misleading the parliament about the facts". I don't see how regulating algorithms and AI could affect those institutional injustices.
- jacquesm 4y ago
  Yes, and besides it is the one government institution fining another, so there is really no consequence for any of this.
  - justin_oaks 4y ago
    It kills me when government institutions get fined as a punishment. It's just the tax payers that are being punished. We as a society need to do better to incentivize good behavior for those in trusted government positions, and hold the individuals responsible when they betray that trust.
    - iostream24 4y ago
      Yes!!! Exactly and precisely this. The actual individuals should be accountable for decisions they made that ruined other peoples lives for wrongful cause. Too often political decisions have been predicated on the notion of cost-free bullying of lower status humans to serve as grist for the mills of polemical political opposition and discourse, to feed careers. The bullfighter needs a bull and ideally it should look scary before the predetermined outcome of slaying the scapegoat takes place. While corporations and governments only punish the people with unilateral action of malice or ignorance, one can never truly punish an institution, one seeks individuals who are responsible in the hopes of reducing the willingness of individuals to commit such acts given the possibility of individual punishment. Nothing says impunity like impunity. Fining the institution does nothing to discourage career hit-and-ruin experts who keep climbing the career ladder upon the backs of others.
Kaotique 4y ago
On the other hand one of the main triggers for this whole operation was the mass fraud by people from Bulgaria who netted a lot of benefits by gaming the system. They traveled to the Netherlands, applied for benefits and then went back to Bulgaria and used Dutch bank accounts and debit cards to get the cash. The tax office never wanted that to happen again and started this program to find fraudsters but ended up classifying a lot of people wrongly. To make it it even more Kafkaesque some of the Bulgarian fraudsters ALSO received the 30k compensation! One of the recommendations is to stop pumping around so much money. If you let less money circulate the system trough taxes and benefits less can go wrong. Why do people need to receive hundreds of thousands of Euros for children daycare? It makes no sense whatsoever. Let the people keep the money, pay less taxes and pay the daycare centers from the government directly for each child. The bureaucracy is the real problem here. When parents requested their paperwork they received hundreds of papers with all the real information blacked out. Imagine completely black papers. It was absolutely disgusting and the Dutch government even stepped down because of the whole affair. Amazingly the same political parties got elected again and are now in office in the same coalition as before...
- jacquesm 4y ago
  This has been trotted out many times before in defense of the tax office's behavior but it doesn't hold water: far more people were affected than Bulgarian fraudsters and besides that compared to the fraud committed by the well-off portion of NL the fraud committed by the poorer part is negligible. But guess who is better at defending themselves against unfair behavior by the authorities.
  - rocqua 4y ago
    This is not a defense of the tax office, but it is the cause. The 'bulgaren-fraude' was the cause for the government to change policy (and I believe law) to be much stricter on fraud and be much more vigilant when trying to detect fraud. The current scandal is one of the end results here. An interesting intermediary result is that the culture at the tax-office changed to be much more about 'getting the fraudsters' and much less about 'helping people'. Throw on a bit of "bureaucrats trying to cover their own and their bosses' ass" when people accidentally get marked for fraud, and you start getting the results we have here. I fear that this culture (both the citizen as adversary 'getting fraudsters' culture and the ass covering) is still in place, and much more widespread throughout the government. Similar things seem to be happening at the National Mining Institute trying to discredit earthquake damage in Groningen for one example. <sarcasm> luckily when the government stepped down because of the last scandal, the new government was a big change and a big step towards fixing these attitudes </sarcasm>
sAbakumoff 4y ago
D66 who I would argue was the true winner of the election does not consist of "exact same players". The next election, given that we don't have the new pandemic or war in Europe, will be the last one for Rutte, I believe.
- jacquesm 4y ago
  D66 also wasn't the major problem here, that was clearly the VVD. As for the last of Rutte, I thought that we had seen that one election before the last and yet he's still there.
vanderZwan 4y ago
> This scandal and the issues at the dutch tax office runs much deeper than just this one set of issues The article doesn't even list all of the issues. For example, on top of the things you rightfully pointed out, this article also fails to mention is that (at least) 1115 children were taken away from their parents as a result of this, and despite this being known for more than half a year, not a single child has been returned to their parents[0]. There just seems to be zero interest by the authorities to help the people who went through life-destroying tragedies as a result of all this. To paraphrase one comedian from the Dutch version of Have I Got News For You[1], "if one child would go missing tomorrow the entire country would be upside down, there would be search parties, the media would go crazy with the story. And yet at least 1115 children have been wrongfully taken from their families, we've known this for half a year, and nothing seems to happen." [0] https://www.dutchnews.nl/news/2021/10/over-1100-children-of-childcare-benefit-victims-taken-into-care/ https://www.dutchnews.nl/news/2021/10/over-1100-children-of-... [1] https://www.youtube.com/watch?v=5JIejxvgjz0 https://www.youtube.com/watch?v=5JIejxvgjz0

sidewndr46 4y ago

> The Dutch tax authorities now face a new €3.7 million fine from the country's privacy regulator. How does this work? Wouldn't the government be fining the government and paying itself with the citizens tax revenues? Also, this doesn't seem to serve as a warning at all. Unless all the politicians and civil servants were removed from their job, barred from public positions and given hefty fines then this is actually a clear sign encouraging more of this behavior from governments. Until there are clear consequences enacted by the populace, governments will continue this kind of behavior.

StreamBright 4y ago
It is part of the theatre, unelected government officials waste taxpayers money on something illegal so they have to pay with taxpayer money for it. The only real punishment would have been is jail time for the people involved and explaining to the rest of the unelected government officials that they can end up in jail. Just like how the rest of society works.
- sidewndr46 4y ago
  This sounds like playing monopoly with taxpayer dollars.
- rjmunro 4y ago
  It doesn't have to be jail time for the people involved, it could just be fines, salary cuts, demotions etc. Only the very worst offenders should go to jail.
yunohn 4y ago
> Until there are clear consequences enacted by the populace, governments will continue this kind of behavior. In fact, the government staged a token dissolution right before scheduled general elections, and proceeded to win the elections and come right back into power. Nothing changed. Truly magical how accountability works.
- rocqua 4y ago

wallaBBB 4y ago

By reading, seems that the algorithm is just a scapegoat used in the title. This is racial discriminations dating way back, purposely introduced in the algorithm, and supported for 6 years. > Having dual nationality was marked as a big risk indicator, as was a low income. >In 2020, Trouw and another Dutch news outlet, RTL Nieuws revealed that the tax authorities also kept secret blacklists of people for two decades, which tracked both credible and unsubstantiated “signals” of potential fraud. Citizens had no way of finding out why they were on the list or defending themselves. >An audit showed that the tax authorities focused on people with “a non-Western appearance,” while having Turkish or Moroccan nationality was a particular focus. Being on the blacklist also led to a higher risk score in the child care benefits system.

ClumsyPilot 4y ago
No fly lists too, no algorythms involved
- droopyEyelids 4y ago
  The only reason it isn't an "algorithm" is because no one thought to use that language when it was created. All "algorithm" means is "a process or set of rules to be followed in calculations"- computers don't need to be involved, nor does "AI". The no-fly list is a process of looking up people in a list of names when calculating whether they can fly. That fits the definition as much as anything else the media calls an algorithm.
  - samhw 4y ago
    Honestly, as someone most of whose life is spent writing algorithms, when I hear any non-technical person say the word "algorithm" that's my cue to tune out. "Algorithms" seems to have a meaning that's approximately "super-intelligent runaway AI robots ... oh, and they're also racist". Meanwhile, neural nets are accomplishing the great, hitherto-unreachable task of [checks notes] grade school math: https://www.tensorflow.org/datasets/catalog/math_dataset https://www.tensorflow.org/datasets/catalog/math_dataset
  - 4y ago

fallingknife 4y ago

> Authorities penalized families over a mere suspicion of fraud based on the system’s risk indicators This is the real story here. The algorithm is incidental.

Sharlin 4y ago
Not sure about that in the real world. The temptation of "algorithms" is in reducing human workload which, realistically, means saving on salary costs by having fewer humans in the loop. The attraction of cost savings – and the externalization of repercussions – biases decisionmakers into believing that algorithms are a great solution. Yes, the real problem is abuse of algorithmic decisionmaking, and a lack of oversight, but in the real world you can't separate a technology from the people who use it.
orangepurple 4y ago
The families were guilty of breaking the community guidelines /s
cool_dude85 4y ago
Can't agree with this. Even if it was just feeding a list to human investigators, who fairly investigated each case, there's still the significant issue of how the list is determined. For example, the tax authority in your country right now catches a small portion of overall fraud. I give them a list of "hot leads" of fraudsters that's really just a list of all the ethnic minorities I hate. Obviously, even if they fairly investigate everyone, and nobody who's not defrauding loses out, this practice would be deeply racist and disproportionately punish the minority. In short, we can't human-wash this problem at the end. Even if everyone punished was a fraudster, strong bias in the initial list is still clearly an issue.
- rocqua 4y ago
  Whilst you are right in general that biased algorithms are bad even when people using them are good, the problem in _this_ scandal was the the people using the algorithm were trigger-happy, and partially so in a biased way.

raxxorraxor 4y ago

> That’s not good enough, argues Renske Leijten, a Socialist member of the Dutch parliament Exactly, and especially in the public sector on the EU level this could be even vastly more damaging. The warning were loud enough and people didn't listen or care, so it is probably necessary that many will have to feel it before anything will change. This is probably too small scale, even if thousands are affected.

liftm 4y ago

> risks of using algorithms Bubblesort so dangerous. Not exactly the best headline, is it?

rocqua 4y ago
Presuming a sorted list gives an 'objective view' which you can act on directly, without needing to check context, interpretation, bias in your input, and unintended consequences of treating this sorted list as truth, is quite dangerous. Whether that is done with bubblesort or not indeed does not matter. But thinking that acting based purely on data and data processing is a good idea, that sure as shit matters. In fact, it is a big mistake.
iostream24 4y ago
You clearly have not encountered the term “human sorting” eh?

smcl 4y ago

Nadine Dorries (UK Secretary of State for Digital, Culture, Media and Sport) was right, we should simply get rid of algorithms. They are clearly harmful /s [0] - https://www.huffingtonpost.co.uk/entry/nadine-dorries-microsoft-algorithm-twitter_uk_62331aaee4b0d39357c37f9c https://www.huffingtonpost.co.uk/entry/nadine-dorries-micros...

BlueTemplar 4y ago

Around here we have a law where any decision taken by the public sector with the help of a computer program must (upon request) be explained in plain language, the calculations detailed step by step. Of course, this indirectly prohibits the use of the kind of programs that seem to have been used in this case, since instead of an algorithm, they use a neural network which is a black box even to his trainer (or architect). Sadly, I hear that our own tax authorities have started to use them anyway, but then they have long had this tendency of consider themselves "special", and that laws didn't apply as much to them...

rjmunro 4y ago
Where is "around here"?
- BlueTemplar 4y ago
  A EU country.

chekibreki 4y ago

I hate how the correlation between income/country of origin and fraud probability is discarded as „racist“ by at least Amnesty International. Maybe we should first do a correlation and causality analysis and see what it yields instead of closing our eyes because of PC?

jacquesm 4y ago
That analysis has been done and the conclusions are devastating for the tax office. https://www.consultancy.nl/nieuws/39000/pwc-belastingdienst-plaatste-mensen-op-basis-van-uiterlijk-en-nationaliteit-op-zwarte-lijst https://www.consultancy.nl/nieuws/39000/pwc-belastingdienst-... (dutch)
- orangepurple 4y ago
  Devastating? I don't see anything devastating in the article you linked. Perhaps you can point it out? The language is very weak. > PwC considers it plausible that the people who ended up on the blacklist on the basis of appearance or nationality were disadvantaged as a result. Like others on the list, they presumably did not qualify for smooth debt restructuring, debt forgiveness, or a personal payment arrangement. Unlike in the allowance affair, according to PwC, the list played no role in the assessment of income tax returns. aannemelijk vermoedelijk They can't point to a single case in which this happened?
  - jacquesm 4y ago
    You could of course start with the first paragraph: "Bij het inschatten van frauderisico’s beoordeelde de Belastingdienst zeker tientallen mensen op hun nationaliteit en uiterlijk, zo blijkt uit onderzoek van PwC. Zo kwamen de mensen terecht op de Fraude Signaleringsvoorziening (FSV), beter bekend als de zwarte lijst."
croes 4y ago
So what could cause a causality between income/country of origin and fraud probability? Is fraud genetic? I don't think so, so it's only a correlation at most.

deleted 4y ago

[deleted]

RcouF1uZ4gsC 4y ago

> Chermaine Leysner’s life changed in 2012, when she received a letter from the Dutch tax authority demanding she pay back her child care allowance going back to 2008. Leysner, then a student studying social work, had three children under the age of 6. The tax bill was over €100,000. A student can get €100,000 for child care? That is amazing. For me, that is the most surprising thing in the entire article.

jacquesm 4y ago
That may well have been including punitive components. They're really good at that.
Loughla 4y ago
€100,000 = ~$108,000 So if that's the credit for 3 children, for 4 years, that's ~$750/month. I have zero idea about how much it would cost for the Dutch, but in the closest city center to me in the US, the average cost for a licensed daycare center is ~$1300/month. Not unrealistic, really, unless I missed something.
- pacifika 4y ago
  210 euro a month for 2 days a week supposedly https://www.xpat.nl/living-netherlands/kids/day-care/ https://www.xpat.nl/living-netherlands/kids/day-care/
Freak_NL 4y ago
She was a student at the time she received the letters claiming 'missing information' that led to the (unjust) claim. This does not mean she didn't have a job before that or some other occupation. She was around 30 at that time, and study was just part of her career path. If you have several children, the child care allowance adds up — I receive over €1000 a month for one preschool child. Multiply this for more than one child and several years… It is a system that is increasingly seen as hostile to citizens, because you need to update your registration with every change in income. It's silly too, because the allowance you receive goes to a child day care centre or kindergarten anyway, so you are just shovelling money around which leads to almost all parents being burdened with something completely avoidable. The answer is to just make child day care free for all children and not bother citizens with subsidies to pay it. It's probably cheaper to do it that way (we would be following in the footsteps of other progressive countries).

nicgrev103 4y ago

Reminds me of this: https://www.bbc.co.uk/news/business-56718036 https://www.bbc.co.uk/news/business-56718036 700 Post office franchisees made criminals by faulty accounting software.

mahesh_rm 4y ago

Once upon a time I made the mistake of incorporating a startup in the Netherlands. Never. Again.

orangepurple 4y ago
What went wrong?
- vanderZwan 4y ago
  Based on their username I would guess "being not-white in a country that is totally convinced it isn't racist but is actually seriously racist". Also, it's not discussed much but the Netherlands has a very serious "old boy network" problem where white douchebags who didn't achieve much during their studies except boozing with other white douchebags keep failing upwards. It's the reason why me and my mixed ethnicity background do not regret emigrating either, despite the superior bike infrastructure and hagelslag.
  - BlargMcLarg 4y ago
    Mind you, literally anyone who doesn't fit this mentality in some way (ethnicity, sex, character, etc.) is quickly considered an outsider with problems getting in. You can be a white heterosexual native guy and still struggle because the Dutch mentality doesn't mesh with you personally. Very much because "boozing with other white douchebags" concerns all descriptors. It's why the Dutch "kan niet"-mentality is so prominent.
    - mtts 4y ago
      Very much this. The Netherlands is essentially a frat house kept afloat by dangerously unregulated immigrant labor, a dwindling number of middle class professionals and tax laws that would be an embarrassment to all but the most unrepentant of tax havens.
  - ChuckNorris89 4y ago
    >but the Netherlands has a very serious "old boy network" problem where white douchebags who didn't achieve much during their studies except boozing with other white douchebags keep failing upwards. This isn't some Dutch thing. I've seen the exact same thing in Germany, Austria and Switzerland at the big-name companies in those countries (like BMW, etc.). Basically, to succeeded, be a white local from a mid-upper/upper class family, go to a famous uni there and party, and you'll fail yourself to the top management in a big company there quite nicely, while immigrants have glass ceilings to make sure they know their place.

tomputer 4y ago

In the documentary Alone Against the State, five mothers who became victims of the Dutch childcare benefits scandal share the chilling stories of how their lives and that of their children were destroyed by a state system deprived of humanity. It is a Dutch documentary, unfortunately without English subtitles: https://www.2doc.nl/documentaires/series/2doc/2021/alleen-tegen-de-staat.html https://www.2doc.nl/documentaires/series/2doc/2021/alleen-te...

ai_ja_nai 4y ago

> "An audit showed that the tax authorities focused on people with “a non-Western appearance,” while having Turkish or Moroccan nationality was a particular focus". This has nothing to do with algorithms, mates. It had to do with how it wasn't tested and monitored. It appears that it was just rolled out live. > "Fraud prediction and predictive policing based on profiling should just be banned" And here are the usual luddists pandering on the confused using wrong arguments. Sigh.

convexfunction 4y ago
It should've been (and almost surely was) some specific person's job to sign off on this thing being at least reasonable before deploying it, which they failed to do. Successfully redirecting attention from that specific person's failure to do their job (and their manager who failed to ensure they were doing their job, etc) to the abstract idea of Algorithms is some exceptional accountability judo.
- rocqua 4y ago
  Our prime minister (Rutte) is known as 'teflon Rutte' because no real scandal seems to stick to him. The entire idea of dodging accountability seems to be present from the very top of government.
raxxorraxor 4y ago

rowanG077 4y ago

I'm dutch and this really is disgusting. But you have to understand that the Dutch population has no real problem with this. The exact same political parties were elected a few months after this scandal. It's just democracy really.

ChuckNorris89 4y ago
Probably because most Dutch people assume shit like this can't/won't happen to them so it's no biggie that the lives of 5 families were ruined. Even less of an issues for them if some of those families affected were immigrants and not Dutch natives.
- amelius 4y ago
  5 families? the article speaks of thousands of people ...
- jb1991 4y ago
  Except it was a lot more than just five families.

turbo_bean 4y ago

Born and raised in EU South, I grew up looking up to Northern Europe with regards to their efficiency, values, etc. This is news I'd expect to hear from a backwater country I'd never heard off. But I'm afraid it is just yet one data point of how wrong I was and how bad things have gone in the heart of EU and maybe the whole western world.

gsliepen 4y ago

Another issue that contributed to this scandal is that the rules for benefits (not just social benefits) are often such that you have to request it up front at the beginning of the year, but if at the end of the year it turns out that your situation changed midway and you no longer had the (full) right to the benefit, you have to pay it back along with a fine. It's really easy to not be aware of this.

mint2 4y ago

This is utterly bonkers. The discrimination sounds completely blatant and is it actually saying any case flagged by the algorithm was considered and treated as proven fraud rather than just raised for investigation?!? That’s absurd.

einpoklum 4y ago

I don't understand the article. Suppose they had an algorithm for computing a risk score. Ok, so someone is determined to have a "high risk score". That doesn't mean that they've done anything wrong; at most, it means prioritizing them for closer scrutiny. That could still be discriminatory (e.g. focus on Turkish/Moroccan immigrants), but in itself - it's not supposed to lose anyone their benefits. Also, the article says: > Citizens had no way of ... defending themselves. Why could they not sue the tax authorities in court, asking the court to order the tax authorities to withdraw their payment demands? If there was no evidence of them having evaded taxes, I mean.

mtts 4y ago
High risk score means you’re under scrutiny. Once you’re under scrutiny every minor thing that goes wrong or is unclear is by default assumed to be fraud and you get slapped with (often outrageously high) fines, payable immediately. You can only appeal after the fact, so you’re drowning in debt even before the first judge looks at your case. That is assuming a judge gets anywhere near your case. This often didn’t happen because access to the law is made extremely difficult in the Netherlands (on purpose). The people this affected mostly did not have the skills to find their way in the Dutch legal system so they were ruined. All of this is by design, by the way: the system assumes ill will by default, spends an inordinate amount of resources on detecting it and punishes it severely all because being tough on fraud polls well.

logifail 4y ago

> Having dual nationality was marked as a big risk indicator Dual nationality indeed has its ups and downs. FWIW, our children have dual nationality, and on top of that, are all bilingual. On enrolling them in their first school the school insisted on tweaking the enrollment data we'd submitted to list the locally-spoken language second, and so put the non-local language first, as "mother tongue" (and yes, that's exactly how it's described round here <sigh>). This meant the school would get extra money from the government from a pot of money intended to support migrant kids with language issues. The first time this happened we pushed straight back and asked to get this corrected (all our kids were born locally and speak the local language and dialect like everyone else round here), but the school wasn't having it. They couldn't understand why we didn't want to support the school getting extra money. What did we learn from this? If the incentives and/or system are broken by design, it can get very hard for anyone wanting to get the system corrected later.

DavideNL 4y ago
> but the school wasn't having it That's rude. I would probably consider getting the school to communicate this in writing, and then report it for fraud at some point; when the kids leave the school, or when I'd be sure it couldn't cause any issues for the kids...
- pessimizer 4y ago
  > I would probably consider getting the school to communicate this in writing You would only catch the stupidest, most pitiable people with this trap.
hef19898 4y ago
How would they now? By no means those dual nationalities are centrally managed. And therw is no obligation to state a second cotizenship on any papers. Disclaimer: Dual citizenship kids in Europe.
- logifail 4y ago
  > And therw is no obligation to state a second cotizenship on any papers You're probably right re: citizenship but this is about language, how could one "hide" being bilingual, especially if we're talking about a young child? (Even in kindergarten all my daughter's friends are 100% aware that she and I speak a language when we're communicating solely with each other that isn't the language my daughter uses the rest of the time in kindergarten.)

drno123 4y ago

If I calculate correctly, Dutch child care allowance is around $2000 per month, which seems generous. What is the average child care allowance in other EU countries?

Tozen 4y ago

This really isn't about "algorithms", so much as it is about xenophobia and ethnic discrimination. The "algorithm", is just another plausible deniability cover for institutionalized discrimination against various minorities and the poor who are least able to fight back. Those in charge, know exactly what they are doing when they create their "blacklists" and "targets". They know the names, they have the bias and prejudices, and they want to target those minority groups. The poorer and less able to fight back, the better. They know the type of damage they are doing to those peoples lives, and sadistically enjoy inflicting it. It's only when they get caught, do they start running around and pulling out excuses from their orifices, and "it was the algorithm" is just one of them.

vanderZwan 4y ago
> The "algorithm", is just another plausible deniability cover for institutionalized discrimination against various minorities and the poor who are least able to fight back. What makes this extra infuriating is that even before these algorithms were implemented in the Netherlands there were already plenty of people warning us that this would happen if we ever would be so foolish to do so. I can only presume they feel a certain frustrated kinship with climate scientists right now

goatcode 4y ago

From what the article spelled out, using algorithms isn't precisely the risk, but using them in a completely horrendously unpracticed and uninformed way seems to be.

kragen 4y ago

"Risks of using algorithms"? Clearly the Enlightenment has ended, because this is a headline that would only make sense in the age of the Inquisition.

matheusmoreira 4y ago

The perils of false positives and making decisions based on probability. These governments are punishing innocents and hoping the guilty is among them. Horrifying.

kkfx 4y ago

That's not really an algorithm usage issue but an administrative ones. Really. The issue does not arise from an ineffective/wrong system but for the absence of quick, powerful and effective human backup. A small example: automated procedures, if well designed tend to be really effective *in means* witch means they are good. BUT they tend to have hard to spot (or even evident, but no one care up front) loopholes and corner cases that makes not them but their use a nightmare. In those case the solution is simple: a human that pick all issues not automatically treated. The *administrative* issue is this: those who choose automation must know it can fail and so must be prepared for a normal and regular and effective backup. Personally if a day my country tax agency ask me 100k euros for a clear error I feel no specific scandal, anyone can makes error, but I expect to have a phone number to call where someone pick the call quickly and in little time we can sort out the issue. It's not a life-or-death thing, and it's complex, issues normally happen and that's is. If I can't call because no one answer or they answer but they are powerless or there is no quick solution than it's not the erroneous claim the scandal, but the absence of means to sort it out rapidly.

jokoon 4y ago

I'm on welfare in france, and sometimes I wonder if the government is going to find some excuse to stop giving me money. Could be "not applying to enough job offers", while most employers are always complaining about candidates not being skilled enough, which is why unemployment is so high.

BlueTemplar 4y ago
This is happening to my half-sick mother, right now, 6 months before retirement (!), because Pôle Emploi is seemingly unable to both check its own database for the presence of a recent job offer she was rejected from and also to reply to (or even read ?) the (e-)mail pointing this out that has been sent to it. Going to have to seize the Ombudsperson now... Some related levity : Nistoire, professeur de chômage - Groland - CANAL+ : https://www.youtube.com/watch?v=n-Yck-aO4Ww https://www.youtube.com/watch?v=n-Yck-aO4Ww (fr)

a_bonobo 4y ago

Sounds a bit like Australia's illegal Robodebt scheme: https://en.wikipedia.org/wiki/Robodebt_scheme https://en.wikipedia.org/wiki/Robodebt_scheme The Australian system was simpler than the Dutch system, comparing social benefits paid with averaged tax-income from the ATO. It was a stupidly wrong calculation, the averaged income is much higher in people with inconsistent income than it actually is, so people were hounded by debt collectors for nonexisting debt. A$1.2 billion false debt for a nation of 25 million people. A good number of associated suicides. Of course, there was no real accountability, with the system having been introduced by the previous government who is now in opposition. Money has been paid back but everybody involved still has their job.

puffoflogic 4y ago

When are we going to bite the bullet and require governmental licensing for algorithms? I know HN hates the idea because we're all too busy using algorithms to ruin people's lives for profit, but it needs to happen and it will happen sooner or later.