4 ms·

It's a structural issue caused by the way wealth creation works for majority of people in tech. Job hopping, trendy frameworks in CV, "high-impact" projects don

It's a structural issue caused by the way wealth creation works for majority of people in tech. Job hopping, trendy frameworks in CV, "high-impact" projects done ASAP, etc. No one wants to do boring, slow pace work with lots of planning, reflection and introspection. And why would they do it? These kind of jobs are usually worst paid. We, the practitioners, have every economic incentive to go the other route. The problem goes far wider in tech than just ML. And unless the society collectively learns to appreciate patience and long-term thinking, as virtues above all else, it won't go away any time soon. What can be done is to discourage use of ML systems if an explainable deterministic system can be used (even one developed in a rush). For example credit scoring. Rules are good while black box artificial neural network isn't, even if the NN has some % more accuracy. Then if the rules are not good then can be amended and in special cases customer support could also override the rules based on human (hopefully unbiased) judgement. The problem mentioned in the article of COVID-19 detection based on radiology scans is an example of a system which needs ANNs due to the nature of image processing (very difficult problem for rules AI). While techniques such as ShAP could be helpful a radiologist still needs to check because ANNs learn a lot of useless noise very often and the prediction can be nonsensical. Here it would be best to use PCR tests, serology or any more traditional and "boring" tool as it works. Luckily that is the case and shit CNN models start and end their lives in some useless paper.

deleted 5y ago
[deleted]
xyzzy21 5y ago
Sadly PCR tests for COVID also test positive for flu and half a dozen other causes. That's why CDC/FDA are seeking proposals for a new test that actually works! https://www.cdc.gov/csels/dls/locs/2021/07-21-2021-lab-alert-Changes_CDC_RT-PCR_SARS-CoV-2_Testing_1.html https://www.cdc.gov/csels/dls/locs/2021/07-21-2021-lab-alert...
- maxerickson 5y ago
  You've fallen for the internet. Please restart and try again. https://www.reuters.com/article/factcheck-covid19-pcr-test-idUSL1N2P42U5 https://www.reuters.com/article/factcheck-covid19-pcr-test-i...
  - nerdponx 5y ago
    Sigh. The deniers and antivaxers will have made up their minds already, and this will just be perceived as part of the mass media coverup. It's hopeless.
- ramchip 5y ago
  You've been repeating this, but it doesn't seem to be true... https://news.ycombinator.com/item?id=28262833 https://news.ycombinator.com/item?id=28262833
- vibrato2 5y ago
  Interesting that you’re getting downvoted for the same sentiment that Kary Mullis, inventor of PCR, talked about. He said the test is not a standalone diagnostic because if the amplification is high enough you can find anything in anyone [paraphrase] For most of the pandemic, in fact roughly coinciding with release of the vaccines, the standard amplification cycle was very high. Then they reduced it. Flagged in 30s, that is also odd.
- convnet 5y ago
  That is simply false on a basic level. That notice by the CDC says the EXACT OPPOSITE of your comment. They're recommending that labs switch to a multiplex test that can screen for both flu and covid at the same time because PCR only detects SARS CoV 2.
  - verygoodname 5y ago
    Note: the "multiplex test" is most likely still a PCR test (just 'multiplex PCR' instead of 'single-probe PCR'), so where you say "PCR only detects SARS CoV 2" it should say "the currently-used PCR test only detects SARS CoV 2".
bsanr2 5y ago
Why would anyone care to fix things? The way they are are perfectly amenable to the blame- and conclusion-laundering many ML clients seek.
TuringNYC 5y ago
I saw a large organization which was the epitome of this -- Executive Directors would propose ambitious ML projects, Directors would create plans and teams, Managers would execute on budgets, create more detailed plans, and then...someone actually needed to do the work. Because of the length of the effort, the annual compensation would already have been handed out and the EDs, Directors, Managers had already "extracted" their compensation for the project, but usually had none left for the workers who eventually needed to do the actual work. Not unexpectedly, a rough job was somehow jammed thru with understaffed, underpaid, and unmotivated low-level workers to actually "deliver" on the "AI" projects -- so victory could be declared at the top level...and new projects could begin. This isnt an ML problem, i'm sure the whole cycle has been repeated with technology-of-the-day generation after generation. It has more to do with governance and organizational maturity to measure real impacts.
- da39a3ee 5y ago
  That sounds truly awful. Not necessarily surprising —- but could you give us some clues as to which company this was so that we can avoid working there?
  - balsam 5y ago
    Not grandparent but that orgchart sounds like.. a (possibly government related) tech org somewhere in the Commonwealth.
  - AwaAwa 5y ago
    I've been in a few over the decades, and sounds like every tech company after the glory/startup years. Was even in one "hype" startup that began this way almost immediately.
  - TuringNYC 5y ago
    This is a common enough occurrence that you probably want to learn how to spot bad setups, where-ever they might be. I think the key is to discern Value vs Vanity. You want to be on value-add projects (those producing revenue, or reducing risk, increasing speed, or reducing cost) but not on Vanity projects. The trouble is that differentiating Vanity vs Innovation is hard. You can discern them though, in two ways I think: 1. By the level of motivation of low-level workers (true innovation is exciting) while underfunded vanity projects are soul crushing 2. By the seeming intentions of senior management -- are they more focused on the stated goal or on press/buzz? I do not have sufficient n-value to come up with hard and fast rules but i'd love to hear others' thoughts
dominotw 5y ago
I had the same comment in a post about Simple Systems Have Less Downtime (2020) https://news.ycombinator.com/item?id=28061998 https://news.ycombinator.com/item?id=28061998
urthor 5y ago
Exactly. Machine learning, and "data driven" business leadership, is being treated as a get rich quick scheme like it's low hanging fruit that's easy to do. When in fact it's been known under multiple different names for a very long time, quantitative management. And the reason it wasn't popular before is that it's very tricky to pull off.

deleted 5y ago

[deleted]

xyzzy21 5y ago

Sadly PCR tests for COVID also test positive for flu and half a dozen other causes. That's why CDC/FDA are seeking proposals for a new test that actually works! https://www.cdc.gov/csels/dls/locs/2021/07-21-2021-lab-alert-Changes_CDC_RT-PCR_SARS-CoV-2_Testing_1.html https://www.cdc.gov/csels/dls/locs/2021/07-21-2021-lab-alert...

maxerickson 5y ago
You've fallen for the internet. Please restart and try again. https://www.reuters.com/article/factcheck-covid19-pcr-test-idUSL1N2P42U5 https://www.reuters.com/article/factcheck-covid19-pcr-test-i...
- nerdponx 5y ago
  Sigh. The deniers and antivaxers will have made up their minds already, and this will just be perceived as part of the mass media coverup. It's hopeless.
ramchip 5y ago
You've been repeating this, but it doesn't seem to be true... https://news.ycombinator.com/item?id=28262833 https://news.ycombinator.com/item?id=28262833
vibrato2 5y ago
Interesting that you’re getting downvoted for the same sentiment that Kary Mullis, inventor of PCR, talked about. He said the test is not a standalone diagnostic because if the amplification is high enough you can find anything in anyone [paraphrase] For most of the pandemic, in fact roughly coinciding with release of the vaccines, the standard amplification cycle was very high. Then they reduced it. Flagged in 30s, that is also odd.
convnet 5y ago
That is simply false on a basic level. That notice by the CDC says the EXACT OPPOSITE of your comment. They're recommending that labs switch to a multiplex test that can screen for both flu and covid at the same time because PCR only detects SARS CoV 2.
- verygoodname 5y ago
  Note: the "multiplex test" is most likely still a PCR test (just 'multiplex PCR' instead of 'single-probe PCR'), so where you say "PCR only detects SARS CoV 2" it should say "the currently-used PCR test only detects SARS CoV 2".

bsanr2 5y ago

Why would anyone care to fix things? The way they are are perfectly amenable to the blame- and conclusion-laundering many ML clients seek.

TuringNYC 5y ago

I saw a large organization which was the epitome of this -- Executive Directors would propose ambitious ML projects, Directors would create plans and teams, Managers would execute on budgets, create more detailed plans, and then...someone actually needed to do the work. Because of the length of the effort, the annual compensation would already have been handed out and the EDs, Directors, Managers had already "extracted" their compensation for the project, but usually had none left for the workers who eventually needed to do the actual work. Not unexpectedly, a rough job was somehow jammed thru with understaffed, underpaid, and unmotivated low-level workers to actually "deliver" on the "AI" projects -- so victory could be declared at the top level...and new projects could begin. This isnt an ML problem, i'm sure the whole cycle has been repeated with technology-of-the-day generation after generation. It has more to do with governance and organizational maturity to measure real impacts.

da39a3ee 5y ago
That sounds truly awful. Not necessarily surprising —- but could you give us some clues as to which company this was so that we can avoid working there?
- balsam 5y ago
  Not grandparent but that orgchart sounds like.. a (possibly government related) tech org somewhere in the Commonwealth.
- AwaAwa 5y ago
  I've been in a few over the decades, and sounds like every tech company after the glory/startup years. Was even in one "hype" startup that began this way almost immediately.
- TuringNYC 5y ago
  This is a common enough occurrence that you probably want to learn how to spot bad setups, where-ever they might be. I think the key is to discern Value vs Vanity. You want to be on value-add projects (those producing revenue, or reducing risk, increasing speed, or reducing cost) but not on Vanity projects. The trouble is that differentiating Vanity vs Innovation is hard. You can discern them though, in two ways I think: 1. By the level of motivation of low-level workers (true innovation is exciting) while underfunded vanity projects are soul crushing 2. By the seeming intentions of senior management -- are they more focused on the stated goal or on press/buzz? I do not have sufficient n-value to come up with hard and fast rules but i'd love to hear others' thoughts

dominotw 5y ago

I had the same comment in a post about Simple Systems Have Less Downtime (2020) https://news.ycombinator.com/item?id=28061998 https://news.ycombinator.com/item?id=28061998

urthor 5y ago

Exactly. Machine learning, and "data driven" business leadership, is being treated as a get rich quick scheme like it's low hanging fruit that's easy to do. When in fact it's been known under multiple different names for a very long time, quantitative management. And the reason it wasn't popular before is that it's very tricky to pull off.