4 ms·

In ml basically every paper from a Western university is either useless to industry or not replicable. It is very much a crisis and one that academia has no in

In ml basically every paper from a Western university is either useless to industry or not replicable. It is very much a crisis and one that academia has no interest to solve. In a sane world we'd start with figuring out what the mystery meat in the version of pytorch used for training is and work up from there. There is zero funding for building up the required infrastructure for doing actual science instead of whatever the fuck you call the current mess. Basically anything on a more complex dataset than cifar is probably a statistical fluke because of hyper parameter tuning. To find this out you will need to spend three months building up all the bit rotten code that doesn't work from the paper. To draw the conclusion that everyone is full of shit you'll need to waste years of your life.

deleted 2y ago
[deleted]
light_hue_1 2y ago
I'm an AI/ML researcher. If this was true, then we would never have made progress in the first place. Instead, we're making astounding progress at a rate that no one really would have expected. Learning to replicate papers is a skill. It's just a matter of practice. You'll get it if you try.
- llm_trw 2y ago
  The progress isn't happening at Western universities.
  - 331c8c71 2y ago
    Why you keep emphasihing the "Western" part? Non-western are any better?
    - llm_trw 2y ago
      Yeah, I can replicate work out of a paper quite easily if half or more of the contributors have Chinese, Indian or Russian email addresses. A lot of small to medium sized companies fighting for mind share also release one script deploy pipelines for their work, which are better than the academic releases. It's western universities and big tech that releases at best bit rotten code and at worse no code what so ever.
  - light_hue_1 2y ago
    To a first order approximation all of the progress is happening at Western universities and the companies they founded. Everyone else is playing catch-up. It's hard to even name mediocre advances that come from anywhere else. As well as the entire hardware and software stack that we have. You clearly have some sort of political agenda here and not a technical argument.
    - llm_trw 2y ago
      To a first approximation all breakthroughs are ignored for years until someone else steals them. Transformers weren't scaled at google because it would eat their ad business. We could have had current level AI in 2020 if they'd been serious about it. They weren't and everyone who worked on it left when the gold rush started without having done anything meaningful in google. Computer vision, which is the field I've been working on the past few months, is even worse. Perverts in their basements are getting better than sota image recognition so their wifus have realistic vaginas, and writing tutorials which provide more working insights than any scientific paper. This isn't a new thing. I dropped out of my PhD working on something similar to AlexNet in 2009 because everyone thought I should just use gradient boosted decision trees like everyone else at the time. I couldn't get $10,000 grant to buy a machine to run the training on. It isn't just me. Yann LeCun's networks were running the postal service, but he couldn't get a paper published anywhere because it wasn't a hot area to publish in. But, yeah. Tell me more about how great Western academia is.

deleted 2y ago

[deleted]

light_hue_1 2y ago

I'm an AI/ML researcher. If this was true, then we would never have made progress in the first place. Instead, we're making astounding progress at a rate that no one really would have expected. Learning to replicate papers is a skill. It's just a matter of practice. You'll get it if you try.

llm_trw 2y ago
The progress isn't happening at Western universities.
- 331c8c71 2y ago
  Why you keep emphasihing the "Western" part? Non-western are any better?
  - llm_trw 2y ago
    Yeah, I can replicate work out of a paper quite easily if half or more of the contributors have Chinese, Indian or Russian email addresses. A lot of small to medium sized companies fighting for mind share also release one script deploy pipelines for their work, which are better than the academic releases. It's western universities and big tech that releases at best bit rotten code and at worse no code what so ever.
- light_hue_1 2y ago
  To a first order approximation all of the progress is happening at Western universities and the companies they founded. Everyone else is playing catch-up. It's hard to even name mediocre advances that come from anywhere else. As well as the entire hardware and software stack that we have. You clearly have some sort of political agenda here and not a technical argument.
  - llm_trw 2y ago
    To a first approximation all breakthroughs are ignored for years until someone else steals them. Transformers weren't scaled at google because it would eat their ad business. We could have had current level AI in 2020 if they'd been serious about it. They weren't and everyone who worked on it left when the gold rush started without having done anything meaningful in google. Computer vision, which is the field I've been working on the past few months, is even worse. Perverts in their basements are getting better than sota image recognition so their wifus have realistic vaginas, and writing tutorials which provide more working insights than any scientific paper. This isn't a new thing. I dropped out of my PhD working on something similar to AlexNet in 2009 because everyone thought I should just use gradient boosted decision trees like everyone else at the time. I couldn't get $10,000 grant to buy a machine to run the training on. It isn't just me. Yann LeCun's networks were running the postal service, but he couldn't get a paper published anywhere because it wasn't a hot area to publish in. But, yeah. Tell me more about how great Western academia is.