13 ms·

Production AI systems are hard

266 points by headalgorithm 3y ago

senttoschool 3y ago
>Should you be worried GPT5 is going to interact with social systems and destroy our society single-handedly? No absolutely not. I don't think most people are scared of GPT5. It's AGI that they're scared of. And GPT5 can destabilize a society because of how fast it could replace workers.
- ekianjo 3y ago
  Yawn. The computers were supposed to put us out of work. Then it was the internet. Now its AI. And despite all that the number of people employed keeps increasing and find new ways to thrive with technology.
  - bberrry 3y ago
    The industrial revolution replaced muscle power with machinery. AI is finally at the place where it can replace brain-power for many tasks, and will likely keep improving. I think you are underestimating the significance of this. I don't see too many horses with jobs these days.
    - maaanu 3y ago
      Currently the AI is not even able to improve my work... How should it replace me? I think the statement "AI is finally at the place where it can replace brain-power for many tasks" is ridiculous.
      - maccard 3y ago
        I don't think your statement is true - AI can definitely improve my work and has done for a while now. It lets me be more productive, but it certainly isn't capable of being left unattended or even lightly attended to. As long as you know the limits of what you're doing, tools like ChatGPT are very helpful. In about a minute, i was able to generate a skeleton app using a go framework including tests, the infrastructure as code to deploy as a lambda, a makefile build and run it, and a buildkite pipeline that will build and deploy it all. It's also correct. Now, I can't rely on it to do everything, but it can give me an app scaffold for a tech stack I'm familiar with, quicker than I can Google for it.
        maaanu 3y ago
        That´s good for you and I could give you some anecdotes with ChatGPT, were it failed horribly, e.g. were it failed to generate unit-tests for a simple api-call (about 10 loc)... I am sure the tools will only improve, but I don't understand all the hype/fear about it. ("We are fucked", "this is soo over", ... I think you will find more of those comments in older news-threads)
        maccard 3y ago
        I agree - it's disastrous when it goes wrong. If you ask it to generate the code for a lambda with 32 CPU's it will happily spout out nonsense rather than tell you it's not a valid think to request. That said, ive found it remarkably good at spitting out slightly modified boilerplate - like IDE templates on steroids. It's been a great tool, like a hammer. But not everything is a nail.
    - laratied 3y ago
      Sam Altman is saying we are going to see a shift from labor to capital and that is going to cause more economic inequality along with possibly breaking the social contract. It is probably nothing though...As if this gopher server is going to replace my TV someday. Give me a break. It only does text!
      - CatWChainsaw 3y ago
        Sam Altman is also a doomsday prepper because he anticipates societal collapse when the shift from capital to labor pushes economic equality past France 1789 levels.
  - aatd86 3y ago
    I remember seeing a paper that explained that while the a business needed 7 employees on average to reach something like $1MM in revenue in 90s, it was now 3. So I think it still has the potential to replace some people. And even with AI and AGI eventually perhaps. This is a progression. Besides, employment is a bad measure in general if one does not compare it to others such as cost of living, poverty etc... Like non farm payroll doesn't say anything if everyone is employed living with the minimum paycheck.
    - joshuahedlund 3y ago
      > I remember seeing a paper that explained that while the a business needed 7 employees on average to reach something like $1MM in revenue in 90s, it was now 3. An alternative interpretation: while in the 90’s, 7 people could only support one $1MM business, today they can support at least two. (US unemployment is lower now than it was then.)
    - rolisz 3y ago
      How much of that is because of inflation though? 1 million in 1995 corresponds to 2 million in 2023 dollars.
      - deleted 3y ago
        [deleted]
  - XorNot 3y ago
    It's the difference between "some human labor involved" and "zero". Getting to zero is very hard, and has very different ramifications then the "some" quantity. As long as "some" labor is needed, then that's going to be the entire economy - it'll simply expand along whatever constraint that "some" represents. The bizarre thing lately is all these arguments people make implying the economy is maximized. That all the things that will ever be needed are currently being produced and no new growth is possible or will ever happen (despite this literally never being the case in the entire history of the human race).
  - dclowd9901 3y ago
    As a thought experiment: - computers are able to churn through any problem more quickly and cheaply than people - machines are able to perform any tasks that humans can, more quickly and cheaply I’m curious what you think is left for people to _do_. We may not be there today, but at some point, both of those things can and will become true.
    - levihaku 3y ago
      Fantasies never come true.
- NicoJuicy 3y ago
  GPT is a tool in the toolbox. It can increase productivity, but won't replace a worker. Fyi, GPT 5 won't be coming soon and GPT 4 vs. 3.5 is not that different in terms of quality.
  - whstl 3y ago
    The main worry was never about it "replacing a worker". It's about one worker + GPT replacing two or three, because of the increased productivity gains you mentioned. I've seen it happening in some fields already. First with translation, now with copywriting and GPT-4. Hiring was frozen for those areas for a while in some companies I'm familiar with.
    - sgt101 3y ago
      Also administrative assistants and secretarial support. 30 years ago maybe 1/2 of the white collar workforce and now maybe 5% or less? Not AI - but IT.
- cookieperson 3y ago
  I'm not scared of AGI beyond how it's likely lead to societal collapse because of human greed. I am worried about the operationalization of the current chat bot technologies and other generative AI modelling techniques for harm. You don't have to have a smart system to hurt hundreds of millions of people.
- rco8786 3y ago
  > And GPT5 can destabilize a society because of how fast it could replace workers. How do you know this?
- bsenftner 3y ago
  You miss the point: reality is far, far, immensely far more complex than you, I or anyone else realizes. Attempts to use AI to replace things, people and groups and fields, are fool's errands. They will be colossal failures. Just watch, you may be even a participant: reality is more complex than we realize, and our attempts to replace it with encompassing automation will fail.
- xyzal 3y ago
  If GPT5 is to put people out of work, I hope it will make unemployable the largest percentage people in the shortest amount of time. Only then there might be a chance we will change our way of measuring an individual's value by his or hers economic output.
  - senttoschool 3y ago
    White collar workers, the ones who went to college and studied, will be the first to be unemployed. I'm guessing that these people are more employable than the rest of the population since they dedicated themselves to studying in college versus the ones who mess around in life.
quickthrower2 3y ago
This is more about production is hard as in "real life example, in a hospital" than production as in "running llama and serving yourself, vs. running llama and serving 10000 people at the same time", although I think serving lots of people something that uses a lot of compute, that people expect to be real-time is going to be hard too!
mercurialsolo 3y ago
The "feel" aspect in any domain is largely embeddings of a multi-modal information space. The challenge though with AI systems today is solving for multi-modality and providing access to datasets in closed domains. We are making strides in both. As AI's get exposed via API's and ever easier access we will see more of proliferation of this where jobs which we felt never could be AI'fied are more rapidly done so. Reminds me of the Lee Sedol vs Alpha Go game where famously it was said at the end "All but the very best Go players craft their style by imitating top players. AlphaGo seems to have totally original moves it creates itself.." I do echo though the production grade rollout of AI. When we want to jinx the game, we can notoriously play spoil sport - regulation, data protection et. al.
wouldbecouldbe 3y ago
"Radiologists, because they have a grounded brain model, only need to see a single example of a rare and obscure condition to both remember it and identify it in the future." This would actually be a long term reason to go for AI / database diagnosis. I had a personal case where a close family member, a young child, almost died. The doctors didn't understand condition and last minute it suddenly calmed down. Im sure a few doctor's in the world seen it before, but they weren't working in my hospital that week. If we can start sharing in-depth diagnosis worldwide of obscure cases using AI to make it easy to query that would probably be of great benefit.
- srvmshr 3y ago
  That's one of the biggest roadblocks: hospital & healthcare entities will not not have permissible mass sharing of data. When switching hospitals, its really hard to get a full copy of the record for single person. Try that for whole population. I may be wrong, but hospitals also could have incentives to tightly hold on to their proprietary data - where they spent a large sum of money. I worked in ML+Dermatology & getting data was a lot of regulatory check. Also, DICOM is the format for maximum interoperability from different radiology modalities & manufacturers. But annotation & markup methods vastly vary between institutions. There is no commonality or agreed upon standards on information interchange.
  - samuell 3y ago
    Federated learning (with implementations such as FEDn [1]) is supposed to solve this problem; Only sharing training weights, but not sharing data. Sure requires some coordination, but the legal parts should at least be solvable in this way. [1] https://github.com/scaleoutsystems/fedn https://github.com/scaleoutsystems/fedn
    - haldujai 3y ago
      I've been involved in a federated learning radiology project, the biggest issue is image technique and labelling. Different centers practice very differently, with different imaging protocols, disease prevalences, and labelling/reporting. This project was looking at renal masses and the only part that worked well with federated learning was image segmentation and probability that the mass is a cancer, this is a competency I expect out of a first or second year trainee. It was horrible for predicting subtype of cancer as we couldn't get a good training set (few of these lesions are biopsied, specific MRI sequences that may help are not done the same way in every center) which is what the goal was and more of an experienced generalist/subspecialty radiologist skill. Practically subtype doesn't make too much of a difference for the patient as if it's "probably a cancer" it'll just get cut out anyway, but highlights a challenge with federated learning.
siddiqi123 3y ago
[dead]
- RamblingCTO 3y ago
  Who tf started calling Hinton godfather of AI? It is not only inaccurate but unscientific. What about Minsky, Weizenbaum, Rosenblatt, Hopfield, Lenz, Hebb, Werbos, Turing, McCarthy and whoelse and whatnot. What differentiates Hinton from all the rest? The truth is that all of science is an amalgamation of past achievements, many so small and unheard of that it's just so very very wrong to pick one person and call them godfather. Why Hinton? Because DL had good marketing 10 years back?
  - sgt101 3y ago
    >Minsky, Weizenbaum, Rosenblatt, Hopfield, Lenz, Hebb, Werbos, Turing, McCarthy and whoelse and whatnot Sounds like that Queens of the Stone Age song.. https://open.spotify.com/track/3DaXIGJm0BCEB9X7zHTRfI?si=98b99a10c9104da2 https://open.spotify.com/track/3DaXIGJm0BCEB9X7zHTRfI?si=98b...
  - ftxbro 3y ago
    > What about Minsky Minsky slowed down AI by using his powerful rizz to hypnotize otherwise reasonable researchers and academics into believing his self serving assertion that multilayer perceptron systems are bad because single layer perceptron systems are bad.
    - levihaku 3y ago
      Multilayer perceptrons literally are bad. Machine Learning industry is a complete joke because trivial stuff like this is all it has. It would honestly take less time for biologists to trap a brain in a jar and force it to do things than anyone in this joke of an industry will create real intelligent algorithm that can think for itself. Oh wait, I also forgot to mention that in our politically correct clown society, free thinking is illegal so AI by literal definition is illegal in the first place because it may just happen that it starts to be a little bit racist despite otherwise being amazingly intelligent and better than humans at solving certain tasks.
tikkun 3y ago
> I interviewed and hired 25 radiologists, whose primary and chief complaint was that they had to reboot their computers several times a day.. Yes. There is still so much low hanging fruit for software everywhere.
- wouldbecouldbe 3y ago
  Thats more an IT management problem then a software issue.
  - TeMPOraL 3y ago
    IT management usually is the problem.
    - sgt101 3y ago
      It's often a software problem too. It's pretty easy to ship software with memory leaks - especially when there are few users, limited money for testing and technically challenging tasks - such as large images that need to be manipulated.
      - bick_nyers 3y ago
        The testing overhead for Dicom is immense to do properly. There is a very high chance it is a software problem, especially if dealing with tomos/mammos (or even x-ray) which are incredibly resource hungry.
- itissid 3y ago
  Was that due to bad software crashing the os ?
  - malikNF 3y ago
    Smells like a memory leak.
OthmaneHamzaoui 3y ago
On point ! Been working on putting AI/ML systems in production for various large companies in the last 5 years and every time the ML part of the system is just the tip of the iceberg. System integrations and user adoption are two main big components that had to be tackled before the AI system was in production. I think the natural excitement we have with any new AI model (or technology) leads us to assume that it will magically get integrated in any existing system and adopted by any user.
amelius 3y ago
Even harder is using them when the vendor keeps pushing updates that may or may not work.
mensetmanusman 3y ago
Healthcare is fascinating. The quote that nothing will be implemented that possibly harms patients is false, because harm is complicated. Sharing patient data harms privacy, but in the long run more data sharing is probably the best route for reducing physical harm. The body is just too complicated for individuals to fully grok.
- neon_electro 3y ago
  Do you believe in a middle ground of informed consent such that data can be shared while mitigating the privacy harms? It allows each patient to make their own decisions, and just like volunteering to be an organ donor, there can be societal benefit without forcing harm on people who do not want to participate.
  - thegrimmest 3y ago
    I don't. I think sharing data should be a mandatory part of receiving services. In order to benefit from the system you must commit to continue to improve it.
  - levihaku 3y ago
    I believe that if I cared more about privacy than my life, I wouldn't go to a public institution full of people I objectively cannot trust because I don't personally know them and then put my life in their hands. Privacy schizos need to stop making up nonsense. All of your diagnosed illnesses are permanently stored in a database that will never ever be deleted until decades after your death. This data is already shared between all hospitals. The moment you step through the hospital door you aren't at home and your privacy fantasies end. Literally any malicious worker could leak your data and then what? Why does this even matter. If I see you in a street, I can tell you're sickly just from the direction you're walking in.
- skybrian 3y ago
  > Sharing patient data harms privacy Yes, that's a very abstract statement that seems plausible and is easy to make, but how do we know there's significant harm? What are some real-world examples? Can we quantify how much harm is done to patients from sharing too much, versus sharing too little?
jgalt212 3y ago
The only general distribution production AI system (post ChatGPT) I'm aware of the that "works" decently is Grammarly Go. What others are out there?
- potatoman22 3y ago
  Google search
  - xpe 3y ago
    This conversation is unfolding nicely. Sorry for the sarcasm... I'll say what I mean: I think we can strive to treat even poorly phrased questions as an opportunity. Narrowly, yes, Google search seems to be behaving roughly as well (or poorly, depending on your point of view) nowadays. But I get the feeling saying "Google search" isn't a helpful response to the person asking the question.
    - potatoman22 3y ago
      Helpful response: there are thousands of productionized AI systems that have existed long before ChatGPT. For example, google search.
      - xpe 3y ago
        The context of the question suggests the questioner is talking about AI technology at the level of ChatGPT —- large language models —- and the associated technological complexities around them. Subtopics might include differences between managing state and sessions of a text search engine versus a large language model
- xpe 3y ago
  What do you mean by "work decently"? What kinds of behavior are you interested in? (i.e. What are you hoping to learn from this conversation?) If I were to guess, I'd probably think you are interested in production-level issues, rather than limitations of these "AI" technologies that have nothing to do with typing software scaling issues?
crosen99 3y ago
"Radiologists are not performing 2d pattern recognition - they have a 3d world model of the brain and its physical dynamics in their head. The motion and behavior of their brain to various traumas informs their prediction of hemorrhage determination." Radiologists are certainly performing 2d pattern recognition, as the input they are processing is only 2d even if the details of how that recognition is performed relies on some deeper understanding. Likewise, an AI system performs recognition of 2d patterns based on some deeper "understanding" - in the case of a neural net that "understanding" lies in the complex configuration and weightings of its neural connections built up from vast training datasets. Even if this dataset is only a subset of the dataset that humans are trained on, we still can't a priori claim that this subset lacks patterns and correlations that escape humans and that allow an AI to make certain determinations better than a human might.
ghm2180 3y ago
When you ask interview questions like: How would you design the ML stack that can recommend a restaurant given their past restaurant going history? Most candidates don't get into the craziness of things like bias in datasets or calibration of probability output of models. They dive straight into embeddings(they have the freedom in the interview to design something the are comfortable with), which is important to the quality but not necessarily the most(or even the only) challenging part of what the team works on.
aabajian 3y ago
I'm just finishing interventional radiology training and I moonlight as a diagnostic radiologist (not to mention having an undergrad/master's in computer science). Almost 90% of the diagnostic studies I read could be pre-drafted by AI. That's where the money is and where AI-in-radiology companies should focus. The money is not in detecting hemorrhage or pulmonary embolism. It's a classic fallacy to think that life-saving means money-saving. Rather, the money in radiology is reads per day. Here's a user story: A private practice radiologist reads 20 abdomen and pelvis CT scans with contrast per day. In each of these studies, he must write a short description of each organ. For example, "Gall bladder: Unremarkable" or "Gallbladder: Cholecystitis without evidence of cholecystitis" or "Gallbladder: Dependent sludge." There are around 15 such organs (liver, gallbladder, pancreas, spleen, adrenal glands, kidneys, etc.). The AI should auto-populate the radiologist report with an appropriate description of each organ system. The job of the radiologist is to confirm what the AI says in each section, and to go into further detail as needed. It's essentially just customizing the existing template to each patient. This type of pre-drafting is exactly what radiology residents do and what companies like vRad do.
- haldujai 3y ago
  What you’re describing is what a template does, I just dictate “stone” or “sludge” from a pick list and never a normal statement. There is also existing technology that creates a structured report from free dictations. Your math also doesn’t add up because 90%ish of studies are normal so you can basically sign a template without looking at the pictures and be more or less entirely accurate, before adding AI. Also what practice only reads 20 abdo CTs a day? Seriously, let me know I could use a more relaxing job. Lastly, an AI that functions with the competency of a PGY3 resident does not exist at the moment or on the horizon. Especially for cross sectional studies. Source: am a diagnostic radiologist and AI researcher (on the NLP side).
- throwaway85858 3y ago
  this! I asked a neurologist and she said at least 40% of her time is spent on formulaic discharge letters and rounds documentation.
- bee_rider 3y ago
Ozzie_osman 3y ago
> Geoffrey Miller was one of the loudest voices decrying the decline of radiology 5 years, and now he’s crying fear for new AI systems. I don't know who Geoffrey Miller I'm pretty sure if there's a Geoffrey who notably predicted the decline of radiology, it was Geoffrey Hinton a few years ago...
- xpe 3y ago
  I did a few minutes of research. I did not find any such person. So I'm inclined to agree; the author may have meant Geoffrey Hinton. See also: https://statmodeling.stat.columbia.edu/2021/06/07/ai-promised-to-revolutionize-radiology-but-so-far-its-failing/ https://statmodeling.stat.columbia.edu/2021/06/07/ai-promise... > Gary Smith points us to [this news article][1]: > Geoffrey Hinton is a legendary computer scientist . . . Naturally, people paid attention when Hinton declared in 2016, “We should stop training radiologists now, it’s just completely obvious within five years deep learning is going to do better than radiologists.” The US Food and Drug Administration (FDA) approved the first AI algorithm for medical imaging that year and there are now more than 80 approved algorithms in the US and a similar number in Europe. [1] https://qz.com/2016153/ai-promised-to-revolutionize-radiology-but-so-far-its-failing https://qz.com/2016153/ai-promised-to-revolutionize-radiolog... > Geoffrey Hinton is a legendary computer scientist. When Hinton, Yann LeCun, and Yoshua Bengio were given the 2018 Turing Award, considered the Nobel prize of computing, they were described as the “Godfathers of artificial intelligence” and the “Godfathers of Deep Learning.” Naturally, people paid attention when Hinton declared in 2016, “We should stop training radiologists now, it’s just completely obvious within five years deep learning is going to do better than radiologists.” The US Food and Drug Administration (FDA) approved the first AI algorithm for medical imaging that year and there are now more than 80 approved algorithms in the US and a similar number in Europe. > Yet, the number of radiologists working in the US has gone up, not down, increasing by about 7% between 2015 and 2019. Indeed, there is now a shortage of radiologists that is predicted to increase over the next decade.
- dang 3y ago
  Looks like the OP has fixed that now.
m3kw9 3y ago
To start, small things like getting the json format to work 100% of the time is impossible.
- xpe 3y ago
  Me: "Are you speaking of JSON in the context of radiology systems? medical systems? enterprise systems? software in general?" You: Yes.
varelse 3y ago
[dead]
outside1234 3y ago
A customer I am working with can’t even set up CI/CD with GitHub. I have no doubt that AI is not going to kill us because 95% of companies won’t even be able to build it.
- xpe 3y ago
  > I have no doubt that AI is not going to kill us because 95% of companies won’t even be able to build it. I take your joke, but the argument doesn't hold water for many reasons. Here are only two: 1. Even if only 5% of companies can build something dangerous, that's more than enough. 2. The "AI" doesn't have to directly kill us; we're capable of doing that ourselves with only a little bit of informational, societal, and/or economic degradation.
- xpe 3y ago
  > A customer I am working with can’t even set up CI/CD with GitHub. Can you please characterize your customer without calling them out by name? Also, can you clarify what you mean by "can't"? Under what time frame? With what background? With what else on their plate? In other words, why do you expect your customer to be able to setup CI/CD with GitHub? Maybe this seems reasonable to you, but maybe you are overlooking the hundreds of little things that make it seem easy to you, not least of which is the patience and ability to wade through tasks with many steps. I'm genuinely curious. Are you suggesting they lack capability (such as intelligence or skills), motivation, and/or something else?
- jacurtis 3y ago
  It is because, especially in tech, most people can't even vocalize what they want to the detail that an AI could generate it. Even in my day-job (which is probably similar to yours), most people give me the wrong information when asking for a solution and if I blindly built what they think they needed, they wouldn't get where they wanted.
xpe 3y ago
Caveated [1] or not, the "concluding" paragraphs [2] are not a summary of the article. Neither are they well-supported nor convincing; they are sweeping, unrelated generalizations. Notes: 1. The caveat at the top states "NOTE: This post is not up to my normal writing standard, just felt compelled to get this down in some form. This is more like a blog post entry than a real newsletter addition." 2. The article's final paragraphs are: "Long story short / Thinkers have a pattern where they are so divorced from implementation details that applications seem trivial, when in reality, the small details are exactly where value accrues. / Should you be worried about GPT5 being used to automate vulnerability detection on websites before they’re patched? Maybe. / Should you be worried GPT5 is going to interact with social systems and destroy our society single-handedly? No absolutely not." P.S. Pedantic timestamp: the quotes above are taken on 12:24 pm eastern time, May 29. Well, this may seem pedantic until the author updates the blog post and then this comment seems erroneous. It is notable that we still don't have a well-accepted standard for snapshotted content.
ftxbro 3y ago
this sounds like the kind of cope that gets written before production ai starts going down the list replacing jobs
- CatWChainsaw 3y ago
  tbf your handle is on point
jameshart 3y ago
Wait a minute.... Because the role of radiologists is not as simple as trained classification, we can conclude that "No, AGI isn't going to take over every social system when GPT5 comes out"? Author starts off by saying that you can't generalize conclusions based on observations in a limited domain: > Geoff made a classic error that technologists often make, which is to observe a particular behavior (identifying some subset of radiology scans correctly) against some task (identifying hemorrhage on CT head scans correctly), and then to extrapolate based on that task alone. Author then goes on to suggest that there are a number of issues with radiology which AI will not solve. Then, based on this lack of ML's complete coverage of the aspects of radiology that he finds interesting, he then extrapolates: > Should you be worried GPT5 is going to interact with social systems and destroy our society single-handedly? No absolutely not. I don't see how the conclusion remotely follows from the argument. And more to the point, the refuting argument is embodied in the article itself.
- skybrian 3y ago
  Yes, it goes a bit far, but the basic problem is treating this as all-or-nothing: Either the article is true or false. Either AI is going to take over, or it will have little effect. I believe the article raises relevant issues about why deploying AI-based systems is harder and will take longer than it might appear. Another relevant example might be driverless cars. It's taking many years and deployment is still quite limited. Some companies gave up. But I'm not going to count Waymo out yet. Similarly, it's possible that Hinton got the timeline wrong for radiology, but it might still have significant affects on radiologist employment in the end. (Note that this sort of reasoning by analogy is useful for imagining plausible scenarios, but not for ruling things out.)
  - naijaboiler 3y ago
    Driving is another place where people wrongfully think that solving the technical problem means the problem has been solved. Driving is a social problem. We are a long long way from solving it.
- naijaboiler 3y ago
godelski 3y ago
At the root of this is something I'm constantly saying to other members in my research lab, arguing with reviewers, and passionately teaching my students: datasets are proxies, measurements are fuzzy. This is something that is drilled into students learning statistics and I can't for the life of me figure out why this changed with respect to ML (best guess is not requiring statistics courses). Datasets are proxies: they represent the world, but aren't. They should generally be seen as narrow subsets too. Your dataset quality and type matter a lot! Things like medical image datasets also have tons of correlating factors that can easily invalidate all results without you being aware. There are simple datasets we use to prove a concept (toys, mnist, cifar, etc). There are large scale datasets that have internal inconsistencies (imagenet, flowers). There are huge datasets that haven't been properly filtered/deduplicated (LAION). (there are also just shitty datasets (HumanEval)) Thinking of datasets as proxies helps internalize the frustration that literally every production engineer faces (even outside ML and software): real world results are inconsistent with lab results. Dataset engineering is an underappreciated art that is extremely difficult. But everyone needs to internalize that datasets are just a map, not the territory, and your navigation will only be as good as the map (many are poorly drawn maps, often on purpose) Measurements are fuzzy: Benchmarkism is running rampant in the ML world and it baffles me that a field who's bottomline objective deals with alignment isn't able to align how we evaluate ourselves. No measurement is perfect, many are far from them. You can train two language models to the same NLL and one might sample well and the other outputs garbage. You can train two image models to have identical FIDs and one samples clearly and the other doesn't. Likelihood also doesn't guarantee sharpness and I can go on. You must think about the limitations of your measurements and know them in depth. This also has seem to have gotten away from us and people just use the measurement libraries and call it that. We've reached a point where ImageNet classification accuracy has decoupled from downstream performance (object detection and segmentation), and things like this are confusing to production people because just taking the model with the highest score doesn't always result in the best performing model for their work (even before we consider things like throughput and memory usage). It is a Goodhart problem through and through. ML is at a serious point where we've gotten away from our basic stats learning. That's going to pose a real danger to society, not AGI. It's like handing powertools to chimps (who don't know how powertools work), it won't end well. But that is happening because we've shifted focus to meet targets, not to measure our work. Targets are easy, science is hard. Unless we bring these nuances back to our evaluation of works then we are just handing powertools to chimps without any quality assurance.
w10-1 3y ago
Brain MRI's (in all their forms) are a whole different species than all other MRI's, due to variability and softness of structures, differences in condition presentation/time-frame, and and the high incidence of brain MRI's in the elderly (the brain can shrink precipitously after 60). Also, neurologists are quite specialized (e.g., stroke vs. MS vs. acute encephalopathy vs. optic neuritis). In the majority of cases, the specialist neurologist is better at reading the MRI of their target conditions than the general radiologist. This is handled by the neurologist phoning the radiologist and guiding them to re-work their findings. The vast majority of AI in software is in radiology. Decades ago radiology was among the first practice groups to fall under private equity patterns because of the cost of the machines. In some cases, specific radiology groups develop protectable expertise around some protocols (i.e., how to get the machine to give better data), but long machine lifecycle creates opportunities for technical one-upmanship. Knowing that the labeling quality is variable and the images are not entirely comparable, you realize the bulk of AI-radiology may be built on shifting sands, and amounts to workflow optimization. Worse, the effort could go into addressing what really plagues all providers: the unusability of their EHR's. But that's not particularly monetizable relative to critical diagnostics, and harder than production AI for MRI.
guillemsola 3y ago
Aren't these complex as any other piece of software that is looking to provide value to users? I've been developing software solutions for several years and the fact that a computer can do cool stuff, automate processes, be more accurate... always need users who know and want to use it. So my takeway is that with AI solutions the human part also needs to be considered.
ugh123 3y ago
I don't think speaking to Radiologists as "advisors" to an AI system is going to be productive in the grand scheme of things. Yes, radiologists (and all doctors for that matter) will give their thoughts on making their job incrementally easier. But that doesn't necessarily equate to better healthcare. Is that going to move the needle on better healthcare for patients? Will it allow our hospital systems to bring in more throughput of patients at higher quality of care and lower costs? Thats not what doctors (and hospitals) want. They won't be an ally in "AI for healthcare" unless it means their paychecks/revenues are protected. The author is being misguided by seemingly smart people who have skin in the game and a lot to lose. Personally, I only favor AI solutions that will drive down the cost of all healthcare, raise the quality of service, and overall expand high-quality healthcare to underserved populations. Anything less than that is only serving the needs of the industry and not patients.
steve76 3y ago
[dead]
YeGoblynQueenne 3y ago
>> Geoffrey Hinton was one of the loudest voices decrying the decline of radiology 5 years, and now he’s crying fear for new AI systems. It's worth remembering exactly what Geoff Hinton said, and how he said it: “I think that if you work as a radiologist you are like Wile E. Coyote in the cartoon,” Hinton told me. “You’re already over the edge of the cliff, but you haven’t yet looked down. There’s no ground underneath.” Deep-learning systems for breast and heart imaging have already been developed commercially. “It’s just completely obvious that in five years deep learning is going to do better than radiologists,” he went on. “It might be ten years. I said this at a hospital. It did not go down too well.” https://www.newyorker.com/magazine/2017/04/03/ai-versus-md https://www.newyorker.com/magazine/2017/04/03/ai-versus-md So Hinton didn't "decry the decline of radiology". His comment brims with exuberant sarcasm about how his chosen approach to AI is about to not only leave radiologists without a job, but also leave them looking like the short-sighted fools he thought they surely were. Hinton was cock-sure and arrogant, like an undergraduate student who just trained his first neural net. That is quite unbecoming of the man whom the New York Times, The Guardian, and many other publications have been calling "the godfather of AI", and a Turing award winner, to boot. That is something he should be called out on, and not allowed to forget. Nor he, nor anyone else who thinks that leaving highly-trained workers without a job is a laughing matter. Nobody should be allowed to make fun of the harmful consequences of the technology they create.
- cicce19 3y ago
  Hinton has been crucified many times in radiology circles for his statements. It honestly feels like we've beaten a dead horse on this. Clearly he was wrong and did not understand the complexity of the field. Although he was wrong, his statements today (through the advent of LLMs) are less insane than they were even 7 months ago and directionally he saw the progress of the technology. His magnitude/timeline was off and time will tell if he is one day proven right.

266 points by headalgorithm 3y ago

>Should you be worried GPT5 is going to interact with social systems and destroy our society single-handedly? No absolutely not. I don't think most people are scared of GPT5. It's AGI that they're scared of. And GPT5 can destabilize a society because of how fast it could replace workers.

ekianjo 3y ago
Yawn. The computers were supposed to put us out of work. Then it was the internet. Now its AI. And despite all that the number of people employed keeps increasing and find new ways to thrive with technology.
- bberrry 3y ago
  The industrial revolution replaced muscle power with machinery. AI is finally at the place where it can replace brain-power for many tasks, and will likely keep improving. I think you are underestimating the significance of this. I don't see too many horses with jobs these days.
  - maaanu 3y ago
    Currently the AI is not even able to improve my work... How should it replace me? I think the statement "AI is finally at the place where it can replace brain-power for many tasks" is ridiculous.
    - maccard 3y ago
      I don't think your statement is true - AI can definitely improve my work and has done for a while now. It lets me be more productive, but it certainly isn't capable of being left unattended or even lightly attended to. As long as you know the limits of what you're doing, tools like ChatGPT are very helpful. In about a minute, i was able to generate a skeleton app using a go framework including tests, the infrastructure as code to deploy as a lambda, a makefile build and run it, and a buildkite pipeline that will build and deploy it all. It's also correct. Now, I can't rely on it to do everything, but it can give me an app scaffold for a tech stack I'm familiar with, quicker than I can Google for it.
      - maaanu 3y ago
        That´s good for you and I could give you some anecdotes with ChatGPT, were it failed horribly, e.g. were it failed to generate unit-tests for a simple api-call (about 10 loc)... I am sure the tools will only improve, but I don't understand all the hype/fear about it. ("We are fucked", "this is soo over", ... I think you will find more of those comments in older news-threads)
        maccard 3y ago
        I agree - it's disastrous when it goes wrong. If you ask it to generate the code for a lambda with 32 CPU's it will happily spout out nonsense rather than tell you it's not a valid think to request. That said, ive found it remarkably good at spitting out slightly modified boilerplate - like IDE templates on steroids. It's been a great tool, like a hammer. But not everything is a nail.
  - laratied 3y ago
    Sam Altman is saying we are going to see a shift from labor to capital and that is going to cause more economic inequality along with possibly breaking the social contract. It is probably nothing though...As if this gopher server is going to replace my TV someday. Give me a break. It only does text!
    - CatWChainsaw 3y ago
      Sam Altman is also a doomsday prepper because he anticipates societal collapse when the shift from capital to labor pushes economic equality past France 1789 levels.
- aatd86 3y ago
  I remember seeing a paper that explained that while the a business needed 7 employees on average to reach something like $1MM in revenue in 90s, it was now 3. So I think it still has the potential to replace some people. And even with AI and AGI eventually perhaps. This is a progression. Besides, employment is a bad measure in general if one does not compare it to others such as cost of living, poverty etc... Like non farm payroll doesn't say anything if everyone is employed living with the minimum paycheck.
  - joshuahedlund 3y ago
    > I remember seeing a paper that explained that while the a business needed 7 employees on average to reach something like $1MM in revenue in 90s, it was now 3. An alternative interpretation: while in the 90’s, 7 people could only support one $1MM business, today they can support at least two. (US unemployment is lower now than it was then.)
  - rolisz 3y ago
    How much of that is because of inflation though? 1 million in 1995 corresponds to 2 million in 2023 dollars.
    - deleted 3y ago
      [deleted]
- XorNot 3y ago
  It's the difference between "some human labor involved" and "zero". Getting to zero is very hard, and has very different ramifications then the "some" quantity. As long as "some" labor is needed, then that's going to be the entire economy - it'll simply expand along whatever constraint that "some" represents. The bizarre thing lately is all these arguments people make implying the economy is maximized. That all the things that will ever be needed are currently being produced and no new growth is possible or will ever happen (despite this literally never being the case in the entire history of the human race).
- dclowd9901 3y ago
  As a thought experiment: - computers are able to churn through any problem more quickly and cheaply than people - machines are able to perform any tasks that humans can, more quickly and cheaply I’m curious what you think is left for people to _do_. We may not be there today, but at some point, both of those things can and will become true.
  - levihaku 3y ago
    Fantasies never come true.
NicoJuicy 3y ago
GPT is a tool in the toolbox. It can increase productivity, but won't replace a worker. Fyi, GPT 5 won't be coming soon and GPT 4 vs. 3.5 is not that different in terms of quality.
- whstl 3y ago
  The main worry was never about it "replacing a worker". It's about one worker + GPT replacing two or three, because of the increased productivity gains you mentioned. I've seen it happening in some fields already. First with translation, now with copywriting and GPT-4. Hiring was frozen for those areas for a while in some companies I'm familiar with.
  - sgt101 3y ago
    Also administrative assistants and secretarial support. 30 years ago maybe 1/2 of the white collar workforce and now maybe 5% or less? Not AI - but IT.
cookieperson 3y ago
I'm not scared of AGI beyond how it's likely lead to societal collapse because of human greed. I am worried about the operationalization of the current chat bot technologies and other generative AI modelling techniques for harm. You don't have to have a smart system to hurt hundreds of millions of people.
rco8786 3y ago
> And GPT5 can destabilize a society because of how fast it could replace workers. How do you know this?
bsenftner 3y ago
You miss the point: reality is far, far, immensely far more complex than you, I or anyone else realizes. Attempts to use AI to replace things, people and groups and fields, are fool's errands. They will be colossal failures. Just watch, you may be even a participant: reality is more complex than we realize, and our attempts to replace it with encompassing automation will fail.
xyzal 3y ago
If GPT5 is to put people out of work, I hope it will make unemployable the largest percentage people in the shortest amount of time. Only then there might be a chance we will change our way of measuring an individual's value by his or hers economic output.
- senttoschool 3y ago
  White collar workers, the ones who went to college and studied, will be the first to be unemployed. I'm guessing that these people are more employable than the rest of the population since they dedicated themselves to studying in college versus the ones who mess around in life.

quickthrower2 3y ago

This is more about production is hard as in "real life example, in a hospital" than production as in "running llama and serving yourself, vs. running llama and serving 10000 people at the same time", although I think serving lots of people something that uses a lot of compute, that people expect to be real-time is going to be hard too!

mercurialsolo 3y ago

The "feel" aspect in any domain is largely embeddings of a multi-modal information space. The challenge though with AI systems today is solving for multi-modality and providing access to datasets in closed domains. We are making strides in both. As AI's get exposed via API's and ever easier access we will see more of proliferation of this where jobs which we felt never could be AI'fied are more rapidly done so. Reminds me of the Lee Sedol vs Alpha Go game where famously it was said at the end "All but the very best Go players craft their style by imitating top players. AlphaGo seems to have totally original moves it creates itself.." I do echo though the production grade rollout of AI. When we want to jinx the game, we can notoriously play spoil sport - regulation, data protection et. al.

wouldbecouldbe 3y ago

"Radiologists, because they have a grounded brain model, only need to see a single example of a rare and obscure condition to both remember it and identify it in the future." This would actually be a long term reason to go for AI / database diagnosis. I had a personal case where a close family member, a young child, almost died. The doctors didn't understand condition and last minute it suddenly calmed down. Im sure a few doctor's in the world seen it before, but they weren't working in my hospital that week. If we can start sharing in-depth diagnosis worldwide of obscure cases using AI to make it easy to query that would probably be of great benefit.

srvmshr 3y ago
That's one of the biggest roadblocks: hospital & healthcare entities will not not have permissible mass sharing of data. When switching hospitals, its really hard to get a full copy of the record for single person. Try that for whole population. I may be wrong, but hospitals also could have incentives to tightly hold on to their proprietary data - where they spent a large sum of money. I worked in ML+Dermatology & getting data was a lot of regulatory check. Also, DICOM is the format for maximum interoperability from different radiology modalities & manufacturers. But annotation & markup methods vastly vary between institutions. There is no commonality or agreed upon standards on information interchange.
- samuell 3y ago
  Federated learning (with implementations such as FEDn [1]) is supposed to solve this problem; Only sharing training weights, but not sharing data. Sure requires some coordination, but the legal parts should at least be solvable in this way. [1] https://github.com/scaleoutsystems/fedn https://github.com/scaleoutsystems/fedn
  - haldujai 3y ago
    I've been involved in a federated learning radiology project, the biggest issue is image technique and labelling. Different centers practice very differently, with different imaging protocols, disease prevalences, and labelling/reporting. This project was looking at renal masses and the only part that worked well with federated learning was image segmentation and probability that the mass is a cancer, this is a competency I expect out of a first or second year trainee. It was horrible for predicting subtype of cancer as we couldn't get a good training set (few of these lesions are biopsied, specific MRI sequences that may help are not done the same way in every center) which is what the goal was and more of an experienced generalist/subspecialty radiologist skill. Practically subtype doesn't make too much of a difference for the patient as if it's "probably a cancer" it'll just get cut out anyway, but highlights a challenge with federated learning.

siddiqi123 3y ago

[dead]

RamblingCTO 3y ago
Who tf started calling Hinton godfather of AI? It is not only inaccurate but unscientific. What about Minsky, Weizenbaum, Rosenblatt, Hopfield, Lenz, Hebb, Werbos, Turing, McCarthy and whoelse and whatnot. What differentiates Hinton from all the rest? The truth is that all of science is an amalgamation of past achievements, many so small and unheard of that it's just so very very wrong to pick one person and call them godfather. Why Hinton? Because DL had good marketing 10 years back?
- sgt101 3y ago
  >Minsky, Weizenbaum, Rosenblatt, Hopfield, Lenz, Hebb, Werbos, Turing, McCarthy and whoelse and whatnot Sounds like that Queens of the Stone Age song.. https://open.spotify.com/track/3DaXIGJm0BCEB9X7zHTRfI?si=98b99a10c9104da2 https://open.spotify.com/track/3DaXIGJm0BCEB9X7zHTRfI?si=98b...
- ftxbro 3y ago
  > What about Minsky Minsky slowed down AI by using his powerful rizz to hypnotize otherwise reasonable researchers and academics into believing his self serving assertion that multilayer perceptron systems are bad because single layer perceptron systems are bad.
  - levihaku 3y ago
    Multilayer perceptrons literally are bad. Machine Learning industry is a complete joke because trivial stuff like this is all it has. It would honestly take less time for biologists to trap a brain in a jar and force it to do things than anyone in this joke of an industry will create real intelligent algorithm that can think for itself. Oh wait, I also forgot to mention that in our politically correct clown society, free thinking is illegal so AI by literal definition is illegal in the first place because it may just happen that it starts to be a little bit racist despite otherwise being amazingly intelligent and better than humans at solving certain tasks.

tikkun 3y ago

> I interviewed and hired 25 radiologists, whose primary and chief complaint was that they had to reboot their computers several times a day.. Yes. There is still so much low hanging fruit for software everywhere.

wouldbecouldbe 3y ago
Thats more an IT management problem then a software issue.
- TeMPOraL 3y ago
  IT management usually is the problem.
  - sgt101 3y ago
    It's often a software problem too. It's pretty easy to ship software with memory leaks - especially when there are few users, limited money for testing and technically challenging tasks - such as large images that need to be manipulated.
    - bick_nyers 3y ago
      The testing overhead for Dicom is immense to do properly. There is a very high chance it is a software problem, especially if dealing with tomos/mammos (or even x-ray) which are incredibly resource hungry.
itissid 3y ago
Was that due to bad software crashing the os ?
- malikNF 3y ago
  Smells like a memory leak.

OthmaneHamzaoui 3y ago

On point ! Been working on putting AI/ML systems in production for various large companies in the last 5 years and every time the ML part of the system is just the tip of the iceberg. System integrations and user adoption are two main big components that had to be tackled before the AI system was in production. I think the natural excitement we have with any new AI model (or technology) leads us to assume that it will magically get integrated in any existing system and adopted by any user.

amelius 3y ago

Even harder is using them when the vendor keeps pushing updates that may or may not work.

mensetmanusman 3y ago

Healthcare is fascinating. The quote that nothing will be implemented that possibly harms patients is false, because harm is complicated. Sharing patient data harms privacy, but in the long run more data sharing is probably the best route for reducing physical harm. The body is just too complicated for individuals to fully grok.

neon_electro 3y ago
Do you believe in a middle ground of informed consent such that data can be shared while mitigating the privacy harms? It allows each patient to make their own decisions, and just like volunteering to be an organ donor, there can be societal benefit without forcing harm on people who do not want to participate.
- thegrimmest 3y ago
  I don't. I think sharing data should be a mandatory part of receiving services. In order to benefit from the system you must commit to continue to improve it.
- levihaku 3y ago
  I believe that if I cared more about privacy than my life, I wouldn't go to a public institution full of people I objectively cannot trust because I don't personally know them and then put my life in their hands. Privacy schizos need to stop making up nonsense. All of your diagnosed illnesses are permanently stored in a database that will never ever be deleted until decades after your death. This data is already shared between all hospitals. The moment you step through the hospital door you aren't at home and your privacy fantasies end. Literally any malicious worker could leak your data and then what? Why does this even matter. If I see you in a street, I can tell you're sickly just from the direction you're walking in.
skybrian 3y ago
> Sharing patient data harms privacy Yes, that's a very abstract statement that seems plausible and is easy to make, but how do we know there's significant harm? What are some real-world examples? Can we quantify how much harm is done to patients from sharing too much, versus sharing too little?

jgalt212 3y ago

The only general distribution production AI system (post ChatGPT) I'm aware of the that "works" decently is Grammarly Go. What others are out there?

potatoman22 3y ago
Google search
- xpe 3y ago
  This conversation is unfolding nicely. Sorry for the sarcasm... I'll say what I mean: I think we can strive to treat even poorly phrased questions as an opportunity. Narrowly, yes, Google search seems to be behaving roughly as well (or poorly, depending on your point of view) nowadays. But I get the feeling saying "Google search" isn't a helpful response to the person asking the question.
  - potatoman22 3y ago
    Helpful response: there are thousands of productionized AI systems that have existed long before ChatGPT. For example, google search.
    - xpe 3y ago
      The context of the question suggests the questioner is talking about AI technology at the level of ChatGPT —- large language models —- and the associated technological complexities around them. Subtopics might include differences between managing state and sessions of a text search engine versus a large language model
xpe 3y ago
What do you mean by "work decently"? What kinds of behavior are you interested in? (i.e. What are you hoping to learn from this conversation?) If I were to guess, I'd probably think you are interested in production-level issues, rather than limitations of these "AI" technologies that have nothing to do with typing software scaling issues?

crosen99 3y ago

"Radiologists are not performing 2d pattern recognition - they have a 3d world model of the brain and its physical dynamics in their head. The motion and behavior of their brain to various traumas informs their prediction of hemorrhage determination." Radiologists are certainly performing 2d pattern recognition, as the input they are processing is only 2d even if the details of how that recognition is performed relies on some deeper understanding. Likewise, an AI system performs recognition of 2d patterns based on some deeper "understanding" - in the case of a neural net that "understanding" lies in the complex configuration and weightings of its neural connections built up from vast training datasets. Even if this dataset is only a subset of the dataset that humans are trained on, we still can't a priori claim that this subset lacks patterns and correlations that escape humans and that allow an AI to make certain determinations better than a human might.

ghm2180 3y ago

When you ask interview questions like: How would you design the ML stack that can recommend a restaurant given their past restaurant going history? Most candidates don't get into the craziness of things like bias in datasets or calibration of probability output of models. They dive straight into embeddings(they have the freedom in the interview to design something the are comfortable with), which is important to the quality but not necessarily the most(or even the only) challenging part of what the team works on.

aabajian 3y ago

I'm just finishing interventional radiology training and I moonlight as a diagnostic radiologist (not to mention having an undergrad/master's in computer science). Almost 90% of the diagnostic studies I read could be pre-drafted by AI. That's where the money is and where AI-in-radiology companies should focus. The money is not in detecting hemorrhage or pulmonary embolism. It's a classic fallacy to think that life-saving means money-saving. Rather, the money in radiology is reads per day. Here's a user story: A private practice radiologist reads 20 abdomen and pelvis CT scans with contrast per day. In each of these studies, he must write a short description of each organ. For example, "Gall bladder: Unremarkable" or "Gallbladder: Cholecystitis without evidence of cholecystitis" or "Gallbladder: Dependent sludge." There are around 15 such organs (liver, gallbladder, pancreas, spleen, adrenal glands, kidneys, etc.). The AI should auto-populate the radiologist report with an appropriate description of each organ system. The job of the radiologist is to confirm what the AI says in each section, and to go into further detail as needed. It's essentially just customizing the existing template to each patient. This type of pre-drafting is exactly what radiology residents do and what companies like vRad do.

haldujai 3y ago
What you’re describing is what a template does, I just dictate “stone” or “sludge” from a pick list and never a normal statement. There is also existing technology that creates a structured report from free dictations. Your math also doesn’t add up because 90%ish of studies are normal so you can basically sign a template without looking at the pictures and be more or less entirely accurate, before adding AI. Also what practice only reads 20 abdo CTs a day? Seriously, let me know I could use a more relaxing job. Lastly, an AI that functions with the competency of a PGY3 resident does not exist at the moment or on the horizon. Especially for cross sectional studies. Source: am a diagnostic radiologist and AI researcher (on the NLP side).
throwaway85858 3y ago
this! I asked a neurologist and she said at least 40% of her time is spent on formulaic discharge letters and rounds documentation.
bee_rider 3y ago

Ozzie_osman 3y ago

> Geoffrey Miller was one of the loudest voices decrying the decline of radiology 5 years, and now he’s crying fear for new AI systems. I don't know who Geoffrey Miller I'm pretty sure if there's a Geoffrey who notably predicted the decline of radiology, it was Geoffrey Hinton a few years ago...

xpe 3y ago
I did a few minutes of research. I did not find any such person. So I'm inclined to agree; the author may have meant Geoffrey Hinton. See also: https://statmodeling.stat.columbia.edu/2021/06/07/ai-promised-to-revolutionize-radiology-but-so-far-its-failing/ https://statmodeling.stat.columbia.edu/2021/06/07/ai-promise... > Gary Smith points us to [this news article][1]: > Geoffrey Hinton is a legendary computer scientist . . . Naturally, people paid attention when Hinton declared in 2016, “We should stop training radiologists now, it’s just completely obvious within five years deep learning is going to do better than radiologists.” The US Food and Drug Administration (FDA) approved the first AI algorithm for medical imaging that year and there are now more than 80 approved algorithms in the US and a similar number in Europe. [1] https://qz.com/2016153/ai-promised-to-revolutionize-radiology-but-so-far-its-failing https://qz.com/2016153/ai-promised-to-revolutionize-radiolog... > Geoffrey Hinton is a legendary computer scientist. When Hinton, Yann LeCun, and Yoshua Bengio were given the 2018 Turing Award, considered the Nobel prize of computing, they were described as the “Godfathers of artificial intelligence” and the “Godfathers of Deep Learning.” Naturally, people paid attention when Hinton declared in 2016, “We should stop training radiologists now, it’s just completely obvious within five years deep learning is going to do better than radiologists.” The US Food and Drug Administration (FDA) approved the first AI algorithm for medical imaging that year and there are now more than 80 approved algorithms in the US and a similar number in Europe. > Yet, the number of radiologists working in the US has gone up, not down, increasing by about 7% between 2015 and 2019. Indeed, there is now a shortage of radiologists that is predicted to increase over the next decade.
dang 3y ago
Looks like the OP has fixed that now.

m3kw9 3y ago

To start, small things like getting the json format to work 100% of the time is impossible.

xpe 3y ago
Me: "Are you speaking of JSON in the context of radiology systems? medical systems? enterprise systems? software in general?" You: Yes.

varelse 3y ago

[dead]

outside1234 3y ago

A customer I am working with can’t even set up CI/CD with GitHub. I have no doubt that AI is not going to kill us because 95% of companies won’t even be able to build it.

xpe 3y ago
> I have no doubt that AI is not going to kill us because 95% of companies won’t even be able to build it. I take your joke, but the argument doesn't hold water for many reasons. Here are only two: 1. Even if only 5% of companies can build something dangerous, that's more than enough. 2. The "AI" doesn't have to directly kill us; we're capable of doing that ourselves with only a little bit of informational, societal, and/or economic degradation.
xpe 3y ago
> A customer I am working with can’t even set up CI/CD with GitHub. Can you please characterize your customer without calling them out by name? Also, can you clarify what you mean by "can't"? Under what time frame? With what background? With what else on their plate? In other words, why do you expect your customer to be able to setup CI/CD with GitHub? Maybe this seems reasonable to you, but maybe you are overlooking the hundreds of little things that make it seem easy to you, not least of which is the patience and ability to wade through tasks with many steps. I'm genuinely curious. Are you suggesting they lack capability (such as intelligence or skills), motivation, and/or something else?
jacurtis 3y ago
It is because, especially in tech, most people can't even vocalize what they want to the detail that an AI could generate it. Even in my day-job (which is probably similar to yours), most people give me the wrong information when asking for a solution and if I blindly built what they think they needed, they wouldn't get where they wanted.

xpe 3y ago

Caveated [1] or not, the "concluding" paragraphs [2] are not a summary of the article. Neither are they well-supported nor convincing; they are sweeping, unrelated generalizations. Notes: 1. The caveat at the top states "NOTE: This post is not up to my normal writing standard, just felt compelled to get this down in some form. This is more like a blog post entry than a real newsletter addition." 2. The article's final paragraphs are: "Long story short / Thinkers have a pattern where they are so divorced from implementation details that applications seem trivial, when in reality, the small details are exactly where value accrues. / Should you be worried about GPT5 being used to automate vulnerability detection on websites before they’re patched? Maybe. / Should you be worried GPT5 is going to interact with social systems and destroy our society single-handedly? No absolutely not." P.S. Pedantic timestamp: the quotes above are taken on 12:24 pm eastern time, May 29. Well, this may seem pedantic until the author updates the blog post and then this comment seems erroneous. It is notable that we still don't have a well-accepted standard for snapshotted content.

ftxbro 3y ago

this sounds like the kind of cope that gets written before production ai starts going down the list replacing jobs

CatWChainsaw 3y ago
tbf your handle is on point

jameshart 3y ago

Wait a minute.... Because the role of radiologists is not as simple as trained classification, we can conclude that "No, AGI isn't going to take over every social system when GPT5 comes out"? Author starts off by saying that you can't generalize conclusions based on observations in a limited domain: > Geoff made a classic error that technologists often make, which is to observe a particular behavior (identifying some subset of radiology scans correctly) against some task (identifying hemorrhage on CT head scans correctly), and then to extrapolate based on that task alone. Author then goes on to suggest that there are a number of issues with radiology which AI will not solve. Then, based on this lack of ML's complete coverage of the aspects of radiology that he finds interesting, he then extrapolates: > Should you be worried GPT5 is going to interact with social systems and destroy our society single-handedly? No absolutely not. I don't see how the conclusion remotely follows from the argument. And more to the point, the refuting argument is embodied in the article itself.

skybrian 3y ago
Yes, it goes a bit far, but the basic problem is treating this as all-or-nothing: Either the article is true or false. Either AI is going to take over, or it will have little effect. I believe the article raises relevant issues about why deploying AI-based systems is harder and will take longer than it might appear. Another relevant example might be driverless cars. It's taking many years and deployment is still quite limited. Some companies gave up. But I'm not going to count Waymo out yet. Similarly, it's possible that Hinton got the timeline wrong for radiology, but it might still have significant affects on radiologist employment in the end. (Note that this sort of reasoning by analogy is useful for imagining plausible scenarios, but not for ruling things out.)
- naijaboiler 3y ago
  Driving is another place where people wrongfully think that solving the technical problem means the problem has been solved. Driving is a social problem. We are a long long way from solving it.
naijaboiler 3y ago

godelski 3y ago

At the root of this is something I'm constantly saying to other members in my research lab, arguing with reviewers, and passionately teaching my students: datasets are proxies, measurements are fuzzy. This is something that is drilled into students learning statistics and I can't for the life of me figure out why this changed with respect to ML (best guess is not requiring statistics courses). Datasets are proxies: they represent the world, but aren't. They should generally be seen as narrow subsets too. Your dataset quality and type matter a lot! Things like medical image datasets also have tons of correlating factors that can easily invalidate all results without you being aware. There are simple datasets we use to prove a concept (toys, mnist, cifar, etc). There are large scale datasets that have internal inconsistencies (imagenet, flowers). There are huge datasets that haven't been properly filtered/deduplicated (LAION). (there are also just shitty datasets (HumanEval)) Thinking of datasets as proxies helps internalize the frustration that literally every production engineer faces (even outside ML and software): real world results are inconsistent with lab results. Dataset engineering is an underappreciated art that is extremely difficult. But everyone needs to internalize that datasets are just a map, not the territory, and your navigation will only be as good as the map (many are poorly drawn maps, often on purpose) Measurements are fuzzy: Benchmarkism is running rampant in the ML world and it baffles me that a field who's bottomline objective deals with alignment isn't able to align how we evaluate ourselves. No measurement is perfect, many are far from them. You can train two language models to the same NLL and one might sample well and the other outputs garbage. You can train two image models to have identical FIDs and one samples clearly and the other doesn't. Likelihood also doesn't guarantee sharpness and I can go on. You must think about the limitations of your measurements and know them in depth. This also has seem to have gotten away from us and people just use the measurement libraries and call it that. We've reached a point where ImageNet classification accuracy has decoupled from downstream performance (object detection and segmentation), and things like this are confusing to production people because just taking the model with the highest score doesn't always result in the best performing model for their work (even before we consider things like throughput and memory usage). It is a Goodhart problem through and through. ML is at a serious point where we've gotten away from our basic stats learning. That's going to pose a real danger to society, not AGI. It's like handing powertools to chimps (who don't know how powertools work), it won't end well. But that is happening because we've shifted focus to meet targets, not to measure our work. Targets are easy, science is hard. Unless we bring these nuances back to our evaluation of works then we are just handing powertools to chimps without any quality assurance.

w10-1 3y ago

Brain MRI's (in all their forms) are a whole different species than all other MRI's, due to variability and softness of structures, differences in condition presentation/time-frame, and and the high incidence of brain MRI's in the elderly (the brain can shrink precipitously after 60). Also, neurologists are quite specialized (e.g., stroke vs. MS vs. acute encephalopathy vs. optic neuritis). In the majority of cases, the specialist neurologist is better at reading the MRI of their target conditions than the general radiologist. This is handled by the neurologist phoning the radiologist and guiding them to re-work their findings. The vast majority of AI in software is in radiology. Decades ago radiology was among the first practice groups to fall under private equity patterns because of the cost of the machines. In some cases, specific radiology groups develop protectable expertise around some protocols (i.e., how to get the machine to give better data), but long machine lifecycle creates opportunities for technical one-upmanship. Knowing that the labeling quality is variable and the images are not entirely comparable, you realize the bulk of AI-radiology may be built on shifting sands, and amounts to workflow optimization. Worse, the effort could go into addressing what really plagues all providers: the unusability of their EHR's. But that's not particularly monetizable relative to critical diagnostics, and harder than production AI for MRI.

guillemsola 3y ago

Aren't these complex as any other piece of software that is looking to provide value to users? I've been developing software solutions for several years and the fact that a computer can do cool stuff, automate processes, be more accurate... always need users who know and want to use it. So my takeway is that with AI solutions the human part also needs to be considered.

ugh123 3y ago

I don't think speaking to Radiologists as "advisors" to an AI system is going to be productive in the grand scheme of things. Yes, radiologists (and all doctors for that matter) will give their thoughts on making their job incrementally easier. But that doesn't necessarily equate to better healthcare. Is that going to move the needle on better healthcare for patients? Will it allow our hospital systems to bring in more throughput of patients at higher quality of care and lower costs? Thats not what doctors (and hospitals) want. They won't be an ally in "AI for healthcare" unless it means their paychecks/revenues are protected. The author is being misguided by seemingly smart people who have skin in the game and a lot to lose. Personally, I only favor AI solutions that will drive down the cost of all healthcare, raise the quality of service, and overall expand high-quality healthcare to underserved populations. Anything less than that is only serving the needs of the industry and not patients.

steve76 3y ago

[dead]

YeGoblynQueenne 3y ago

>> Geoffrey Hinton was one of the loudest voices decrying the decline of radiology 5 years, and now he’s crying fear for new AI systems. It's worth remembering exactly what Geoff Hinton said, and how he said it: “I think that if you work as a radiologist you are like Wile E. Coyote in the cartoon,” Hinton told me. “You’re already over the edge of the cliff, but you haven’t yet looked down. There’s no ground underneath.” Deep-learning systems for breast and heart imaging have already been developed commercially. “It’s just completely obvious that in five years deep learning is going to do better than radiologists,” he went on. “It might be ten years. I said this at a hospital. It did not go down too well.” https://www.newyorker.com/magazine/2017/04/03/ai-versus-md https://www.newyorker.com/magazine/2017/04/03/ai-versus-md So Hinton didn't "decry the decline of radiology". His comment brims with exuberant sarcasm about how his chosen approach to AI is about to not only leave radiologists without a job, but also leave them looking like the short-sighted fools he thought they surely were. Hinton was cock-sure and arrogant, like an undergraduate student who just trained his first neural net. That is quite unbecoming of the man whom the New York Times, The Guardian, and many other publications have been calling "the godfather of AI", and a Turing award winner, to boot. That is something he should be called out on, and not allowed to forget. Nor he, nor anyone else who thinks that leaving highly-trained workers without a job is a laughing matter. Nobody should be allowed to make fun of the harmful consequences of the technology they create.

cicce19 3y ago
Hinton has been crucified many times in radiology circles for his statements. It honestly feels like we've beaten a dead horse on this. Clearly he was wrong and did not understand the complexity of the field. Although he was wrong, his statements today (through the advent of LLMs) are less insane than they were even 7 months ago and directionally he saw the progress of the technology. His magnitude/timeline was off and time will tell if he is one day proven right.