4 ms·

To contextualize even further. Stable diffusion will fit on any persons computer AND the time it takes to run is minutes. But that's just about stable diffusio

To contextualize even further. Stable diffusion will fit on any persons computer AND the time it takes to run is minutes. But that's just about stable diffusion. You need to measure the speed of how fast stable diffusion came to be. 3 years ago all of this stuff was thought to be impossible. Now it's a SAS. What does the next 3 years bring? Contextualize that. Nobody knows when that day will come, BUT within our lifetimes is a very realistic bet given the speed in which the ML industry is changing and growing.

didibus 4y ago
Trained Stable Diffusion can, but training it costed 600 000$ and took 150 000 GPU hours. Also, Stable Diffusion isn't a revolution in model architecture, it's still a deep neural network. There's been a lot of innovation in how you can best apply DNNs to problems, and that's where CLIP and stable diffusion innovate, but also a lot of the advancements has been purely due to sheer brute force from funding more compute resources to train deeper networks using larger datasets. I'm optimistic, but I'm also trying to contextualize to have the right expectations.
- deltasevennine 4y ago
  > Trained Stable Diffusion can, but training it costed 600 000$ and took 150 000 GPU hours. So? The iphone as it is today took EVEN more man hours and billions of dollars to create. Yet every single person on the face the earth has one in their pocket. The effort to create stable diffusion is trivial when compared with many marvels of modern technology that we take for granted today. Stable diffusion in my pocket? It's already reality. >There's been a lot of innovation in how you can best apply DNNs to problems, and that's where CLIP and stable diffusion innovate, So? you talk as if this is a minor thing. It's very possible that enough innovations in this area and we'll find out that the entire human brain is layers and layers of complex "applications of DNNs". DNNs in theory can compute anything, so this path is forward is possible. > but also a lot of the advancements has been purely due to sheer brute force from funding more compute resources to train deeper networks using larger datasets. The training was trivial. Nobody expects AI technology to be able to be built by a single man. Like your iphone which lives in your pocket many lifetime hours of blood sweat and tears will need to be put into it. The half a million and 150,000 GPU hours is nothing. I expect more hours of training, larger datasets and eventually I expect all of tech to be encapsulated into some product that lives in the cloud or aka effectively in my pocket. >I'm optimistic, but I'm also trying to contextualize to have the right expectations. You don't realize it but this sentence is a declaration equivalent of saying "I'm a biased person." Optimism and pessimism is a form of bias. You state that you are optimistic, thus you stated you are biased. True logic has no leaning in either direction. This sentence here reveals what you are and that your "contextualization's" are inherently flawed. Additionally, research finds that pessimistic people tend to be right. Yes logic dictates that neither pessimism or optimism is the way to go, but the data driven conclusion is that people who are pessimistic tend to be more realistic. The data shows that most humans are biased with an optimistic delusional bent. This behavior is so pervasive that you even called yourself optimistic without realizing that it was a declaration of your own bias. Most reading your sentences miss it as well unconsciously thinking "optimism" of any form is more rational "pessimism" even though BOTH the logical and data driven conclusion show that this is not the case. Contextualize yourself first before you contextualize the external world.
  - didibus 4y ago
    > You don't realize it but this sentence is a declaration equivalent of saying "I'm a biased person." That was the point of my sentence lol, I'm explicitly stating my bias in order to address it. Emotionally and for my own personal benefit, I want breakthroughs in AI to be achieved in my lifetime, and I hope they do. I'm explicitly acknowledging and stating my bias for this. You address bias by recognizing it and taking it into account. Then, I'm contextualizing my emotional stance, by providing data points about the reality of what the current state of AI is so that others have the context required so they can apply their own judgement. Look, I get it, it's clear we're both excited about the prospects of AI, but you're getting emotionally defensive, that's exactly what I'm trying to avoid, let's remove our clout of excitement, and contextualize by looking at the data. Yes we've made a lot of progress, but there's a lot more required, and unfortunately progress in any field of research has never followed a linearly constant progression, there was a AI winter before, there could be another. The claim that we're close to having machines able to do everything that humans can do and do it even better doesn't seem to follow from the data. There are gaps as I provided, we're only achieving close to human ability for tasks where we have hundreds of millions of examples. Even in those scenarios, there are catastrophic failure points in the models that humans would never fail at. Hardware wise, we're approaching the limits of how big a DNN we can train as well and how much data examples we have to feed it. Models are getting really good at predictions over data, but they've yet to show any ability for creativity and invention. Right now they all need human assistance to output an interesting result. Take GPT-3, and it can't even interpret what you're saying, you've got to tailor how you talk to it to get interesting responses. Take stable diffusion, it hasn't been able to invent a new drawing style yet, only mimicking the ones it learns as another example. That's pretty major limitations compared to a human, and those are tasks that we have the most data available to us for training. I think we'll need some other breakthrough to break past this barrier, and that I think could easily take much longer to achieve. Your assessment could be different, maybe you take in all this data and predict a more optimistic timeline for overcoming all these issues, that's fair, what I wanted is to contextualize, not push my conclusion, contextualizing means providing others with the rationale and data so they can do their own assessment and arrive at their own conclusions. There is a lot of optimism around Stable Diffusion and GPT-3 and others, you're flooded with the "OMG look at all it can do" data points, and then all the dreams of sci-fi becoming reality. There isn't as much exposure and discussions around, "okay but look at all it can't do and where the capabilities for what it does come from and how they were achieved". I wanted to provide that balance so people can have realistic expectations.

didibus 4y ago

Trained Stable Diffusion can, but training it costed 600 000$ and took 150 000 GPU hours. Also, Stable Diffusion isn't a revolution in model architecture, it's still a deep neural network. There's been a lot of innovation in how you can best apply DNNs to problems, and that's where CLIP and stable diffusion innovate, but also a lot of the advancements has been purely due to sheer brute force from funding more compute resources to train deeper networks using larger datasets. I'm optimistic, but I'm also trying to contextualize to have the right expectations.

deltasevennine 4y ago
> Trained Stable Diffusion can, but training it costed 600 000$ and took 150 000 GPU hours. So? The iphone as it is today took EVEN more man hours and billions of dollars to create. Yet every single person on the face the earth has one in their pocket. The effort to create stable diffusion is trivial when compared with many marvels of modern technology that we take for granted today. Stable diffusion in my pocket? It's already reality. >There's been a lot of innovation in how you can best apply DNNs to problems, and that's where CLIP and stable diffusion innovate, So? you talk as if this is a minor thing. It's very possible that enough innovations in this area and we'll find out that the entire human brain is layers and layers of complex "applications of DNNs". DNNs in theory can compute anything, so this path is forward is possible. > but also a lot of the advancements has been purely due to sheer brute force from funding more compute resources to train deeper networks using larger datasets. The training was trivial. Nobody expects AI technology to be able to be built by a single man. Like your iphone which lives in your pocket many lifetime hours of blood sweat and tears will need to be put into it. The half a million and 150,000 GPU hours is nothing. I expect more hours of training, larger datasets and eventually I expect all of tech to be encapsulated into some product that lives in the cloud or aka effectively in my pocket. >I'm optimistic, but I'm also trying to contextualize to have the right expectations. You don't realize it but this sentence is a declaration equivalent of saying "I'm a biased person." Optimism and pessimism is a form of bias. You state that you are optimistic, thus you stated you are biased. True logic has no leaning in either direction. This sentence here reveals what you are and that your "contextualization's" are inherently flawed. Additionally, research finds that pessimistic people tend to be right. Yes logic dictates that neither pessimism or optimism is the way to go, but the data driven conclusion is that people who are pessimistic tend to be more realistic. The data shows that most humans are biased with an optimistic delusional bent. This behavior is so pervasive that you even called yourself optimistic without realizing that it was a declaration of your own bias. Most reading your sentences miss it as well unconsciously thinking "optimism" of any form is more rational "pessimism" even though BOTH the logical and data driven conclusion show that this is not the case. Contextualize yourself first before you contextualize the external world.
- didibus 4y ago
  > You don't realize it but this sentence is a declaration equivalent of saying "I'm a biased person." That was the point of my sentence lol, I'm explicitly stating my bias in order to address it. Emotionally and for my own personal benefit, I want breakthroughs in AI to be achieved in my lifetime, and I hope they do. I'm explicitly acknowledging and stating my bias for this. You address bias by recognizing it and taking it into account. Then, I'm contextualizing my emotional stance, by providing data points about the reality of what the current state of AI is so that others have the context required so they can apply their own judgement. Look, I get it, it's clear we're both excited about the prospects of AI, but you're getting emotionally defensive, that's exactly what I'm trying to avoid, let's remove our clout of excitement, and contextualize by looking at the data. Yes we've made a lot of progress, but there's a lot more required, and unfortunately progress in any field of research has never followed a linearly constant progression, there was a AI winter before, there could be another. The claim that we're close to having machines able to do everything that humans can do and do it even better doesn't seem to follow from the data. There are gaps as I provided, we're only achieving close to human ability for tasks where we have hundreds of millions of examples. Even in those scenarios, there are catastrophic failure points in the models that humans would never fail at. Hardware wise, we're approaching the limits of how big a DNN we can train as well and how much data examples we have to feed it. Models are getting really good at predictions over data, but they've yet to show any ability for creativity and invention. Right now they all need human assistance to output an interesting result. Take GPT-3, and it can't even interpret what you're saying, you've got to tailor how you talk to it to get interesting responses. Take stable diffusion, it hasn't been able to invent a new drawing style yet, only mimicking the ones it learns as another example. That's pretty major limitations compared to a human, and those are tasks that we have the most data available to us for training. I think we'll need some other breakthrough to break past this barrier, and that I think could easily take much longer to achieve. Your assessment could be different, maybe you take in all this data and predict a more optimistic timeline for overcoming all these issues, that's fair, what I wanted is to contextualize, not push my conclusion, contextualizing means providing others with the rationale and data so they can do their own assessment and arrive at their own conclusions. There is a lot of optimism around Stable Diffusion and GPT-3 and others, you're flooded with the "OMG look at all it can do" data points, and then all the dreams of sci-fi becoming reality. There isn't as much exposure and discussions around, "okay but look at all it can't do and where the capabilities for what it does come from and how they were achieved". I wanted to provide that balance so people can have realistic expectations.