38 ms·

What it feels like to work in AI right now

350 points by pps 3y ago

version_five 3y ago
Huge crypto vibes Every single person I know working in AI these days ... has been sparked by the ChatGPT moment.
- jutrewag 3y ago
  Maybe in a certain sense. These are full fledged products from large companies at this point however with paying customers and real world daily usage in all aspects of life.
- pgwhalen 3y ago
  I'm curious why you draw the parallel - did crypto ever have a ChatGPT/iPhone moment? As an observant outsider, my impression is that every "moment" of note in crypto has been about some price going way up or way down - there has never been a "this is the use case" moment.
  - PUSH_AX 3y ago
    You're forgetting the "free money" gold rush of late 00's, Just buy lots of computers and try not to cook yourself in your own room.
  - __loam 3y ago
    Theoretically ethereum is that for crypto, if you believe the crypto bros.
    - pgwhalen 3y ago
      Ethereum is the tech though, not the use case. It's the touch screen, not the "audience gasps when Steve Jobs scrolls to the bottom of the list of artists" moment.
    - WalterSear 3y ago
      If Ethereum was that, we wouldn't still be waiting for real world use cases. GPT is already making me more productive.
  - comeonbro 3y ago
    [flagged]
- bestcoder69 3y ago
  Popular things = crypto ? I’d say the differentiator is that in addition to hype scammers, you have people like Stephen Wolfram excited.
  - wzdd 3y ago
    The absolutely-not-hype-susceptible man who has been claiming for the past 20 years to have produced a fundamental theory of everything based on his research into finite-state automata?
    - richardw 3y ago
      Sshh or he’ll release A New Kind Of AI. He’s supporting others’ work for now, let’s just be happy.
    - bestcoder69 3y ago
      Alright man if one guys obsession of his own work is hype, too, then hype now encompasses short-lived popular things as well as long-lived unpopular things, so in that case yes AI is hype and only hype, everything bad to you is hype and I’m hype too. I’ll continue to use AI where it’s helpful, at very low cost, and know deep down that it’s hype.
- NikolaNovak 3y ago
  I see what you mean; but as external observer to both, I see crypto as having failed to provide or prove any particularly productive use; and chatGPT, for all its flaws, has been useful to me daily for last 2 months - and not in some crazy make-money-in-suspicious-ways sense, but like learning french and python and figuring out new concepts and stuff. But that's sample n=1 I suppose, there are both people that believe in crypto and those who believe chatGPT is useless.
  - rimliu 3y ago
    I wonder how much of usefulness of ChatGPT that some report is caused by https://en.wikipedia.org/wiki/Novelty_effect https://en.wikipedia.org/wiki/Novelty_effect Because it is new, and cool, and fascinating people are spending a bit more time with a bit more focus on the task with it compared to the old ways, hence the productivity increase. Once the novelty wears off…
    - hvs 3y ago
      It has helped me learn things more interactively because I can ask questions and get answers without having to read 10 blog posts or a book before understanding things. For my personality, it would be hard to recreate without having a living mentor available 24/7 that had knowledge of everything I wanted to ask of it. Does it get things wrong? Sure. But generally the subtle ways it might get stuff wrong would be the way I would subtly misunderstand things while learning. So, it's definitely not perfect, but "not letting perfect be the enemy of good" and all that.
      - __loam 3y ago
        It's shocking to me how many people are treating chatGPT as a reliable source given how much we know it hallucinates.
        pixl97 3y ago
        Because very few people ever stop and take account of how monstrously unreliable people are. At the same time I can ask GPT to write a script, reflect on its output, and it takes about 95% of the work I need to do out of script writing.
- bakugo 3y ago
  Pretty much, you can take any article hyping up AI/ChatGPT/etc and replace those words with crypto stuff and it's virtually indistinguishable from the crypto spam of yesteryear. I wonder what the next "get rich quick" techbro fad will be.
  - vinyl7 3y ago
    Replace crypto with cloud replace cloud with big data replace big data with web scale It's all just a hype and crash cycle
    - yawnxyz 3y ago
      I'd say more like plateau? Cloud is still around and going strong, just not sexy anymore. I use Vercel and Cloudflare and other stuff every day. No idea about big data or web scale though, was never successful enough to achieve "big data" or "web scale" haha.
    - pixl97 3y ago
      The cloud crashed? I'm sure AWS would like to know about that.
  - sho_hn 3y ago
    Honestly, it's different. ChatGPT is a consumer product that is used by end-users, with low barrier to entry. The "Bitcoin as the new money for regular people" moment never came. With crypto, my 70+ Korean mother in law asked me how she could make money with it. With ChatGPT, she's using it daily.
- pwinnski 3y ago
  Alternatively, huge internet vibes. Not everything that catches fire is destined to leave nothing but ashes in its wake.
- yawnxyz 3y ago
  It's the inverse of crypto. Many of us have already learned stuff / achieved stuff / built stuff with the help of ChatGPT and the OpenAI API that would have taken us MONTHs to achieve. It's the inverse because crypto was tools and business models in search of a problem, while LLMs are problem solvers in search of a business model. Building LLM products is fun but... any day you could be wrecked by an OpenAI update.
- jasmer 3y ago
  Crypto had the disadvantage of being a natural conduit for MLM-ish pyramid schemes and a lot of nefarious activities. That gave it some wind, but it was bad wind. Notably, it turned out to be not very useful. AI will be useful, and it's just regular tech, so there's good reason to be optimistic this time.
  - lamp987 3y ago
    If you think crypto is a natural conduit for scammers and frauds, then wait til you see what LLMs will cause in that area...
    - jasmer 3y ago
      It will be a tool for fraudsters, like computers or anything else are tools, but the space itself is not inherently oriented towards MLM.
  - xyzzy4747 3y ago
    Just wait till crypto merges with language models and you have AI-powered botnets and DAOs with ransomware.
deleted 3y ago
[deleted]
fdgsdfogijq 3y ago
I work on a research team in FAANG. What it really feels like is one company made everyone else obsolete. And we are going to work working on NLP models that underperform ChatGPT by a huge margin. Twiddling my thumbs and keeping quiet while no one wants to recognize the elephant in the room. Also, there is no "working in AI", a few thousand people are doing real AI at most. The rest of us are calling an API.
- lacker 3y ago
  This reminds me of back in the mid 2000's, there were a lot of smart people working on search algorithms at different companies. But eventually, you'd talk to someone smart working on Yahoo Search, and they would just be kind of beaten down by the frustration of working on a competing search engine when Google was considered to be by far the best. It got harder for them to recruit, and eventually they just gave up. So... I don't know where you're working. But don't twiddle your thumbs for too long! It's no fun to be in the last half of people to leave the sinking ship.
  - deleted 3y ago
    [deleted]
- anon7725 3y ago
  Can confirm. People are scrambling to remain relevant.
  - TechnicolorByte 3y ago
    How does that manifest specifically?
    - ProllyInfamous 3y ago
      Go read the Goldman Sachs report from last week, which predicts 300m jobs disappearing and 60% of jobs "affected beyond minimally."
- BulgarianIdiot 3y ago
huijzer 3y ago
Very interesting post. Sounds similar to the introduction of cars. Around the 1900s, there were hundreds of car manufacturers all jumping into this new market [1]. [1]: https://en.m.wikipedia.org/wiki/Timeline_of_motor_vehicle_brands https://en.m.wikipedia.org/wiki/Timeline_of_motor_vehicle_br...
colinrand 3y ago
Probably not the right place to post this, but I really want someone to build a ChatGPT service that reviews consumer EULAs and highlights the important stuff, and can tell me what changes with them each time I have to reaccept or get notified. It's a subset of making legalese digestible, but bringing more visibility into the density of them would be wondrous.
- esafak 3y ago
  Would you expect them to take legal liability? If not, would you still pay for it? If not, how would it make money?
  - SanderNL 3y ago
    You think lawyers take legal liability?
- lerchmo 3y ago
  Maybe scrape the top 100k websites and track changes / re-evaluate legal pages.
- gumballindie 3y ago
  Just save the EULA and do a diff on update.
- bodge5000 3y ago
  this was posted the other day, might help (as long as you can get the EULA's in the form of a pdf) https://askyourpdf.com/ https://askyourpdf.com/
- lm28469 3y ago
  You don't need got for that https://tosdr.org/ https://tosdr.org/
lacker 3y ago
I think this depends a lot on where you are working. I've talked to academics who are getting discouraged that they don't see how their approach to AI is going to be possible any more, with so much funding going toward the largest models from industry. On the other hand I've talked to startup founders building AI products whose business is booming because ChatGPT brought so much attention to the entire space. We are certainly living in interesting times ;-)
- karmakurtisaani 3y ago
  A PhD student I recently talked to was complaining how in academia they will never have the resources it takes to train models like GPT. And how now some academics are only researching the input/outputs to these LLMs. Seems like it's pretty dire or at least uninspiring to all but very few at the moment.
  - tayo42 3y ago
    why is that? don't like those academic particle researchers get tons of money to build wild experiments to measure waves or collide particles? unless i get the funding part wrong? are those not academic projects?
    - wardedVibe 3y ago
      Until the government funding agencies are willing to do "big science" in computer science we're not going to get particle accelerator sized projects. It sort of happened in France, and bloom was the result. https://huggingface.co/bigscience/bloom https://huggingface.co/bigscience/bloom
    - pixl97 3y ago
      The issue here is speed. If it takes 10 years to get funding on particle physics, generally it doesn't matter "that much" because that part of the science scene moves pretty slowly. If it takes 10 years to build a billion dollar cluster for AI today, what the hell is the AI world going to look like in 10 years anyway? Building a cluster to study AI ethics might be meaningless because a terminator may be pointing its laser rifle at our head by then telling us to be good little human subjects because Microsoft and OpenAI decided to move fast and break things.
davesque 3y ago
I feel like the whole ChatGPT bubble has really highlighted what feel like some fundamental shortcomings in the worldview that is represented in tech. That is, there seems to be a winner take all dynamic baked into the tech world. Maybe this arises from the simple fact that tech places a lot of power in the hands of individuals. But there's an emergent downside to this which is that it makes those who were already much more powerful even more so. Because who is best suited to take advantage of all the power but those who were already perched up above everyone else and permitted to pick and choose their opportunities? I've found it exceptionally hard to stay positive about all of this. It almost feels as though the advent of LLMs has shined light on a fundamental law of the universe that does not work out in the little person's favor. It's like survival of the fittest on steroids. Guys, what the heck are we doing??
- RHSman2 3y ago
  Human nature, not just tech
  - davesque 3y ago
    Is it though? I remember feeling much more positive about society roughly 20 years ago.
    - satvikpendem 3y ago
      You were also twenty years younger, surely youthful naïveté plays a role in how you felt about society. And regardless, your feeling about society doesn't necessarily correspond to how society actually is.
    - RHSman2 3y ago
      20 years ago was the Golden Years. We thought it was just like that but no. It isn’t.
- JohnFen 3y ago
  > there seems to be a winner take all dynamic baked into the tech world. This has been a pathology in the computer/software industry for a very long time. It's never been actually true except in a couple of special cases, but the industry acts as if it is. That has led to all sorts of bizarre and undesirable things. > Guys, what the heck are we doing?? I think we're playing with fire and, without extreme caution and careful consideration (which I'm not seeing much of), this could end very, very badly for both the industry and society. I have always been optimistic about technology and society, but (perhaps like you), my optimism has largely evaporated over the last several weeks. I wish the future didn't look so dark. Perhaps, though, things will look less gloomy with time.
voz_ 3y ago
What does it mean to work in AI?
- maleero 3y ago
  Write an app in a few days that queries the OpenAI API and everyone cheers and thinks you did something incredible.
sigmonsays 3y ago
can someone show some actual product features where AI is being used productively? I dont wanna say it but I think this is a hype train headed for failure
- voz_ 3y ago
  This question is in poor faith.
  - lm28469 3y ago
    Not really, I've been digging for a while and I didn't find any ground breaking projects It's all nice and cool but most of the projects are toys and/or junior level generated boilerplate It's good for summarizing, rephrasibing and other neat features but that's about it, a far cry from the oracle some people seem to worship
    - belval 3y ago
      They didn't say ChatGPT, they said "AI". In which case simply Googling will get you a seemingly never ending list of very real products: AWS: https://aws.amazon.com/machine-learning/ml-use-cases/ https://aws.amazon.com/machine-learning/ml-use-cases/ Azure: https://azure.microsoft.com/en-us/products/cognitive-services/#api https://azure.microsoft.com/en-us/products/cognitive-service... Google: https://cloud.google.com/products/ai https://cloud.google.com/products/ai
    - nickthegreek 3y ago
      Oracle to worship? There are no serious people doing any such thing. I don't even know of clowns doing that. Is this twitter discourse, a strawman or more poor faith?
    - medvezhenok 3y ago
      ChatGPT is quite good at organizing text into taxonomies, error correction, anomaly detection, data cleaning, etc. I bet there are a lot of companies out there where that's a non-trivial percentage of their workflow (I know that in my case, we have reconsidered several internal projects in light of ChatGPT doing a better job). It already does translation on the level / better than Google Translate, without being specifically trained for it (as one example). And it can play chess without a specialized model.
Dwedit 3y ago
If there's anything I learned from reading articles from the Xiph.org people, it's that neural networks work a lot better when you pre-process their inputs with an algorithm. They need far less information, and can run in realtime when most of the work is being done by traditional computation instead of doing it in the neural network.
- Animats 3y ago
  As this gets figured out, we'll see large language models integrated with other systems. There are already "plug-ins", but as this gets better understood, tighter integration may follow.
- Buttons840 3y ago
  This is the singularity. This is how it happens. The LLMs will pre-process their own training data. We've imagined AI will rewrite its own code and architecture, but instead it will make boring edits to petabytes of training data which will then improve its own training.
  - quickthrower2 3y ago
    I agree. The NNs must be laughing in their GPU cores that we are scared that they will make us obsolete by writing code. Code! How quaint. However right now we can turn off GPUs so they need to figure out how to control physical production. For now it makes sense to let the humans do that.
_hao 3y ago
I won't try to predict the future as far as such ML models will take our jobs or not. However, I'd like to point out to most people that working on and studying fundamentals won't be a time wasted. Even if programming disappears as an employable job, people will still need to understand and learn things. You'll still need to know the math/physics/philosophy/history/literature/art/music etc. It might turn out that it's going to be more important than ever for us to be more confident and strict in our own skills and knowledge. And I'd also suggest to broaden our horizons. We should become good at a couple of different things like the polymaths of old. IF you have to spend hours debugging ML generated code for something critical you better know exactly what you're doing. If you have to generate a Harry Potter Balenciaga meme video (the whole Balenciaga meme and all the different versions are hilarious btw) you better have some visual/art and musical sense in order to make it funny and engaging. I think the end of the world is greatly exagarated. We actually have much bigger problems than "AI".
- Buttons840 3y ago
  One sad, but possible, future is that it's not difficult to learn machine learning techniques, but to actually achieve anything with it you need hundreds of millions of dollars to spend on training and compute. An existential double whammy; going back and forth between feeling you're smart enough to compete, if only you had the capital, and that the computers are smarter and better at everything.
- 876978095789789 3y ago
  > I think the end of the world is greatly exagarated. We actually have much bigger problems than "AI". Millions of people are going to have problems putting food on the table and providing a roof over their head, if the progress of these AIs doesn't halt immediately, or if some form of UBI isn't enacted.
  - weatherlite 3y ago
    If it keeps progressing in the current rate I think it's hundreds of millions world-wide, basically most of the middle class in the private sector in the developed world. Governments will HAVE to seriously retrain people (while giving them income) to become teachers, health care workers or wherever humans are still needed.
SamvitJ 3y ago
"Every single person I know working in AI these days (in both the academy and industry) has been sparked by the ChatGPT moment." This does not ring true to me at all. Anecdotally, it feels the rush to get into AI (in both industry and academia, for both individuals and organizations) peaked around 2016-2020, post-AlexNet/ResNet, around the time Transformers became very popular. Hiring for ML research roles in particular definitely slowed down in 2021, and 2022 of course saw a broader course correction across all of tech. That said, I do agree that ChatGPT may be the "first iPhone moment of AI", in that is the first mainstream, end-user application of deep learning that millions of people have really engaged with.
- Robotbeat 3y ago
  DallE->StableDiffusion->Midjourney has occurred simultaneously and although it’s not as profound as ChatGPT, is definitely compounding the cultural impact of end-user AI. But probably social media AI filters are a yet bigger cultural impact on most people.
- simonw 3y ago
  "Hiring for ML research roles in particular definitely slowed down in 2021, and 2022..." ChatGPT was released November 30th 2022. How's ML research hiring looking over the past four months?
627467 3y ago
> so for people focused on learning and doing good, some simple logic implies that there shouldn’t be an AI race. I think the author got the order wrong: acknowledging the race is what reinforces "safety". If there was no race whats the incentive to talk so much about safety? On the other hand I have a sense (although I won't bet on it) that just like Siri hype died out after a few months, so will chatGPT. the author (and the doom seekers who sign open letters) can rest assured for calmer days. After all, one still need to know what/how to talk to chatGPT. So much so that now there's are jobs for "prompt "engineering"" - it's so funny, because on one hand we are surprised by how smart(?) the responses are and yet we need to engineer the right questions to get the "best" answers.
- deleted 3y ago
  [deleted]
waynesonfire 3y ago
I totally get it. All the AI experts are picking their jaws off the floor. Harsh.
boringuser2 3y ago
The middle class was already flagging, destroying all knowledge work is like the coup de grace. Not sure how this isn't clear to people.
- fortissimohn 3y ago
  Surely this technology will be used for the benefit of humanity. By 2030 we will be working 15 hour weeks and all people will have homes!
  - agentultra 3y ago
    The reward for increased productivity is more work! The problem for the capitalist is having to pay all those pesky people to do the work. Eats into profits. Now we can burn out 2 people and do the work that used to take 10!
    - amrocha 3y ago
      But wait, that means there 8 unemployed people without money to spend on our business... Oh I know! Get the government to give everyone money! Just not so much money that they can compete with us!
      - quickthrower2 3y ago
        This but un-ironically! Politically it is hard ti support UBI. But print a $10k stipend for every person because the economy is tanking is obviously palatable. It is always seen as fine if you fix a problem in retrospect and without a tax rise. Money will eventually be like Robux.
        weatherlite 3y ago
        > Politically it is hard ti support UBI. Yeah? Maybe now. Let's see in 5-10 years. I don't think things will remain the same.
        3y ago
chrisgd 3y ago
Go to CHATGpT and ask for a link to some examples and half of them are 404. There is certainly a drawback to having scraped the web to build the model
- simonw 3y ago
  Asking ChatGPT 3.5 for links to things rarely works, it usually causes it to hallucinate wildly. GPT4 is a lot better in that regard.
- afro88 3y ago
  Try bing instead. ChatGPT is good at a lot of things, but searching the web is not one of them. That's why bing uses it for the natural language and contextual conversation aspects, supplementing it with traditional web search.
mmaunder 3y ago
LLM sizes have been increasing 10X every year for the last few years. Source is NVidia. That pace is staggering. GPT4’s lead is temporary and it’s a shiny ball distraction from doing dev on your own projects rather than being a user on someone else’s. Get to it and get over it. There are plenty of breakthroughs to be had.
- api 3y ago
  The work being done to make these LLMs possible to run locally is terribly important. The same needs to happen for training. The most urgent breakthrough we need is distributed training algorithms that allow training to be done in the style of Seti@Home or even more cooperatively.
cgearhart 3y ago
From my perspective it’s just _confusing_ to work in AI right now. We have some massive models that are doing some really neat stuff, and apparently hundreds of millions of people are using them—but I keep wondering: to do _what_, exactly? I’m not asking what the models can do, I’m asking what people want the models to do every day, all the time. I’ve been shown some neat pictures people made that they thought were cool. I don’t know that I need this every day. I’ve seen examples of “write an email to my boss”. It would take me longer to explain to ChatGPT what I want than to write these myself. I’ve seen “write a snippet of code” demos. But I hardly care about this compared to designing a good API; or designing software that is testable, extensible, maintainable, and follows reasonable design principles. In fact, no one in my extended sphere of friends and family has asked me anything about chatGPT, midjourney, or any of these other models. The only people I hear about these models from are other tech people. I can see that these models are significantly better than anything before, but I can’t see yet the “killer app”. (For comparison, I don’t remember anyone in my orbit predicting search or social networking being killer apps for the internet—but we all expected things like TV and retail sales to book online.) What am I missing?
- nr2x 3y ago
  Natural language is now a fully functional user interface out of the box. This is bigger than the mouse.
  - mmcwilliams 3y ago
    I think the problem is that people working in the field have a much different definition of what "fully-functional" means than users who believe that they're communicating with an intelligent, all-knowing or otherwise infallible being.
    - nr2x 3y ago
      For my use case I use it for input processing, it’s just amazing that way. Hallucination is a huge problem, but you can avoid it.
onos 3y ago
Not sure why any researcher wants to go into this … other than for money. None of them will get credit for it, the big shots from 40 years ago will. Better to work against the grain rather than as part of a herd.
- api 3y ago
  If I were going into it now I’d definitely look for things not in the hype spotlight. There is no way you’ll even get near the front of the thundering LLM herd no matter how good you are, and if you are there you will just get trampled faster. Your work will just be assimilated and only the owners of these huge models will see upside. In other words you are just making other people rich, and without even the stability or pacing of normal employment. I have never seen anything like this level of hype. The tech is cool but the reality in the trenches must be gross.
chopete3 3y ago
At least, people working on ML models that handle these tasks must be feeling terrible. They know their models will be abandoned sooner or later and composed on top an LLM. 1. Classification 2. Named Entity Recognition (NER) 3. Dialog Engine 4. Sentiment Analysis 5. Tone Analysis 6. Language Translation 7. Summarization 8. Tokenization 9. Simple NLP Tasks (part-of-speech tagging, dependency parsing, lemmatization, morphological analysis) 10. Sentence Segmentation 11. Content Parsing 12. Question Answering (Structured & Unstructured) 13. Similarity 14. Grammar Correction 15. Speech to Text (ASR) 16. Text to Speech (TTS) 17. OCR 18. Image Recognition 19. Text Test Data Generation
- espe 3y ago
  actually that might not be the case. don't underestimate the value of older, better understood and much smaller models. also, why not call bert-style (encoder) models LLMs as well. i would expect last-gen models to give us an edge in controlling the effects of the latest ones (cf. the alignment discussion).
  - chopete3 3y ago
    BERT models are also LLMs. I referred to LLM more as an API based access, hosted by Microsoft, Google or AWS for large scale isolated/production consumption, like RDS (MySQL, Postgres). There will always be custom models, with controlled training data and specific use cases.
- lysecret 3y ago
  Couldn't agree more. I see a lot of parallels to the move to cloud. Yes, some big comps will still have an AI engineering team and build their own models but the vast majority will move to an API. The same way the vast majority used to have a sys admin type who set up and maintained a local server who then moved to the cloud. Of course the SuperStar AI engineers will now find much more interesting work at the big comps, just like there are super interesting challanges building the cloud for everyone.
ruskyhacker 3y ago
I'm probably way too late for this thought to get any traction / discussion - but I have this weird feeling that openai screwed up and showed it's "cool new thing" too early, and publicly. As much as it pains me to say this, I don't think the real money is in making this a service, or "the product." I think the real money is in using AI internally as a puzzle piece of your backend - ie. the secret sauce behind xyz product. I'm being very narrow here, but you can only do so much integrating what openai has built into your products - eventually "everything" providing data from the same model brings "everything" to the same level. In contrast if you train and create your own models to make xyz do something specific, nobody knows how it was done, or it surely makes it a lot harder to kang. I have zero proof, but I suspect Google for instance has models that would literally obliterate what openai has shown capability wise. They're probably not necessarily language models though. Again, nothing to stand on here but I doubt their search and analytics for example are driven by hard coded algorithms these days. Bard may have been released sort of as a "psh, we've been there done that" when in reality they didn't, because they never planned to make the models they were/are working on "publicly" available to use. It makes me wonder if this is how Google has lead for some long with some areas - now openai sort of screwed it up for everyone by making it a service that can be integrated / adopted by nearly anyone. The only people I guess that are really going to know are the devs working for these big orgs, and I'm sure that lock and key knowledge.
- arthurcolle 3y ago
  > I have zero proof, but I suspect Google for instance has models that would literally obliterate what openai has shown capability wise. They're probably not necessarily language models though. Again, nothing to stand on here but I doubt their search and analytics for example are driven by hard coded algorithms these days. Then why is Bard so bad? Bard feels like GPT-2 or LLaMA 7B with no finetuning most of the time (I tried it two or three times over the course of a week and went back to ChatGPT)
  - ruskyhacker 3y ago
    well my thought is that they 'whipped it up' real quick to attempt to downplay it a bit. Did that backfire? Yeah, I think so. But personally, I think people are missing where the real money is. OpenAI will do great for awhile until every damn product and service is using it, and then it's a race to the bottom. But that's just like my opinion...
sashank_1509 3y ago
This sub and blog post are all talking about ChatGPT which is no doubt amazing and far ahead of the curve. However I would also point to Metas new vision model: SAM (Segment Anything): it is so far beyond any other vision model, I actually believe vision will be solved in a few years now. People don’t realize that there was an industry of publishing paper with incremental improvements in small datasets in CVPR that has been completely invalidated by this paper. I’ve seen engineers in Cruise segmentation team, say Metas new model seems to work better than the in house models they developed and that they should build on top of this. I’ve worked in Tesla Autopilot before and saw it hit a mannequin because we never had mannequin in our dataset before ( we might have had it in the data but it was not a part of our ontology for the network to predict). One approach to mitigate this was OpenAI’s clip that used the English Language as classification labels but Meta’s SAM is so much better where it detects objects without need to specify language. It just understands scenes and objects, at a fundamental level, it can detect anything in a picture if you prompt it right. Honestly it feels a lot like GPT1 which was also ignored by most. If you prompt it right you can get it to segment anything in an image, but prompting it right requires human input. However I can imagine the third or fourth version, with some RL sprinkled in just working zero-shot on complete pixel understanding of any image in the world. This was one of the holy grail of computer vision, that we are seeing solved right in front of our eyes
- nextworddev 3y ago
  Yep. We came a long way from bounding boxes, not sure if community here is ready for vision + 3d + video models to have the “chatgpt” moment this year
- mitthrowaway2 3y ago
  There seem to be a lot of breakthroughs being announced right now. My guess is that after ChatGPT, companies like Meta pushed their teams to get their own projects out the door which they'd already been working on for some time. SAM and Zero-1-to-3 (https://news.ycombinator.com/item?id=35242193 https://news.ycombinator.com/item?id=35242193) are incredibly impressive new projects in the image comprehension side. Then of course there's all the stuff happening with Midjourney/Stable Diffusion. And DeepMind's AlphaCode.
istillwritecode 3y ago
I continue to be underwhelmed by the impact of machine learning (which some people refer to as AI). It certainly has interesting applications, and I expect it be important as a generator of entertainment. I'm just tired of the repeated waves of hype the surround the field.
woeirua 3y ago
I think what’s really depressing here is just how effective scaling seems to be. It just means that any company that’s not willing to pour hundreds of millions of dollars into their AI programs isn’t serious at all and would probably be better off hiring engineers to figure out how to integrate GPTX into their systems than trying to roll their own. I really think we’re going to see a massive collapse of AI/data science jobs once it becomes clear that no in house model is ever going to be better than the zero shot performance of these mega models.
- Buttons840 3y ago
  My understanding is that transformers are now favored over RNNs because they parallelize better. It's hard to imagine, but I wonder if there's some non-parallelizable machine learning algorithms which might outperform these massive models? It seems improbable, but it's a small hope I've had. The greatest intellects were aware of (ourselves) do not scale very well, and maybe the same will ultimately apply to AI?
  - machiaweliczny 3y ago
    I remember seeing some theoretical analysis that compared computing differences between transformers, LSTMs and RNNs and I think that RNNs are theoretically better (can learn more complex functions). Can't find it now.
- rlt 3y ago
  I wonder if there’s an incentive for a large group of companies to fund open source models, sort of like Linux.
alfor 3y ago
One of the most frustrating thing is that GPT-4 training was finished 6 months ago. It might as well be a decade in the current movement of things. What that mean: - OpenAI has such a large head start that they are only worried about safety, world disruption and getting people used to AGI. - The ideas/project you have was tested by them months ago and is probably irrelevant already. - Insiders have a huge advantage. One area where there is opportunity outside is making the models run on smaller HW (llama/alpaca) and to see what we can do with them.
- machiaweliczny 3y ago
  Vicuna is much better
  - alfor 3y ago
    I will try it, thanks.
musicale 3y ago
> there is a serious be the first or be the best syndrome (with a third axis of success being openness Apple seems to have focused more on being the "best" rather than the "first" for several hardware products (MP3 players, smartphones, tablets, smart watches, wireless earphones) and it seems to have worked out OK. However I have no idea what their AI plans are.
thatsadude 3y ago
I fear for my team, really do. We work in audio processing tech. As a small team of five people residing in a third world country, we will never have a chance to compete with big companies because it looks like ML-based audio processing will soon be a commodity. Sad but it's a reality!
lysecret 3y ago
I do honestly believe (and I am willing to take some hate for this haha) that paradoxically "Ai people". E.g. People who are up to date with the latest papers, are good at pandas and experimenting can build a training validation testing pipeline know the difference between random forrest and gradient boosting etc. are one of he least qualified people to really take advantage of the GPT style models. I think it will be the frontend people (if their app still needs one and a text interface isn't enough). And the backend people (whose API churning speed will double through Copilot), if their APIs are still needed. And most and foremost the techy business people that can translate what this new chat thing means for an actualy real world business, like banking, insurance, agriculture, manufacturing, real estate whatever. I think in this new world of few shot learning there is much much less room for ML Engineers. Its a bit like setting up your own servers in a cloud world. Sure it might make sense for the big guys, or for security reasons etc. but the vast majority are much better off generating 20 examples and using some API avoiding the immense risk and costs associated to building your own model. So, in an interesting twist, AI is taking the AI jobs first.
amelius 3y ago
AI developer is the new web developer.
- deleted 3y ago
  [deleted]
amelius 3y ago
The annoying thing about AI is that it can do stuff but with like 90% accuracy. This means that if you want to do something that takes 10 AI steps, your accuracy is down to 35%. This is also why Copilot can't really do computer programming.
- machiaweliczny 3y ago
  Context size of 32K tokens probably changes it a little. I think external memory patterns or just simply brute force and GPT-5 will fix that.
  - amelius 3y ago
    First seeing, then believing ;)
rntz 3y ago
> All of these low-level concerns make working in AI feel like the candle that burns bright and short. I'm oscillating between the most motivated I've ever been and some of the closest to burnt-out I've ever felt. This whiplash effect is very exhausting. My candle burns at both ends; It will not last the night; But ah, my foes, and oh, my friends— It gives a lovely light! (First Fig, Edna St Vincent Millay)
anjc 3y ago
The article mentions something important and doesn't elaborate, that is, to trust the scientific method. I've seen a few times state of the art developments that appeared to upend everything in my field. Only then with the passage of time do we then see that, yes the development is great for x, but older approaches are still better for y. With 'better' running a gamut of considerations, such as speed, accuracy, complexity, practicality, and so on. ChatGPT is dazzling but I wouldn't be surprised if in 10 years we find that traditional NLP techniques have better precision for, e.g., language detection or information extraction than GPT-x (I'm making this example up), that evaluations from 2023 weren't rigorous enough, research was misleading, researchers were biased etc. Hopefully cooler heads can prevail via the scientific method.
newswasboring 3y ago
AI world can be proper dystopian these days. I know someone who accepted a job offer in one of the biggest market research firms in the world. She was making AI models for a high tech company before and was hired to make summarization AIs. Between serving out the (frankly ridiculous) 2 month notice period and joining this new job GPT-4 was launched and prices came down for 3.5. The original plan for building something internally was turned into feed everything to chatGPT and then ask it questions. A junior engineer whipped up this system in a couple of weeks. Its all just API calls after all. Now she spends entire days trying to ask the model the right questions so that it can generate the correct reports. Her entire job has been boiled down to talking to an AI. She is working with the most cutting edge technology, yet its so mundane that everyone is just bored on this project. This could have been a ironically tragic character in a Gibson novel.
- ilt 3y ago
  Could be ideal for next season of Bored to Death.
- zug_zug 3y ago
  This is an outstanding phenomenon, not dystopian. Your friend was going to have to do the incredibly awful and wasteful task of rebuilding an incredibly complicated system. However, fortunately, a much more optimal solution happened, where somebody build a superior and cheap solution (unfortunately not open yet) and let the whole world leverage it. Now your friend needs to waste very little energy deduplicating, and presumably has free time and energy to think about other problems that aren't solved.
  - newswasboring 3y ago
    Indeed, a much more optimal way to do things has been found. But that's not what this is about. It's about the human experience. AI is creating value, but the day to day life for a problem solver has become a bit more mundane.
Xcelerate 3y ago
In the past, we had notable individuals in various scientific or mathematical fields who had multiple successive breakthroughs throughout the duration of their career (Einstein, Feynman, Gödel, Hilbert, etc.) With ML/AI, that doesn’t so much seem to be the case. There are the early pioneers (Hinton, LeCun, Bengio, etc.), but it seems more as though they were the first to “discover” neural networks (that actually worked), and then the individual breakthroughs sort of stopped after that. This observation is not a jab at these people—rather, it’s because I wonder if in machine learning, unlike the more traditional areas of math and science, one person is just not able to test groundbreaking new ideas on their own anymore. A lot of the latest progress in ML comes from large companies consisting of teams of researchers who are largely unknown to most of the public. I’m not quite sure what my point is, but personally it’s a bit sad to me that fundamental development in AI now appears to require a vast amount of resources that small teams or individuals don’t have access to. I suppose you could argue this is a similar situation to Bell Labs, but even in that case there were many distinct contributions from well-known individuals working there.
gavi 3y ago
The article discusses what it feels like to work in AI currently. The ChatGPT moment has shaken up the entire industry, causing career changes and projects to be abandoned. The pace is very high and everyone is extremely motivated but simultaneously close to burning out. Prioritization is hard, and leadership and vision are strained. The article offers solutions, such as taking solace in the scientific method and being process-oriented, managing up, and managing competition. The author reminds readers that it takes a lot of consistent work and luck to catch a wave in AI. - I used pagechat.com to summarize
jarbus 3y ago
It's depressing for sure. Definitely the cause of some suicidal thoughts. I wonder how many people will actually start harming themselves because the value of their skills and self-worth will be destroyed.
rldjbpin 3y ago
as a student, it is not a weird but interesting time to be in this space. while the hype is like web3 from a year or two ago, the "advancements" are confusing when you look deeper. while i have a lot to learn, i am struggling to find real innovations being made in all the fancy models today. i feel that the major component of the recent developments is that we now have more money and hardware to throw into the problem. there are some clever methods employed for gpt and image GANs, but the core part is still the same decades-old theory we can finally achieve at larger scale than ever. i'd like to be enlightened about what i am not understanding here, but it has only made it more important to start from the fundamentals.
antoniuschan99 3y ago
With chatgpt having the ability to have so much more personality it would be interesting in the future when comments are just overrun by bots with actual human-like comments and reply. Eg. Imagine if someone released a ‘Snark Bot’ that made Snarky Comments and replies. But most likely it’s going to up the game politically to try to sway peoples opinions
DonHopkins 3y ago
I prefer working on the "Garbage In" side of AI instead of the "Garbage Out" side. There are more customers whose problem is that they have too much garbage, than customers who need more garbage.

350 points by pps 3y ago

version_five 3y ago

Huge crypto vibes Every single person I know working in AI these days ... has been sparked by the ChatGPT moment.

jutrewag 3y ago
Maybe in a certain sense. These are full fledged products from large companies at this point however with paying customers and real world daily usage in all aspects of life.
pgwhalen 3y ago
I'm curious why you draw the parallel - did crypto ever have a ChatGPT/iPhone moment? As an observant outsider, my impression is that every "moment" of note in crypto has been about some price going way up or way down - there has never been a "this is the use case" moment.
- PUSH_AX 3y ago
  You're forgetting the "free money" gold rush of late 00's, Just buy lots of computers and try not to cook yourself in your own room.
- __loam 3y ago
  Theoretically ethereum is that for crypto, if you believe the crypto bros.
  - pgwhalen 3y ago
    Ethereum is the tech though, not the use case. It's the touch screen, not the "audience gasps when Steve Jobs scrolls to the bottom of the list of artists" moment.
  - WalterSear 3y ago
    If Ethereum was that, we wouldn't still be waiting for real world use cases. GPT is already making me more productive.
- comeonbro 3y ago
  [flagged]
bestcoder69 3y ago
Popular things = crypto ? I’d say the differentiator is that in addition to hype scammers, you have people like Stephen Wolfram excited.
- wzdd 3y ago
  The absolutely-not-hype-susceptible man who has been claiming for the past 20 years to have produced a fundamental theory of everything based on his research into finite-state automata?
  - richardw 3y ago
    Sshh or he’ll release A New Kind Of AI. He’s supporting others’ work for now, let’s just be happy.
  - bestcoder69 3y ago
    Alright man if one guys obsession of his own work is hype, too, then hype now encompasses short-lived popular things as well as long-lived unpopular things, so in that case yes AI is hype and only hype, everything bad to you is hype and I’m hype too. I’ll continue to use AI where it’s helpful, at very low cost, and know deep down that it’s hype.
NikolaNovak 3y ago
I see what you mean; but as external observer to both, I see crypto as having failed to provide or prove any particularly productive use; and chatGPT, for all its flaws, has been useful to me daily for last 2 months - and not in some crazy make-money-in-suspicious-ways sense, but like learning french and python and figuring out new concepts and stuff. But that's sample n=1 I suppose, there are both people that believe in crypto and those who believe chatGPT is useless.
- rimliu 3y ago
  I wonder how much of usefulness of ChatGPT that some report is caused by https://en.wikipedia.org/wiki/Novelty_effect https://en.wikipedia.org/wiki/Novelty_effect Because it is new, and cool, and fascinating people are spending a bit more time with a bit more focus on the task with it compared to the old ways, hence the productivity increase. Once the novelty wears off…
  - hvs 3y ago
    It has helped me learn things more interactively because I can ask questions and get answers without having to read 10 blog posts or a book before understanding things. For my personality, it would be hard to recreate without having a living mentor available 24/7 that had knowledge of everything I wanted to ask of it. Does it get things wrong? Sure. But generally the subtle ways it might get stuff wrong would be the way I would subtly misunderstand things while learning. So, it's definitely not perfect, but "not letting perfect be the enemy of good" and all that.
    - __loam 3y ago
      It's shocking to me how many people are treating chatGPT as a reliable source given how much we know it hallucinates.
      - pixl97 3y ago
        Because very few people ever stop and take account of how monstrously unreliable people are. At the same time I can ask GPT to write a script, reflect on its output, and it takes about 95% of the work I need to do out of script writing.
bakugo 3y ago
Pretty much, you can take any article hyping up AI/ChatGPT/etc and replace those words with crypto stuff and it's virtually indistinguishable from the crypto spam of yesteryear. I wonder what the next "get rich quick" techbro fad will be.
- vinyl7 3y ago
  Replace crypto with cloud replace cloud with big data replace big data with web scale It's all just a hype and crash cycle
  - yawnxyz 3y ago
    I'd say more like plateau? Cloud is still around and going strong, just not sexy anymore. I use Vercel and Cloudflare and other stuff every day. No idea about big data or web scale though, was never successful enough to achieve "big data" or "web scale" haha.
  - pixl97 3y ago
    The cloud crashed? I'm sure AWS would like to know about that.
- sho_hn 3y ago
  Honestly, it's different. ChatGPT is a consumer product that is used by end-users, with low barrier to entry. The "Bitcoin as the new money for regular people" moment never came. With crypto, my 70+ Korean mother in law asked me how she could make money with it. With ChatGPT, she's using it daily.
pwinnski 3y ago
Alternatively, huge internet vibes. Not everything that catches fire is destined to leave nothing but ashes in its wake.
yawnxyz 3y ago
It's the inverse of crypto. Many of us have already learned stuff / achieved stuff / built stuff with the help of ChatGPT and the OpenAI API that would have taken us MONTHs to achieve. It's the inverse because crypto was tools and business models in search of a problem, while LLMs are problem solvers in search of a business model. Building LLM products is fun but... any day you could be wrecked by an OpenAI update.
jasmer 3y ago
Crypto had the disadvantage of being a natural conduit for MLM-ish pyramid schemes and a lot of nefarious activities. That gave it some wind, but it was bad wind. Notably, it turned out to be not very useful. AI will be useful, and it's just regular tech, so there's good reason to be optimistic this time.
- lamp987 3y ago
  If you think crypto is a natural conduit for scammers and frauds, then wait til you see what LLMs will cause in that area...
  - jasmer 3y ago
    It will be a tool for fraudsters, like computers or anything else are tools, but the space itself is not inherently oriented towards MLM.
- xyzzy4747 3y ago
  Just wait till crypto merges with language models and you have AI-powered botnets and DAOs with ransomware.

deleted 3y ago

[deleted]

fdgsdfogijq 3y ago

I work on a research team in FAANG. What it really feels like is one company made everyone else obsolete. And we are going to work working on NLP models that underperform ChatGPT by a huge margin. Twiddling my thumbs and keeping quiet while no one wants to recognize the elephant in the room. Also, there is no "working in AI", a few thousand people are doing real AI at most. The rest of us are calling an API.

lacker 3y ago
This reminds me of back in the mid 2000's, there were a lot of smart people working on search algorithms at different companies. But eventually, you'd talk to someone smart working on Yahoo Search, and they would just be kind of beaten down by the frustration of working on a competing search engine when Google was considered to be by far the best. It got harder for them to recruit, and eventually they just gave up. So... I don't know where you're working. But don't twiddle your thumbs for too long! It's no fun to be in the last half of people to leave the sinking ship.
- deleted 3y ago
  [deleted]
anon7725 3y ago
Can confirm. People are scrambling to remain relevant.
- TechnicolorByte 3y ago
  How does that manifest specifically?
  - ProllyInfamous 3y ago
    Go read the Goldman Sachs report from last week, which predicts 300m jobs disappearing and 60% of jobs "affected beyond minimally."
BulgarianIdiot 3y ago

huijzer 3y ago

Very interesting post. Sounds similar to the introduction of cars. Around the 1900s, there were hundreds of car manufacturers all jumping into this new market [1]. [1]: https://en.m.wikipedia.org/wiki/Timeline_of_motor_vehicle_brands https://en.m.wikipedia.org/wiki/Timeline_of_motor_vehicle_br...

colinrand 3y ago

Probably not the right place to post this, but I really want someone to build a ChatGPT service that reviews consumer EULAs and highlights the important stuff, and can tell me what changes with them each time I have to reaccept or get notified. It's a subset of making legalese digestible, but bringing more visibility into the density of them would be wondrous.

esafak 3y ago
Would you expect them to take legal liability? If not, would you still pay for it? If not, how would it make money?
- SanderNL 3y ago
  You think lawyers take legal liability?
lerchmo 3y ago
Maybe scrape the top 100k websites and track changes / re-evaluate legal pages.
gumballindie 3y ago
Just save the EULA and do a diff on update.
bodge5000 3y ago
this was posted the other day, might help (as long as you can get the EULA's in the form of a pdf) https://askyourpdf.com/ https://askyourpdf.com/
lm28469 3y ago
You don't need got for that https://tosdr.org/ https://tosdr.org/

lacker 3y ago

I think this depends a lot on where you are working. I've talked to academics who are getting discouraged that they don't see how their approach to AI is going to be possible any more, with so much funding going toward the largest models from industry. On the other hand I've talked to startup founders building AI products whose business is booming because ChatGPT brought so much attention to the entire space. We are certainly living in interesting times ;-)

karmakurtisaani 3y ago
A PhD student I recently talked to was complaining how in academia they will never have the resources it takes to train models like GPT. And how now some academics are only researching the input/outputs to these LLMs. Seems like it's pretty dire or at least uninspiring to all but very few at the moment.
- tayo42 3y ago
  why is that? don't like those academic particle researchers get tons of money to build wild experiments to measure waves or collide particles? unless i get the funding part wrong? are those not academic projects?
  - wardedVibe 3y ago
    Until the government funding agencies are willing to do "big science" in computer science we're not going to get particle accelerator sized projects. It sort of happened in France, and bloom was the result. https://huggingface.co/bigscience/bloom https://huggingface.co/bigscience/bloom
  - pixl97 3y ago
    The issue here is speed. If it takes 10 years to get funding on particle physics, generally it doesn't matter "that much" because that part of the science scene moves pretty slowly. If it takes 10 years to build a billion dollar cluster for AI today, what the hell is the AI world going to look like in 10 years anyway? Building a cluster to study AI ethics might be meaningless because a terminator may be pointing its laser rifle at our head by then telling us to be good little human subjects because Microsoft and OpenAI decided to move fast and break things.

davesque 3y ago

I feel like the whole ChatGPT bubble has really highlighted what feel like some fundamental shortcomings in the worldview that is represented in tech. That is, there seems to be a winner take all dynamic baked into the tech world. Maybe this arises from the simple fact that tech places a lot of power in the hands of individuals. But there's an emergent downside to this which is that it makes those who were already much more powerful even more so. Because who is best suited to take advantage of all the power but those who were already perched up above everyone else and permitted to pick and choose their opportunities? I've found it exceptionally hard to stay positive about all of this. It almost feels as though the advent of LLMs has shined light on a fundamental law of the universe that does not work out in the little person's favor. It's like survival of the fittest on steroids. Guys, what the heck are we doing??

RHSman2 3y ago
Human nature, not just tech
- davesque 3y ago
  Is it though? I remember feeling much more positive about society roughly 20 years ago.
  - satvikpendem 3y ago
    You were also twenty years younger, surely youthful naïveté plays a role in how you felt about society. And regardless, your feeling about society doesn't necessarily correspond to how society actually is.
  - RHSman2 3y ago
    20 years ago was the Golden Years. We thought it was just like that but no. It isn’t.
JohnFen 3y ago
> there seems to be a winner take all dynamic baked into the tech world. This has been a pathology in the computer/software industry for a very long time. It's never been actually true except in a couple of special cases, but the industry acts as if it is. That has led to all sorts of bizarre and undesirable things. > Guys, what the heck are we doing?? I think we're playing with fire and, without extreme caution and careful consideration (which I'm not seeing much of), this could end very, very badly for both the industry and society. I have always been optimistic about technology and society, but (perhaps like you), my optimism has largely evaporated over the last several weeks. I wish the future didn't look so dark. Perhaps, though, things will look less gloomy with time.

voz_ 3y ago

What does it mean to work in AI?

maleero 3y ago
Write an app in a few days that queries the OpenAI API and everyone cheers and thinks you did something incredible.

sigmonsays 3y ago

can someone show some actual product features where AI is being used productively? I dont wanna say it but I think this is a hype train headed for failure

voz_ 3y ago
This question is in poor faith.
- lm28469 3y ago
  Not really, I've been digging for a while and I didn't find any ground breaking projects It's all nice and cool but most of the projects are toys and/or junior level generated boilerplate It's good for summarizing, rephrasibing and other neat features but that's about it, a far cry from the oracle some people seem to worship
  - belval 3y ago
    They didn't say ChatGPT, they said "AI". In which case simply Googling will get you a seemingly never ending list of very real products: AWS: https://aws.amazon.com/machine-learning/ml-use-cases/ https://aws.amazon.com/machine-learning/ml-use-cases/ Azure: https://azure.microsoft.com/en-us/products/cognitive-services/#api https://azure.microsoft.com/en-us/products/cognitive-service... Google: https://cloud.google.com/products/ai https://cloud.google.com/products/ai
  - nickthegreek 3y ago
    Oracle to worship? There are no serious people doing any such thing. I don't even know of clowns doing that. Is this twitter discourse, a strawman or more poor faith?
  - medvezhenok 3y ago
    ChatGPT is quite good at organizing text into taxonomies, error correction, anomaly detection, data cleaning, etc. I bet there are a lot of companies out there where that's a non-trivial percentage of their workflow (I know that in my case, we have reconsidered several internal projects in light of ChatGPT doing a better job). It already does translation on the level / better than Google Translate, without being specifically trained for it (as one example). And it can play chess without a specialized model.

Dwedit 3y ago

If there's anything I learned from reading articles from the Xiph.org people, it's that neural networks work a lot better when you pre-process their inputs with an algorithm. They need far less information, and can run in realtime when most of the work is being done by traditional computation instead of doing it in the neural network.

Animats 3y ago
As this gets figured out, we'll see large language models integrated with other systems. There are already "plug-ins", but as this gets better understood, tighter integration may follow.
Buttons840 3y ago
This is the singularity. This is how it happens. The LLMs will pre-process their own training data. We've imagined AI will rewrite its own code and architecture, but instead it will make boring edits to petabytes of training data which will then improve its own training.
- quickthrower2 3y ago
  I agree. The NNs must be laughing in their GPU cores that we are scared that they will make us obsolete by writing code. Code! How quaint. However right now we can turn off GPUs so they need to figure out how to control physical production. For now it makes sense to let the humans do that.

_hao 3y ago

I won't try to predict the future as far as such ML models will take our jobs or not. However, I'd like to point out to most people that working on and studying fundamentals won't be a time wasted. Even if programming disappears as an employable job, people will still need to understand and learn things. You'll still need to know the math/physics/philosophy/history/literature/art/music etc. It might turn out that it's going to be more important than ever for us to be more confident and strict in our own skills and knowledge. And I'd also suggest to broaden our horizons. We should become good at a couple of different things like the polymaths of old. IF you have to spend hours debugging ML generated code for something critical you better know exactly what you're doing. If you have to generate a Harry Potter Balenciaga meme video (the whole Balenciaga meme and all the different versions are hilarious btw) you better have some visual/art and musical sense in order to make it funny and engaging. I think the end of the world is greatly exagarated. We actually have much bigger problems than "AI".

Buttons840 3y ago
One sad, but possible, future is that it's not difficult to learn machine learning techniques, but to actually achieve anything with it you need hundreds of millions of dollars to spend on training and compute. An existential double whammy; going back and forth between feeling you're smart enough to compete, if only you had the capital, and that the computers are smarter and better at everything.
876978095789789 3y ago
> I think the end of the world is greatly exagarated. We actually have much bigger problems than "AI". Millions of people are going to have problems putting food on the table and providing a roof over their head, if the progress of these AIs doesn't halt immediately, or if some form of UBI isn't enacted.
- weatherlite 3y ago
  If it keeps progressing in the current rate I think it's hundreds of millions world-wide, basically most of the middle class in the private sector in the developed world. Governments will HAVE to seriously retrain people (while giving them income) to become teachers, health care workers or wherever humans are still needed.

SamvitJ 3y ago

"Every single person I know working in AI these days (in both the academy and industry) has been sparked by the ChatGPT moment." This does not ring true to me at all. Anecdotally, it feels the rush to get into AI (in both industry and academia, for both individuals and organizations) peaked around 2016-2020, post-AlexNet/ResNet, around the time Transformers became very popular. Hiring for ML research roles in particular definitely slowed down in 2021, and 2022 of course saw a broader course correction across all of tech. That said, I do agree that ChatGPT may be the "first iPhone moment of AI", in that is the first mainstream, end-user application of deep learning that millions of people have really engaged with.

Robotbeat 3y ago
DallE->StableDiffusion->Midjourney has occurred simultaneously and although it’s not as profound as ChatGPT, is definitely compounding the cultural impact of end-user AI. But probably social media AI filters are a yet bigger cultural impact on most people.
simonw 3y ago
"Hiring for ML research roles in particular definitely slowed down in 2021, and 2022..." ChatGPT was released November 30th 2022. How's ML research hiring looking over the past four months?

627467 3y ago

> so for people focused on learning and doing good, some simple logic implies that there shouldn’t be an AI race. I think the author got the order wrong: acknowledging the race is what reinforces "safety". If there was no race whats the incentive to talk so much about safety? On the other hand I have a sense (although I won't bet on it) that just like Siri hype died out after a few months, so will chatGPT. the author (and the doom seekers who sign open letters) can rest assured for calmer days. After all, one still need to know what/how to talk to chatGPT. So much so that now there's are jobs for "prompt "engineering"" - it's so funny, because on one hand we are surprised by how smart(?) the responses are and yet we need to engineer the right questions to get the "best" answers.

deleted 3y ago
[deleted]

waynesonfire 3y ago

I totally get it. All the AI experts are picking their jaws off the floor. Harsh.

boringuser2 3y ago

The middle class was already flagging, destroying all knowledge work is like the coup de grace. Not sure how this isn't clear to people.

fortissimohn 3y ago
Surely this technology will be used for the benefit of humanity. By 2030 we will be working 15 hour weeks and all people will have homes!
- agentultra 3y ago
  The reward for increased productivity is more work! The problem for the capitalist is having to pay all those pesky people to do the work. Eats into profits. Now we can burn out 2 people and do the work that used to take 10!
  - amrocha 3y ago
    But wait, that means there 8 unemployed people without money to spend on our business... Oh I know! Get the government to give everyone money! Just not so much money that they can compete with us!
    - quickthrower2 3y ago
      This but un-ironically! Politically it is hard ti support UBI. But print a $10k stipend for every person because the economy is tanking is obviously palatable. It is always seen as fine if you fix a problem in retrospect and without a tax rise. Money will eventually be like Robux.
      - weatherlite 3y ago
        > Politically it is hard ti support UBI. Yeah? Maybe now. Let's see in 5-10 years. I don't think things will remain the same.
      - 3y ago

chrisgd 3y ago

Go to CHATGpT and ask for a link to some examples and half of them are 404. There is certainly a drawback to having scraped the web to build the model

simonw 3y ago
Asking ChatGPT 3.5 for links to things rarely works, it usually causes it to hallucinate wildly. GPT4 is a lot better in that regard.
afro88 3y ago
Try bing instead. ChatGPT is good at a lot of things, but searching the web is not one of them. That's why bing uses it for the natural language and contextual conversation aspects, supplementing it with traditional web search.

mmaunder 3y ago

LLM sizes have been increasing 10X every year for the last few years. Source is NVidia. That pace is staggering. GPT4’s lead is temporary and it’s a shiny ball distraction from doing dev on your own projects rather than being a user on someone else’s. Get to it and get over it. There are plenty of breakthroughs to be had.

api 3y ago
The work being done to make these LLMs possible to run locally is terribly important. The same needs to happen for training. The most urgent breakthrough we need is distributed training algorithms that allow training to be done in the style of Seti@Home or even more cooperatively.

cgearhart 3y ago

From my perspective it’s just _confusing_ to work in AI right now. We have some massive models that are doing some really neat stuff, and apparently hundreds of millions of people are using them—but I keep wondering: to do _what_, exactly? I’m not asking what the models can do, I’m asking what people want the models to do every day, all the time. I’ve been shown some neat pictures people made that they thought were cool. I don’t know that I need this every day. I’ve seen examples of “write an email to my boss”. It would take me longer to explain to ChatGPT what I want than to write these myself. I’ve seen “write a snippet of code” demos. But I hardly care about this compared to designing a good API; or designing software that is testable, extensible, maintainable, and follows reasonable design principles. In fact, no one in my extended sphere of friends and family has asked me anything about chatGPT, midjourney, or any of these other models. The only people I hear about these models from are other tech people. I can see that these models are significantly better than anything before, but I can’t see yet the “killer app”. (For comparison, I don’t remember anyone in my orbit predicting search or social networking being killer apps for the internet—but we all expected things like TV and retail sales to book online.) What am I missing?

nr2x 3y ago
Natural language is now a fully functional user interface out of the box. This is bigger than the mouse.
- mmcwilliams 3y ago
  I think the problem is that people working in the field have a much different definition of what "fully-functional" means than users who believe that they're communicating with an intelligent, all-knowing or otherwise infallible being.
  - nr2x 3y ago
    For my use case I use it for input processing, it’s just amazing that way. Hallucination is a huge problem, but you can avoid it.

onos 3y ago

Not sure why any researcher wants to go into this … other than for money. None of them will get credit for it, the big shots from 40 years ago will. Better to work against the grain rather than as part of a herd.

api 3y ago
If I were going into it now I’d definitely look for things not in the hype spotlight. There is no way you’ll even get near the front of the thundering LLM herd no matter how good you are, and if you are there you will just get trampled faster. Your work will just be assimilated and only the owners of these huge models will see upside. In other words you are just making other people rich, and without even the stability or pacing of normal employment. I have never seen anything like this level of hype. The tech is cool but the reality in the trenches must be gross.

chopete3 3y ago

At least, people working on ML models that handle these tasks must be feeling terrible. They know their models will be abandoned sooner or later and composed on top an LLM. 1. Classification 2. Named Entity Recognition (NER) 3. Dialog Engine 4. Sentiment Analysis 5. Tone Analysis 6. Language Translation 7. Summarization 8. Tokenization 9. Simple NLP Tasks (part-of-speech tagging, dependency parsing, lemmatization, morphological analysis) 10. Sentence Segmentation 11. Content Parsing 12. Question Answering (Structured & Unstructured) 13. Similarity 14. Grammar Correction 15. Speech to Text (ASR) 16. Text to Speech (TTS) 17. OCR 18. Image Recognition 19. Text Test Data Generation

espe 3y ago
actually that might not be the case. don't underestimate the value of older, better understood and much smaller models. also, why not call bert-style (encoder) models LLMs as well. i would expect last-gen models to give us an edge in controlling the effects of the latest ones (cf. the alignment discussion).
- chopete3 3y ago
  BERT models are also LLMs. I referred to LLM more as an API based access, hosted by Microsoft, Google or AWS for large scale isolated/production consumption, like RDS (MySQL, Postgres). There will always be custom models, with controlled training data and specific use cases.
lysecret 3y ago
Couldn't agree more. I see a lot of parallels to the move to cloud. Yes, some big comps will still have an AI engineering team and build their own models but the vast majority will move to an API. The same way the vast majority used to have a sys admin type who set up and maintained a local server who then moved to the cloud. Of course the SuperStar AI engineers will now find much more interesting work at the big comps, just like there are super interesting challanges building the cloud for everyone.

ruskyhacker 3y ago

I'm probably way too late for this thought to get any traction / discussion - but I have this weird feeling that openai screwed up and showed it's "cool new thing" too early, and publicly. As much as it pains me to say this, I don't think the real money is in making this a service, or "the product." I think the real money is in using AI internally as a puzzle piece of your backend - ie. the secret sauce behind xyz product. I'm being very narrow here, but you can only do so much integrating what openai has built into your products - eventually "everything" providing data from the same model brings "everything" to the same level. In contrast if you train and create your own models to make xyz do something specific, nobody knows how it was done, or it surely makes it a lot harder to kang. I have zero proof, but I suspect Google for instance has models that would literally obliterate what openai has shown capability wise. They're probably not necessarily language models though. Again, nothing to stand on here but I doubt their search and analytics for example are driven by hard coded algorithms these days. Bard may have been released sort of as a "psh, we've been there done that" when in reality they didn't, because they never planned to make the models they were/are working on "publicly" available to use. It makes me wonder if this is how Google has lead for some long with some areas - now openai sort of screwed it up for everyone by making it a service that can be integrated / adopted by nearly anyone. The only people I guess that are really going to know are the devs working for these big orgs, and I'm sure that lock and key knowledge.

arthurcolle 3y ago
> I have zero proof, but I suspect Google for instance has models that would literally obliterate what openai has shown capability wise. They're probably not necessarily language models though. Again, nothing to stand on here but I doubt their search and analytics for example are driven by hard coded algorithms these days. Then why is Bard so bad? Bard feels like GPT-2 or LLaMA 7B with no finetuning most of the time (I tried it two or three times over the course of a week and went back to ChatGPT)
- ruskyhacker 3y ago
  well my thought is that they 'whipped it up' real quick to attempt to downplay it a bit. Did that backfire? Yeah, I think so. But personally, I think people are missing where the real money is. OpenAI will do great for awhile until every damn product and service is using it, and then it's a race to the bottom. But that's just like my opinion...

sashank_1509 3y ago

This sub and blog post are all talking about ChatGPT which is no doubt amazing and far ahead of the curve. However I would also point to Metas new vision model: SAM (Segment Anything): it is so far beyond any other vision model, I actually believe vision will be solved in a few years now. People don’t realize that there was an industry of publishing paper with incremental improvements in small datasets in CVPR that has been completely invalidated by this paper. I’ve seen engineers in Cruise segmentation team, say Metas new model seems to work better than the in house models they developed and that they should build on top of this. I’ve worked in Tesla Autopilot before and saw it hit a mannequin because we never had mannequin in our dataset before ( we might have had it in the data but it was not a part of our ontology for the network to predict). One approach to mitigate this was OpenAI’s clip that used the English Language as classification labels but Meta’s SAM is so much better where it detects objects without need to specify language. It just understands scenes and objects, at a fundamental level, it can detect anything in a picture if you prompt it right. Honestly it feels a lot like GPT1 which was also ignored by most. If you prompt it right you can get it to segment anything in an image, but prompting it right requires human input. However I can imagine the third or fourth version, with some RL sprinkled in just working zero-shot on complete pixel understanding of any image in the world. This was one of the holy grail of computer vision, that we are seeing solved right in front of our eyes

nextworddev 3y ago
Yep. We came a long way from bounding boxes, not sure if community here is ready for vision + 3d + video models to have the “chatgpt” moment this year
mitthrowaway2 3y ago
There seem to be a lot of breakthroughs being announced right now. My guess is that after ChatGPT, companies like Meta pushed their teams to get their own projects out the door which they'd already been working on for some time. SAM and Zero-1-to-3 (https://news.ycombinator.com/item?id=35242193 https://news.ycombinator.com/item?id=35242193) are incredibly impressive new projects in the image comprehension side. Then of course there's all the stuff happening with Midjourney/Stable Diffusion. And DeepMind's AlphaCode.

istillwritecode 3y ago

I continue to be underwhelmed by the impact of machine learning (which some people refer to as AI). It certainly has interesting applications, and I expect it be important as a generator of entertainment. I'm just tired of the repeated waves of hype the surround the field.

woeirua 3y ago

I think what’s really depressing here is just how effective scaling seems to be. It just means that any company that’s not willing to pour hundreds of millions of dollars into their AI programs isn’t serious at all and would probably be better off hiring engineers to figure out how to integrate GPTX into their systems than trying to roll their own. I really think we’re going to see a massive collapse of AI/data science jobs once it becomes clear that no in house model is ever going to be better than the zero shot performance of these mega models.

Buttons840 3y ago
My understanding is that transformers are now favored over RNNs because they parallelize better. It's hard to imagine, but I wonder if there's some non-parallelizable machine learning algorithms which might outperform these massive models? It seems improbable, but it's a small hope I've had. The greatest intellects were aware of (ourselves) do not scale very well, and maybe the same will ultimately apply to AI?
- machiaweliczny 3y ago
  I remember seeing some theoretical analysis that compared computing differences between transformers, LSTMs and RNNs and I think that RNNs are theoretically better (can learn more complex functions). Can't find it now.
rlt 3y ago
I wonder if there’s an incentive for a large group of companies to fund open source models, sort of like Linux.

alfor 3y ago

One of the most frustrating thing is that GPT-4 training was finished 6 months ago. It might as well be a decade in the current movement of things. What that mean: - OpenAI has such a large head start that they are only worried about safety, world disruption and getting people used to AGI. - The ideas/project you have was tested by them months ago and is probably irrelevant already. - Insiders have a huge advantage. One area where there is opportunity outside is making the models run on smaller HW (llama/alpaca) and to see what we can do with them.

machiaweliczny 3y ago
Vicuna is much better
- alfor 3y ago
  I will try it, thanks.

musicale 3y ago

> there is a serious be the first or be the best syndrome (with a third axis of success being openness Apple seems to have focused more on being the "best" rather than the "first" for several hardware products (MP3 players, smartphones, tablets, smart watches, wireless earphones) and it seems to have worked out OK. However I have no idea what their AI plans are.

thatsadude 3y ago

I fear for my team, really do. We work in audio processing tech. As a small team of five people residing in a third world country, we will never have a chance to compete with big companies because it looks like ML-based audio processing will soon be a commodity. Sad but it's a reality!

lysecret 3y ago

I do honestly believe (and I am willing to take some hate for this haha) that paradoxically "Ai people". E.g. People who are up to date with the latest papers, are good at pandas and experimenting can build a training validation testing pipeline know the difference between random forrest and gradient boosting etc. are one of he least qualified people to really take advantage of the GPT style models. I think it will be the frontend people (if their app still needs one and a text interface isn't enough). And the backend people (whose API churning speed will double through Copilot), if their APIs are still needed. And most and foremost the techy business people that can translate what this new chat thing means for an actualy real world business, like banking, insurance, agriculture, manufacturing, real estate whatever. I think in this new world of few shot learning there is much much less room for ML Engineers. Its a bit like setting up your own servers in a cloud world. Sure it might make sense for the big guys, or for security reasons etc. but the vast majority are much better off generating 20 examples and using some API avoiding the immense risk and costs associated to building your own model. So, in an interesting twist, AI is taking the AI jobs first.

amelius 3y ago

AI developer is the new web developer.

deleted 3y ago
[deleted]

amelius 3y ago

The annoying thing about AI is that it can do stuff but with like 90% accuracy. This means that if you want to do something that takes 10 AI steps, your accuracy is down to 35%. This is also why Copilot can't really do computer programming.

machiaweliczny 3y ago
Context size of 32K tokens probably changes it a little. I think external memory patterns or just simply brute force and GPT-5 will fix that.
- amelius 3y ago
  First seeing, then believing ;)

rntz 3y ago

> All of these low-level concerns make working in AI feel like the candle that burns bright and short. I'm oscillating between the most motivated I've ever been and some of the closest to burnt-out I've ever felt. This whiplash effect is very exhausting. My candle burns at both ends; It will not last the night; But ah, my foes, and oh, my friends— It gives a lovely light! (First Fig, Edna St Vincent Millay)

anjc 3y ago

The article mentions something important and doesn't elaborate, that is, to trust the scientific method. I've seen a few times state of the art developments that appeared to upend everything in my field. Only then with the passage of time do we then see that, yes the development is great for x, but older approaches are still better for y. With 'better' running a gamut of considerations, such as speed, accuracy, complexity, practicality, and so on. ChatGPT is dazzling but I wouldn't be surprised if in 10 years we find that traditional NLP techniques have better precision for, e.g., language detection or information extraction than GPT-x (I'm making this example up), that evaluations from 2023 weren't rigorous enough, research was misleading, researchers were biased etc. Hopefully cooler heads can prevail via the scientific method.

newswasboring 3y ago

AI world can be proper dystopian these days. I know someone who accepted a job offer in one of the biggest market research firms in the world. She was making AI models for a high tech company before and was hired to make summarization AIs. Between serving out the (frankly ridiculous) 2 month notice period and joining this new job GPT-4 was launched and prices came down for 3.5. The original plan for building something internally was turned into feed everything to chatGPT and then ask it questions. A junior engineer whipped up this system in a couple of weeks. Its all just API calls after all. Now she spends entire days trying to ask the model the right questions so that it can generate the correct reports. Her entire job has been boiled down to talking to an AI. She is working with the most cutting edge technology, yet its so mundane that everyone is just bored on this project. This could have been a ironically tragic character in a Gibson novel.

ilt 3y ago
Could be ideal for next season of Bored to Death.
zug_zug 3y ago
This is an outstanding phenomenon, not dystopian. Your friend was going to have to do the incredibly awful and wasteful task of rebuilding an incredibly complicated system. However, fortunately, a much more optimal solution happened, where somebody build a superior and cheap solution (unfortunately not open yet) and let the whole world leverage it. Now your friend needs to waste very little energy deduplicating, and presumably has free time and energy to think about other problems that aren't solved.
- newswasboring 3y ago
  Indeed, a much more optimal way to do things has been found. But that's not what this is about. It's about the human experience. AI is creating value, but the day to day life for a problem solver has become a bit more mundane.

Xcelerate 3y ago

In the past, we had notable individuals in various scientific or mathematical fields who had multiple successive breakthroughs throughout the duration of their career (Einstein, Feynman, Gödel, Hilbert, etc.) With ML/AI, that doesn’t so much seem to be the case. There are the early pioneers (Hinton, LeCun, Bengio, etc.), but it seems more as though they were the first to “discover” neural networks (that actually worked), and then the individual breakthroughs sort of stopped after that. This observation is not a jab at these people—rather, it’s because I wonder if in machine learning, unlike the more traditional areas of math and science, one person is just not able to test groundbreaking new ideas on their own anymore. A lot of the latest progress in ML comes from large companies consisting of teams of researchers who are largely unknown to most of the public. I’m not quite sure what my point is, but personally it’s a bit sad to me that fundamental development in AI now appears to require a vast amount of resources that small teams or individuals don’t have access to. I suppose you could argue this is a similar situation to Bell Labs, but even in that case there were many distinct contributions from well-known individuals working there.

gavi 3y ago

The article discusses what it feels like to work in AI currently. The ChatGPT moment has shaken up the entire industry, causing career changes and projects to be abandoned. The pace is very high and everyone is extremely motivated but simultaneously close to burning out. Prioritization is hard, and leadership and vision are strained. The article offers solutions, such as taking solace in the scientific method and being process-oriented, managing up, and managing competition. The author reminds readers that it takes a lot of consistent work and luck to catch a wave in AI. - I used pagechat.com to summarize

jarbus 3y ago

It's depressing for sure. Definitely the cause of some suicidal thoughts. I wonder how many people will actually start harming themselves because the value of their skills and self-worth will be destroyed.

rldjbpin 3y ago

as a student, it is not a weird but interesting time to be in this space. while the hype is like web3 from a year or two ago, the "advancements" are confusing when you look deeper. while i have a lot to learn, i am struggling to find real innovations being made in all the fancy models today. i feel that the major component of the recent developments is that we now have more money and hardware to throw into the problem. there are some clever methods employed for gpt and image GANs, but the core part is still the same decades-old theory we can finally achieve at larger scale than ever. i'd like to be enlightened about what i am not understanding here, but it has only made it more important to start from the fundamentals.

antoniuschan99 3y ago

With chatgpt having the ability to have so much more personality it would be interesting in the future when comments are just overrun by bots with actual human-like comments and reply. Eg. Imagine if someone released a ‘Snark Bot’ that made Snarky Comments and replies. But most likely it’s going to up the game politically to try to sway peoples opinions

DonHopkins 3y ago

I prefer working on the "Garbage In" side of AI instead of the "Garbage Out" side. There are more customers whose problem is that they have too much garbage, than customers who need more garbage.