7 ms·
2023: The Year of AI
- Flux159 3y agoI think that it would also make sense to have a diagram that has open source achievements in Ai for 2023 (at least open weight & inference code). A lot of the announcements in the chart are behind an api so can’t be run locally. Just thinking off the top of my head, Segment Anything, Llama 1 and 2, Mistral, Stable diffusion XL, ControlNet, Whisper are all open source AI releases this year.
- derac 3y agoMixtral, CogAgent/CogVLM, Emu-2, tinyllama, phi-2, Bark, xtts, exllamav2 are a handful of other interesting projects that come to mind.
- rightbyte 3y agoThis article seems very corporate centric. Like, I am able to run a ChatGPT3-ish code LLM locally on a 2015 midrange laptop. Just like this: wget https://huggingface.co/TheBloke/deepseek-coder-6.7B-instruct-GGUF/resolve/main/deepseek-coder-6.7b-instruct.Q5_K_M.gguf git clone https://github.com/ggerganov/llama.cpp cd llama.cpp make ./main -ngl 32 -m ../deepseek-coder-6.7b-instruct.Q5_K_M.gguf --color -c 2048 --temp 0.7 --repeat_penalty 1.1 -n -1 -i -ins Haven't people realized what they can run themself?
- swsieber 3y agoI appreciate the sentiment. It definitely seems possible, but it doesn't ever seem as easy as copy + paste if you ever want performance or to go outside the same generic tutorial. > Just like this: Just is doing a lot of heavy lifting there. Is this the first model you came across? We're there any dependencies you had to install? Did you have to check video card compatibility? Where did you get the command arguments. It looks awfully complicated for a "just", as if there's a lot of options that aren't straightforward to use. Etc
- rightbyte 3y agoYe ... with "just like this" I actually mean "proficient with compiling C projects on an Unixy system" which is like years of dev, "power user" or admin experience (not being sarcastic here). For the audience here I would say that "just" is about right though. I have no clue about anything LLM related. I just made it run after reading some comment on HN pointing in its direction. My point is that these locally run LLMs seems way "underreported". I even tried to make a "Show HN" post about it but it got zero interest. But maybe I am missing something?
- ilaksh 3y agoThe Makefile for llama.cpp is really good and doesn't require hacking to make it work.
- mitthrowaway2 3y agoEven on Windows?
- marcosdumay 3y agoIt's probably a hell to run it on Windows. But again, for the audience here, that's not expected to be a large roadblock.
- derac 3y agoIt's actually really easy to run this stuff on Windows. I've run oogabooga, koboldcpp, tabbyapi, and more with no issues.
- benlivengood 3y agoIt builds on Windows with a couple tweaks to the includes with mingw64, I think some of the Windows-related includes need to be explicitly included. This was a few months ago and things may be better now.
- 3y ago
- 23B1 3y agoNo they haven't. It's actually quite weird how the whole hype-cycle is working this time; I think tech journalists are so caught up in the drameh of OAI and whatever they haven't a clue that most people can run most of these novelty UCs on their own. You don't even need a speedy rig!
- rightbyte 3y agoYe it is strange. And in the tail around Sam Altman there seems to be these semi-religious lunatics believing that only some government mandated big tech monopoly can save mankind from the LLMs. Meanwhile amateurs and uni researchers post really interesting work on a site named after a hugging emoji ...
- thefz 3y ago> Haven't people realized what they can run themself? Lots, lots of people do not care.
- Kiro 3y agoWhy would I run it myself when I can run GPT-4 for less than the electricity cost? Running these yourself is not free. OpenAI is heavily subsidized.
- rightbyte 3y agoYe well at some point getting a sausage from Deliveroo or Uber Eats was a no-brainer since it was cheaper than in the actual fast food joint, due to some VC selling a dollar for pennies. Should limit the stock price prospects though.
- tempodox 3y agoI followed your instructions, and it works without a hitch. macOS-13.6 on an Intel iMac with a Radeon Pro 5700 GPU. My first direct contact with an LLM, and I don't understand most of the command line options yet, but it's certainly interesting.
- phillipcarter 3y agoA key difference is that for basic use cases, yes, this is a comparable experience. Where it stops being comparable is general application across literally millions of use cases. The ChatGPT system has proven itself a valuable utility across industries, people, and use cases. No open model I know of can match it yet.
- simbolit 3y agoThey don't have to match the market leader, they just have to be "good enough". There are oodles of use-cases where sending your data to an outside provider is a complete no-go. In these cases OpenAI/Google/whoever-products aren't relevant competition.
- YetAnotherNick 3y agoI keep hearing this, but which company has this policy? The privacy policy of openAI's enterprise(or Azure's) is not a lot different than say AWS, which everyone uses.
- simbolit 3y ago"everyone"? What are you talking about? Have you ever worked in any sector that has security policies? Even if you haven't, perhaps spend 2 minutes using a search engine? Here is a first page result for you: https://www.tomshardware.com/news/samsung-fab-workers-leak-confidential-data-to-chatgpt https://www.tomshardware.com/news/samsung-fab-workers-leak-c...
- austhrow743 3y agoMilitary, banking, health care all store their data on third party servers. It's the standard thing basically everywhere to do so. Your own link shows Samsung using ChatGPT. Not sure what point you're trying to make with it.
- simbolit 3y ago
- guitarlimeo 3y agoThis is generating answers a lot slower on my machine (X1 carbon) than GPT-3.5. :(
- forevernoob 3y agoWould you say that this could run (smoothly) on my X220 with i7-2640M?
- rightbyte 3y agoImagine some old sci-fi movie where the computer write out the answer to a query, as if it actually was some scene worker typing it to some terminal. On this: NVIDIA GeForce GTX 1050 Ti Intel® Core™ i5-8300H × 8 32,0 GiB ram So I guess your computer would be a faster typist?
- vasco 3y agoA few years ago peak HN comment was complaining about wget | sh. Nowadays we just run full on opaque models directly from huggingface without thinking twice about reading anything. Interesting how times change. I wonder what supply chain attacks will come from huggingface, must not be long now.
- fragmede 3y agoThat is, by the by, why models are no longer distributed as Python Pickle files, which can root your box if the model being loaded is malicious.
- nneonneo 3y agoNever underestimate the value of usability. I appreciate being able to run these locally - I do! - but for a lot of people a website is going to be infinitely more convenient. I’m reminded of the classic Dropbox HN comment (https://news.ycombinator.com/item?id=8863 https://news.ycombinator.com/item?id=8863): why use Dropbox when any Linux user can just use curlftpfs?
- andy99 3y agoSomething I find interesting is that in the last year or so, the talk around AI shifted from the model architecture to the trained model. People talk about Mistral 7B e.g., not transformer with rotary position embedding and gelu feed forward network (I don't know Mistral's architecture). Contrast this to a few years ago we'd talk about Resnet or Retina-net or whatever, not so much about the facebook pertain on Image-net when describing the model. In 2018 I remember hearing "architecture is the new feature engineering" (mostly meaning over-fitting I undetstood). Now it's all (mostly) about the dataset and training, the architecture, in 2023, was a minor detail. I personally think architecture will make a comeback soon.
- dartos 3y agoIt’s because weights are where the magic lies, for the most part. Annnd nowadays tools like transformers can automatically select the architecture based on the name of the weights, so when interacting with LLMs, you generally just name the weights. Also MANY models are just using the llama or llama2 architecture.
- kleiba 3y ago> It’s because weights are where the magic lies, for the most part. What's that supposed to mean? Weights are not independent of the underlying architecture.
- dartos 3y agoThey’re not, but the difference between the architecture outputting random garbage and impressive text is all in the weights.
- Nevermark 3y agoAnd a fitting architecture can save orders of magnitude in numbers of weight and training time. There isn’t one that’s more important than the other. There are no weights at all without an architecture. And any architecture improvements that pay off have a multiplier effect on efficiency. Nobody on Earth had the resources to get the performance they are getting on these big models with just a default fully connected layer architecture.
- greatpostman 3y agoTLDR: openai still crushing the competition. Competing models beat benchmarks, but are borderline worthless on real tasks.
- dmezzetti 3y agoGoing to call BS on this. I've delivered multiple projects this year using open models.
- apwell23 3y ago> delivered multiple projects Very curious about the type of projects you've delivered.
- dmezzetti 3y agoYou can look at https://github.com/neuml/txtai https://github.com/neuml/txtai. Biggest thing of 2023 was RAG with models like Mistral.
- apwell23 3y agoGot it. Thank you. I thought you had delivered consumer facing applications when i read that comment.
- dmezzetti 3y agoI've delivered consulting projects using RAG and Mistral 7B. Not saying it's easy to get the prompts right but I've been able to build systems that work well. Just trying to counter the myth that paid APIs are the only way to do this.
- benrow 3y agoAlso curious. I'm a sample of 1 and also relatively inexperienced. But I felt I quickly reached the limits of what was possible when I tried doing sentence classification with OSS sentence embedding models. The issue was with negation. I'd attributed too much magic to embedding models - they don't really understand language. Not to say there isn't very capable tech out there. Just to add a datapoint that "sentiment analysis"-like approaches in blogs don't always scale to your particular use-case. Edit: conscious I've drifted from the topic of chatbot type models, but felt relevant somehow.
- api 3y agoAI has been gradually improving for decades, but this is the year we finally noticed. The big thing was huge progress in natural language understanding. 2023 was the year the Turing test was smashed. Seeing computers win games, drive cars, optimize systems, even design things wasn’t as subjectively impressive to most of us as being able to talk to them. This was the year we first saw AI that could sort of communicate with us the way we can with each other.
- UrineSqueegee 3y agonot only talk to them, but ask them to perform tasks, sometimes obscure and they do it.
- Nullabillity 3y agoPeople were amazed by ELIZA back in '67. That doesn't mean that it did anything useful...
- steveoscaro 3y agoWhat’s your point? Because I can’t imagine it’s to suggest that GPT4 doesn’t do anything useful.
- bamboozled 3y agoTheir point is, we’ve been here before.
- stavros 3y agoIn the way a Lamborghini and an oxcart are both "here".
- steveoscaro 3y agoNot really
- Nullabillity 3y ago
- dumbaccount123 3y ago[dead]
- hiddencost 3y agoI dunno. The year of AI hype? LLMs are gonna change the world but the real incredible stuff is gonna take a few years before it really hits.
- chime 3y agoThings are already changing for early adopters. Yesterday, I used a PDF GPT to review an 80+ page contract and GPT4 to explain specific parts of it in depth using layman's terms. I could have hired a lawyer like I have done in the past but that itself is a slow, laborious process and I often feel like my questions don't get thoroughly answered. I feel very confident of my understanding of the contract now. I also used StableDiffusion to mock-up landscaping designs for our back patio and make ceramic art ideas for my wife last month. I was finally able to use super resolution to improve the quality of an old video CD of a VHS tape of a theater play my dad starred in with his friends in the 80s. I tried without super resolution many times in the past but it was never good enough. You are right that it will take a few years before it hits but AI in 2023 is not hype if you know which tool to use when.
- bamboozled 3y agoWhen we use it to solve real problems, make some real progress on climate change, then it will be something else from hype and it will be the year of AI. When you as an individual use it to sort PDFs or something it’s really hype.
- deleted 3y ago[deleted]
- squigz 3y agoAI isn't going to magically solve the issue of climate change. That's going to require global coordinated effort. In fact, I would say thinking that AI is going to solve 'real' issues like that, is more 'hype' than the above commenter.
- 3y ago