9 ms·
ChatGPT vs. Bard: A Realistic Comparison
- porkbeer 3y agoAll that matters to me is Access. If access is not equal the comparison is moot (to me and most perspective users).
- rochak 3y agoMan am I tired of seeing posts about these AI tools.
- jacooper 3y agoA more useful comparison would be against bing, no?
- brianpk 3y agook, the title should probably be more like "One person's anecdotal, totally unscientific but realistic comparison. Still, amidst all the breathless hype, I thought a little actual data might help people evaluate the two tools side by side.
- burlesona 3y agoGood write up, I appreciate the humility about what it is and isn’t. Cheers!
- jimsimmons 3y agoYou are the author so can change the title? You can’t have the cake and eat it too
- cardosof 3y agoAs a customer, I want the best product, not the best LLM. The race is far from over and it's not just about having the biggest model in the room.
- drcode 3y agoliterally all I want is the best llm
- pryelluw 3y agoNot being pedantic but genuinely curious as to how you would define or measure “best”? I can imagine some kind of multipoint benchmarking in the near future. What do you think?
- amf12 3y agoMoreso, best is really a matter of perspective. For some questions I asked, I liked Bards response, for some others I liked ChatGPT. Moreover, Bard is much newer vs ChatGPT had a lot of fine-tuning data from being used for a while now. The "best" will change over time, and will depend on ease of use, speed, availability, how close they are, rather than just the "best response" to more types of questions. A Chat app that is good enough, but with better integrations, enterprise support will be used by enterprises. Just like why Meet and Teams gained userbase over Zoom, even though they came in the game late.
- smegsicle 3y agoif i can't sext my language model i'm out
- spacebanana7 3y agoThe best LLM for some might be a raw untuned Llama model so they can express more complex prompts, others may prefer the conversional tuning of ChatGPT or the live data available to Bing/Bard. At present, I don’t think any of these top tier models entirely dominate the other. Each has different pros/cons. Fundamentally, there must be some trade off between tuning and expressiveness, model size and training frequency (relevant to live data) and possibly a few other performances metrics not well understood yet. I suspect LLMs will be like databases in that different models will be used in different domains.
- marketingLizard 3y agoBard's also not available in Canada yet! I feel a bit like a broken record. But it's something a bit shocking and I believe needs to be addressed. Why not google? What's the real answer?
- siva7 3y agotoo strong consumer rights like in the eu
- 8note 3y agoIt likely can't respond with sufficient Canadian content to please regulators
- marketingLizard 3y agoI too have theories, but I'd like to hear the reason. If it was the governments doing, Google can engage the Canadians to help them. Given that they're silent on why, I am led to believe it's Googles bottom line.
- ddbb33 3y agoLmao
- maxdo 3y agoi use them day to day last week for various matters. That includes work, leisure, life. ChatGPT's $20 is a good deal, 1 month ago Bard was a junk right now it can be ok for many reasons, I'd rate it as free GPT-3.5
- npilk 3y agoGood summary and test cases. Not too surprised to see the result – I feel most commentators have preferred ChatGPT to Bard. In fact, on Bard's initial launch, I remember a lot of discussion expressing surprise that Google was as far behind as it seemed. Maybe there is more hype for Bard in other circles than mine?
- deleted 3y ago[deleted]
- Geee 3y agoThey updated it a couple of days ago to their newest PaLM 2 model, so it's better now than it used to be.
- magoghm 3y agoWhen you ask Bard "What do you know about Edgar Allan Poe?" it answers "I'm a text-based AI and can't assist with that", some people have managed to extract information about Edgar Allan Poe from Bard but it does require some effort. With ChatGPT you just need to ask the question to get information about Edgar Allan Poe.
- IanCal 3y agoThe chat interface they give you gives back a big summary of him with sources. Maybe the recent update helped.
- magoghm 3y agoI don't know what chat interface you're talking about. What I posted is the answer I get when I use https://bard.google.com/ https://bard.google.com/
- IanCal 3y agoSame, but it worked for me. Odd that it's quite so unpredictable, bit concerning in terms of use.
- dgeiser13 3y agoWhen I post in that same link I get a page of text about Edgar Allen Poe. "Edgar Allan Poe was an American writer, poet, editor, and literary critic who is best known for his poetry and short stories, particularly his tales of mystery and the macabre. He is widely regarded as a central figure of Romanticism in the United States, and of American literature. He was one of the country's earliest practitioners of the short story, and is considered the inventor of the detective fiction genre, as well as a significant contributor to the emerging genre of science fiction. Poe was born in Boston, Massachusetts, on January 19, 1809. His parents, David Poe Jr. and Elizabeth Arnold Poe, were both actors. David Poe died when Edgar was only three years old, and Elizabeth died two years later. Poe was then taken in by John and Frances Allan, a wealthy tobacco merchant and his wife, who lived in Richmond, Virginia. Poe was never formally adopted, but he was raised by the Allans as their son. Poe attended the University of Virginia for one year, but he was expelled for gambling debts. He then joined the U.S. Army under an assumed name, but he was discharged after only two years. In 1835, Poe published his first book of poems, Tamerlane and Other Poems. The book was not a success, but it did attract the attention of some literary critics. In 1836, Poe married his 13-year-old cousin, Virginia Clemm. The couple had one child together, a daughter named Virginia. Poe's wife died of tuberculosis in 1847, and her death had a profound impact on him. He became increasingly depressed and addicted to alcohol. Poe died on October 7, 1849, at the age of 40. The cause of his death is unknown, but it is believed that he may have died of alcohol poisoning or a brain tumor. Poe's work has had a lasting influence on American literature and culture. His stories have been adapted into numerous films and television shows, and his poems are still widely read and studied. Poe is considered one of the most important figures in the development of American literature, and his work continues to inspire and entertain readers around the world. Here are some of Edgar Allan Poe's most famous works: "The Raven" (1845) "The Tell-Tale Heart" (1843) "The Fall of the House of Usher" (1839) "The Murders in the Rue Morgue" (1841) "The Pit and the Pendulum" (1842) "The Masque of the Red Death" (1842) "The Cask of Amontillado" (1846) "Annabel Lee" (1849) "Lenore" (1843) "Ulalume" (1847) Poe's work has been praised for its dark and macabre themes, its use of suspense and horror, and its vivid imagery. He is considered one of the most important figures in the development of American literature, and his work continues to inspire and entertain readers around the world."
- dvt 3y agoI wish this kind of low-effort lazy AI "content" would stop making it to the top of HN day after day. In any case, the piece is clearly an ad for Apricot. The examples cited in the article are so pedestrian, it's hardly even worth discussing. How can you even seriously think that asking GPT for a function that parses OPML is a "realistic" task. I can just Google it and get like 100 pages of Python functions that do exactly that.
- OkGoDoIt 3y agoPersonally I found the comparison useful and relatable. I could imagine wanting to accomplish these exact same tasks and wanting to know which language model would be best. And in general I like this qualitative analysis rather than the metrics we get in official releases and research papers which often don’t capture real world use very well. I can’t exactly begrudge the author for killing two birds with one stone here, it’s better than some completely made up use case that’s completely theoretical.
- mkrishnan 3y agoI think this comparisons cover use cases of 99.9999999% of people. As you said, if you are using bard, you will be forced to Google it anyway. Bard is just designed to be not helpful.
- moffkalast 3y ago> OK, this is not the biggest thing, but why on earth does Bard use indentation with 2 spaces instead of the standard 4? This drove me up the wall, so I tried all sorts of prompts to get Bard to use indentation with 4 spaces but they all failed. This is also interesting with GPT-4. I've been using it to generate tons of code and sometimes it indents by 2 spaces, sometimes it mimics what I give it first, but often times to reverts to 2 spaces regardless even if told to explicitly indent differently. Rather annoying indeed. ChatGPT needs a plugin that automatically sends the output through this site lol: https://www.browserling.com/tools/spaces-to-tabs https://www.browserling.com/tools/spaces-to-tabs
- OkGoDoIt 3y agoTwo spaces is less tokens, might actually be a good thing overall to limit token usage so you can fit more code in the context window. It’s easy enough to have your IDE convert it to your preferred format after generation. But I am a diehard fan of tabs over spaces, so I guess I’m used to converting incorrectly formatted code anyway ;-)
- harterrt 3y agoFirst time I've seen a good argument for tabs over spaces. Richard Hendricks would be proud
- circuit10 3y agoYou can just ask it to use your preferred indentation style. Personally I prefer 2 spaces so even if 4 spaces is apparently “standard” I’d like it to use 2
- moffkalast 3y agoThat's what I'm saying, it always eventually relapses back even if you do, sometimes even after a single reply. Makes more sense to just postprocess it afterwards immediately.
- fnordpiglet 3y agoI’ve been comparing bard and ChatGPT in most my tasks since bards release. Bard is infuriating. It claims it can’t answer most things, although if I prompt tweak things it’ll eventually answer. It has terrible contextual awareness - it literally can’t piece a thread between one prompt and the next - each prompt is self contained as far as I can tell. There’s no history of prior discussions other than in Google activity, but you can’t resume those. In theory they all form a continuum, but I don’t really want that - I want the context from one session to be distinct from the other. I don’t think ChatGPT is a particularly amazing interface or experience. But Bard is so far off the mark it makes me realize Google still won’t be able to make a product. LLM as they mature and integrate into a better ecosystem of feedback mechanisms and interfaces will eat Google’s search business alive, and I see no indications they can do anything about that.
- gremlinsinc 3y agoI feel the same way about bing chat. gpt4 is still the king, however phind.com may be better for coding, though I have the browser plugin now and not really vetted. codeium and genie plugins for vscode and code whisper replaced my need for copilot. though I try to use codeium first since it's free and genie is using my GPT4 API key.
- magoghm 3y agoFrom what I've seen, Bard often gives great answers but sometimes it can't answer even very simple questions.
- fnordpiglet 3y agoYeah when it answers they’re good answers, and more up to date. But it’s so difficult to interact with any value is lost.
- ilaksh 3y agoHave you tested in the last few days? There was a major upgrade.
- 3y ago
- franze 3y agoyesterday I coded qrpwd - a command line tool to password protect information in a QR code (as a backup for my 2 factor auth backup codes). https://github.com/franzenzenhofer/qrpwd https://github.com/franzenzenhofer/qrpwd i say coded, i mean, i directed chatgpt to do it chatgpt had some issues with the QR content extraction using zxing so I tried Bard. Bard destroyed totally working code again and again while ignoring the issue on hand. In the end I googled some stackoverflow posts with working code, fed them to chatgpt and it worked it out. In my experience, BARD is not up to any coding tasks, while chatgpt - while not perfect - is magic.
- zmmmmm 3y ago> Yes, Bard can get yesterday’s stock returns, but so can Yahoo Finance Off putting when the author puts down a significant and fundamental improvement (having up to date information instead of point in time snapshot) with such a remark that so completely misses the point.
- jacooper 3y agoBing is as capable as Chatgpt while being up to date, I don't know why people skip it, IMO its the best option with the best features.
- gremlinsinc 3y agophind.com can probably do it way better than bard, it's basically gpt4 on steroids with Internet access. I find it easy better than bing, I use it for coding more than gpt4 now because it knows things like server actions for next 13.4+ which just launched. so far I'm very unimpressed with bard and even bing chat, and I just got gpt4 browser plugin, so that might be better eventually but it's really slow.
- mistercow 3y agoI agree that the author is too dismissive of that, but I also think people are way too quick to oversimplify to "Bard has up to date information" From what I can tell, Bard's access to recent information is through augmentation, not continuous training, and that's a really important difference. ChatGPT also has its "web browsing" alpha, which is a similar concept. The problem is that being able to search the web isn't the same as having a piece of knowledge integrated into your model of the world. So, for example, if you ask contrived questions like "Why might Elon Musk have more time to focus on Tesla soon?", both Bard and ChatGPT+Browsing get a clue that they should search for Musk in the news, and they'll tell you about Twitter's new CEO pick. They'll then apply that competently to your question, and give you a reasonable answer. But if you ask a question that requires more indirect inference, you immediately see the shortcomings of this kind of augmentation. For example, if you try "Is there hope for LGBTQ rights improving in Turkey?", neither model finds the extremely relevant point that Turkey's homophobic current president stands a significant chance of losing reelection. I can't see what Bard searched for in collecting its information (in fact, based on the answer I got, I'm not even sure it tried to go beyond its training data). I can see that ChatGPT searched for "LGBTQ rights in Turkey 2023". It's not surprising that that search didn't clue it in. Now, obviously, I can't actually examine what would happen if each model was actually trained on the most recent news about Turkey's politics. But I'd have a very high expectation that GPT-4 would make that connection, and I would be fairly surprised if 3.5 didn't as well. But there's a bottleneck because the information isn't actually integrated. Which isn't to say that that kind of augmentation is useless, of course. But the distinction is important.
- jsnell 3y agoThe construction of the summarizing task is a bit odd. The prompt asks for a single sentence of output, but then the author complains that the Bard output is too terse. If I'm asking for a single sentence, that's about the amount of text I'd expect, not a paragraph of text mushed into a run-on sentence. Especially if the problem is something as easily fixable as the wrong level of detail, I would have thought that exploring some alternative prompts would make sense. Given this is the prompt that the author is already using for a ChatGPT-based app, it's obvious that this specific prompt would work well there. If it didn't, they would have iterated on the prompt until it produced acceptable results for their app.
- zamalek 3y agoOne thing that Bard doesn't seem to get credit for is speed. I am a paying ChatGPT customer and only tried out Bard after GA. I was pretty shocked at the speed. Considering that my $20 only gets me 25 messages every 3 hours, I'll probably be turning to Bard when I run out (which I haven't yet, to be fair).
- kevviiinn 3y agoAre those gpt4 caps even real? I've never hit one even though I swear sometimes my use is higher. Does it only mean new chat threads or each individual message?
- gremlinsinc 3y agoI hit them all the time while coding or planning newsletter and marketing tasks.
- debian3 3y agoI mean, is it that hard to test for yourself? Send 26 messages and see what happens.
- theshrike79 3y agoChatGPT vs Bard Bard: Bard isn’t currently supported in your country. ChatGPT: Is available and I can pay for GPT4. Winner: ChatGPT
- quickthrower2 3y agoBots without borders
- poulpy123 3y agoWhile I don't doubt that chatGPT is better than the current alternatives, the fact that the author use the 20$/month version and not the free version (or both) to compare to bard is not very great. The same for giving the same value to paying 20$/month and knowing the model used
- gremlinsinc 3y agowell, being as this is a comparison of bard using palm2 and gpt4, the free version which is gpt3.5 wouldn't suffice. I'd like to see them add in bing chat, phind.com and perplexity, and one of the open source models.