5 ms·

It still seems to have the problems most other LLMs suffer with except Gemini: it loses context so quickly. I asked it about a paper I was looking at (SLOG [0]

by scuol 1y ago

It still seems to have the problems most other LLMs suffer with except Gemini: it loses context so quickly. I asked it about a paper I was looking at (SLOG [0]) and it basically lost the context of what "slog" referred to after 3 prompts. 1. I asked for an example transaction illustrating the key advantages of the SLOG approach. It responded with some general DB transaction stuff. 2. I then said "no use slog like we were talking about" and then it gave me a golang example using the log/slog package Even without the weird political things around Grok, it just isn't that good. [0] https://www.vldb.org/pvldb/vol12/p1747-ren.pdf https://www.vldb.org/pvldb/vol12/p1747-ren.pdf

Gigachad 1y ago
[flagged]
- HenryBemis 1y ago
  So.. If the 'source' of data is 9gag, 4chan, you will get 'this' material. If you feed it Tumlr, you will get Harry Potter and rope-porn-thingies. If you feed it Hitler's speeches, you will get 'that' material. If you feed it algebra, you will get 'that' material. Then.. Do we want 'open' or 'curated' LLMs? And how far from reality are the curated LLMs? And how far can curated LLMs take us (black Nazis? female US founding fathers?). Pick your poison I say.. and be careful what you wish for. There is no "perfect" LLM because there is no "perfect" dataset, and Sam-Altman-types-of-humans are definitely deeply flawed. But life is flawed, so our tools are/will be flawed.
  - Gigachad 1y ago
    The problem was not the source of the training data. xAI confirmed that the system prompt had been modified to make grok talk about South African white genocide. While they didn’t say who modified it. It’s hard to believe it wasn’t Elon.
    - jeffhuys 1y ago
      Eye roll
    - drkleiner 1y ago
      It's hard to believe it is Elon either, I don't think he would know how unless they made a special interface for him
      - WhyIsItAlwaysHN 1y ago
        He is the CEO of a company, he can personally ask someone to do it for him.
        drkleiner 1y ago
        But implementation was too messy for someone with expertise to do it on CEO's request
    - AnthonyMouse 1y ago
      > While they didn’t say who modified it. It’s hard to believe it wasn’t Elon. Is it really that hard to understand how these things happen? The boss says "remove bias" but the peons don't really know how to do that and the naive approach to unbiasing a thing is to introduce bias in the other direction. And then if you're Google and the boss thinks it has a right-wing bias you crook it and get black Nazis and if you're xAI and the boss thinks it has a left-wing bias you get white genocide. In both cases the actual problem is when people think bias operates like an arithmetic sum, because it doesn't.
      - archagon 1y ago
        Except someone clearly wanted Grok to talk about some very specific South African phrases and events, not just “remove bias” in the general sense.
        AnthonyMouse 1y ago
- srmarm 1y ago
  Be careful saying things like that or you'll get [flagged] - discussion of what seemed an incredibly important subject is forbidden on here it seems.
  - rsynnott 1y ago
    [flagged]
    - tormeh 1y ago
      Effective Altruism is still great, and never stopped being great. Guilt does not transfer by association in this way.
      - mschuster91 1y ago
        It's not. Feeding kids, researching vaccines and a bunch of other things that billionaires are funding should not depend on the graces and whims of billionaires, it should be something provided for by the government.
      - rsynnott 1y ago
        There's definitely _something_ there, but, as with all philosophies, the internet has taken it and run with it to a fairly absurd degree, to the point where, for many adherents, it's basically a religion.
    - mschuster91 1y ago
      HN crowd is ... mixed, it's perhaps the one last true melting pot we have on the Internet. A curse and a blessing, if you ask me. You got truly anything here. Europeans that in general tend to lean more towards "democratic socialism" and its various offshoots, American libertarians (which have a large intersection with Musk fanboys), a bunch of extremely rich startup founders, American progressives, conservatives of all kinds, Zionists and Hamas apologists, probably Russian and Chinese psy-ops, accelerationists, preppers... name any ideology and you'll find supporters on HN. What has changed a bit is that tribalism seems to have taken over from civilized or at least arguments and fact oriented discourse. Personally, I'd prefer if downvotes and especially flags would require one to give a reason so that repeat offenders that just flag and downvote everything they disagree with can get suspended for ruining discussion.
- bilbo0s 1y ago
  You never know when it will start spouting it either. That kind of uncertainty in the responses landing in your interface is just not sustainable. Your money is coming from the quality of the content your system is putting out. If it's being used for dentistry, and it randomly spits out white supremacist content, dentists will look for a system that won't do that. Because they asked about, say, intaglio surfaces for a wearable dental appliance. Not a treatise on white genocide. At this point, to use Grok, you'd be intentionally setting your startup to detonate itself at some random point in the future. That's just not how you make money.
convivialdingo 1y ago
When I use the "think" mode it retains context for longer. I tested with 5k lines of c compiler code and I could 6 prompts in before it started forgetting or generalizing I'll say that grok is really excellent at helping my understand the codebase, but some miss-named functions or variables will trip it up..
- pomtato 1y ago
  not from a tech field at all but would it do the context window any good to use "think" mode but discard them once the llm gives the final answer/reply? is that even possible to disregard genrated token's selectively?
dahcryn 1y ago
it also doesn't help that many of these companies tend to either limit the context of the chat to the 10 most recent messages (5 back and forths), or rewrite the history summarized in a few sentences. Both ways lose a ton of information, but you can avoid that behaviour by going through the APIs. Especially Azure OpenAI et... on the web is useless, but it's quite capable through custom APs I think Gemini is just the only one that by default keeps the entire history verbatim.
voidspark 1y ago
The paid version "SuperGrok" has a larger context window, but nothing beats Gemini for that. I tried your question with SuperGrok. Here's the result. https://grok.com/share/bGVnYWN5_d298dd12-9942-411c-900c-29947cf41cc3 https://grok.com/share/bGVnYWN5_d298dd12-9942-411c-900c-2994... I use Grok for similar tasks and usually prefer Grok's explanations. Easier to understand. For some problems where I've asked Grok to use formal logical reasoning I have seen Grok outperform both Gemini 2.5 Pro and ChatGPT-o3. It is well trained on logic. I've seen Grok generate more detailed and accurate descriptions of images that I uploaded. Grok is natively multimodal. There is no single LLM that outperforms all of the others at all tasks. I've seen all of the frontier models strongly outperform each other at specific tasks. If I was forced to use only one, that would be Gemini 2.5 Pro (for now) because it can process a million tokens and generate much longer output than the others.
aibrother 1y ago
for me xAI has its place mainly for 1) exclusive access to tweets and 2) being uncensored. and it's decent enough (even if it's not the best) in terms other capabilities
- touristtam 1y ago
  > being uncensored With the recent article on how it was easily manipulated, I wouldn't be so confident it is uncensored, just that its bias is leaning into its owner's beliefs; which isn't great. Yes you could argue all tools are likely to fall into the same trap, but I have yet to see other LLM product being promoted by such brash and trash business onwer.

by scuol 1y ago

Gigachad 1y ago

[flagged]

HenryBemis 1y ago
So.. If the 'source' of data is 9gag, 4chan, you will get 'this' material. If you feed it Tumlr, you will get Harry Potter and rope-porn-thingies. If you feed it Hitler's speeches, you will get 'that' material. If you feed it algebra, you will get 'that' material. Then.. Do we want 'open' or 'curated' LLMs? And how far from reality are the curated LLMs? And how far can curated LLMs take us (black Nazis? female US founding fathers?). Pick your poison I say.. and be careful what you wish for. There is no "perfect" LLM because there is no "perfect" dataset, and Sam-Altman-types-of-humans are definitely deeply flawed. But life is flawed, so our tools are/will be flawed.
- Gigachad 1y ago
  The problem was not the source of the training data. xAI confirmed that the system prompt had been modified to make grok talk about South African white genocide. While they didn’t say who modified it. It’s hard to believe it wasn’t Elon.
  - jeffhuys 1y ago
    Eye roll
  - drkleiner 1y ago
    It's hard to believe it is Elon either, I don't think he would know how unless they made a special interface for him
    - WhyIsItAlwaysHN 1y ago
      He is the CEO of a company, he can personally ask someone to do it for him.
      - drkleiner 1y ago
        But implementation was too messy for someone with expertise to do it on CEO's request
  - AnthonyMouse 1y ago
    > While they didn’t say who modified it. It’s hard to believe it wasn’t Elon. Is it really that hard to understand how these things happen? The boss says "remove bias" but the peons don't really know how to do that and the naive approach to unbiasing a thing is to introduce bias in the other direction. And then if you're Google and the boss thinks it has a right-wing bias you crook it and get black Nazis and if you're xAI and the boss thinks it has a left-wing bias you get white genocide. In both cases the actual problem is when people think bias operates like an arithmetic sum, because it doesn't.
    - archagon 1y ago
      Except someone clearly wanted Grok to talk about some very specific South African phrases and events, not just “remove bias” in the general sense.
      - AnthonyMouse 1y ago
srmarm 1y ago
Be careful saying things like that or you'll get [flagged] - discussion of what seemed an incredibly important subject is forbidden on here it seems.
- rsynnott 1y ago
  [flagged]
  - tormeh 1y ago
    Effective Altruism is still great, and never stopped being great. Guilt does not transfer by association in this way.
    - mschuster91 1y ago
      It's not. Feeding kids, researching vaccines and a bunch of other things that billionaires are funding should not depend on the graces and whims of billionaires, it should be something provided for by the government.
    - rsynnott 1y ago
      There's definitely _something_ there, but, as with all philosophies, the internet has taken it and run with it to a fairly absurd degree, to the point where, for many adherents, it's basically a religion.
  - mschuster91 1y ago
    HN crowd is ... mixed, it's perhaps the one last true melting pot we have on the Internet. A curse and a blessing, if you ask me. You got truly anything here. Europeans that in general tend to lean more towards "democratic socialism" and its various offshoots, American libertarians (which have a large intersection with Musk fanboys), a bunch of extremely rich startup founders, American progressives, conservatives of all kinds, Zionists and Hamas apologists, probably Russian and Chinese psy-ops, accelerationists, preppers... name any ideology and you'll find supporters on HN. What has changed a bit is that tribalism seems to have taken over from civilized or at least arguments and fact oriented discourse. Personally, I'd prefer if downvotes and especially flags would require one to give a reason so that repeat offenders that just flag and downvote everything they disagree with can get suspended for ruining discussion.
bilbo0s 1y ago
You never know when it will start spouting it either. That kind of uncertainty in the responses landing in your interface is just not sustainable. Your money is coming from the quality of the content your system is putting out. If it's being used for dentistry, and it randomly spits out white supremacist content, dentists will look for a system that won't do that. Because they asked about, say, intaglio surfaces for a wearable dental appliance. Not a treatise on white genocide. At this point, to use Grok, you'd be intentionally setting your startup to detonate itself at some random point in the future. That's just not how you make money.

convivialdingo 1y ago

When I use the "think" mode it retains context for longer. I tested with 5k lines of c compiler code and I could 6 prompts in before it started forgetting or generalizing I'll say that grok is really excellent at helping my understand the codebase, but some miss-named functions or variables will trip it up..

pomtato 1y ago
not from a tech field at all but would it do the context window any good to use "think" mode but discard them once the llm gives the final answer/reply? is that even possible to disregard genrated token's selectively?

dahcryn 1y ago

it also doesn't help that many of these companies tend to either limit the context of the chat to the 10 most recent messages (5 back and forths), or rewrite the history summarized in a few sentences. Both ways lose a ton of information, but you can avoid that behaviour by going through the APIs. Especially Azure OpenAI et... on the web is useless, but it's quite capable through custom APs I think Gemini is just the only one that by default keeps the entire history verbatim.

voidspark 1y ago

The paid version "SuperGrok" has a larger context window, but nothing beats Gemini for that. I tried your question with SuperGrok. Here's the result. https://grok.com/share/bGVnYWN5_d298dd12-9942-411c-900c-29947cf41cc3 https://grok.com/share/bGVnYWN5_d298dd12-9942-411c-900c-2994... I use Grok for similar tasks and usually prefer Grok's explanations. Easier to understand. For some problems where I've asked Grok to use formal logical reasoning I have seen Grok outperform both Gemini 2.5 Pro and ChatGPT-o3. It is well trained on logic. I've seen Grok generate more detailed and accurate descriptions of images that I uploaded. Grok is natively multimodal. There is no single LLM that outperforms all of the others at all tasks. I've seen all of the frontier models strongly outperform each other at specific tasks. If I was forced to use only one, that would be Gemini 2.5 Pro (for now) because it can process a million tokens and generate much longer output than the others.

aibrother 1y ago

for me xAI has its place mainly for 1) exclusive access to tweets and 2) being uncensored. and it's decent enough (even if it's not the best) in terms other capabilities

touristtam 1y ago
> being uncensored With the recent article on how it was easily manipulated, I wouldn't be so confident it is uncensored, just that its bias is leaning into its owner's beliefs; which isn't great. Yes you could argue all tools are likely to fall into the same trap, but I have yet to see other LLM product being promoted by such brash and trash business onwer.