4 ms·

ChatGPT has one trade that is guaranteed to be bad. I'm not saying unprofitable, just bad. GBTC is the bitcoin ETF with biggest expense ratio - 1.5%. If you wan

ChatGPT has one trade that is guaranteed to be bad. I'm not saying unprofitable, just bad. GBTC is the bitcoin ETF with biggest expense ratio - 1.5%. If you want to bet on bitcoin, a better choice would be BITB (0.20%) or BTC (0.15%). Also, the reasoning is partially a hallucination - "The holding period of 9 months aligns with the expected completion of Grayscale's pivotal Phase 3 Bitcoin ETF trial, a major catalyst for unlocking investor demand and driving trust value realization." There is no such thing as a "holding period", nor are they doing a "Phase 3 Bitcoin ETF trial". It's possible the "Phase 3" thing is picked up from news about a drug company.

pavlov 2y ago
ChatGPT does a good job of imitating the average crypto influencer. They don’t know what they’re saying either, and 99% of crypto investors would be thrilled by the prospect of a “pivotal Phase 3 Bitcoin ETF trial” that will “drive trust value realization”. Sounds great, can’t miss out on that! The hallucinations are simply a mirror of a community that thrives on this nonsense. When nothing is real, you can’t blame the LLM for not figuring it out.
- attentionmech 2y ago
  This made me chuckle. You made a very interesting point that if LLMs are copying hallucinations those hallucinations are not infact hallucinations.
  - mvdtnz 2y ago
    If you believe (as many HNers do, although certainly not me) that LLMs have intelligence and awareness then you necessarily must also believe that the LLM is lying (call it hallucinating if you want).
    - attentionmech 2y ago
      Reminds me of recent paper where they found LLMs are scheming to meet certain goals; And that is a scientific paper done by a big lab. Are you referring from that context? Words and their historical contexts aside, systems which are based on optimization can take actions which can appear like intermediate lying to us. When deepmind used to play those atari games - the agents started cheating but that was just optimisation wasn't it? similarly when a language based agent does a optimisation, what we might perceive it as is scheming/lying. I will start believing that LLM is self aware when a research paper from a top lab like Deepmind/Anthropic put such a paper in a peer reviewed journal. Otherwise, it's just matrix multiplication to me so far.
      - Terr_ 2y ago
        > [paper claimed] LLMs are scheming IMO a much better framing is that the system was able to autocomplete stories/play-scripts. The document was already set up to contain a character that was a smart computer program with coincidentally the same name. Then humans trick themselves into thinking the puppet-play is a conversation with the author.
    - sdwr 2y ago
      Intelligence is a prerequisite for lying, but its foundation is morality and agency. To lie, you have to know that you are not telling the truth, and arguably have to be able to held accountable for that action. It's easy to babble a series of untruths, but lying requires intention, which requires an entity that can be recognized as having intentions. I'd argue that ChatGPT's lack of a cohesive self prevents it from lying, no matter how many untruths it creates.
      - jhghikvhu 2y ago
        If you ask chatgpt to tell a story of a liar it is able to do so. So while it doesn't have a motivated self to lie for it can imagine a motivated other to project the lie on.
        fatata123 2y ago
        [dead]
  - karmakaze 2y ago
    This is the interesting part of the experiment. Since these LLMs are general and not specifically trained on historical (and current) stock prices and (business) news stories, it isn't a measure of how good they could be today.
    - attentionmech 2y ago
      My first through after seeing this post was that it's a real world eval. We are running out of evals lately (arc-agi test, then sudden jump on frontier math, etc). So it's good to have such real world tests which show how far we are.
  - JohnMakin 2y ago
    This is a great insight and fascinating to me as well. What even is the solution though? It does seem like it follows logically though, since the earliest days of the internet huge swaths of wrong, fraudulent, or misleading info has plagued it and you’d usually have been wise to check your sources when trusting anything you read online. Then we had these models ingest the entire web, so we shouldn’t be surprised at how often it is confidently wrong.
    - attentionmech 2y ago
      I guess reasoning and healthy self-doubt to be built in system. Already the reasoning thing seems like 2025's candidate for what large labs will be zeroing down on.
  - Terr_ 2y ago
    Simpler than that: It's all hallucinations, some of them just happen to be ones humans approve-of. It's kind of like a manufacturer of Ouija boards promising that they'll fix the "channeling the wrong spirits from beyond the mortal plane" problem. It falsely suggests that "normal" output is fundamentally different.
- WalterBright 2y ago
  When I'd watch the financial news on TV, they would always bring on the "technical analyst", show a graph of the stock price, and then hand-draw some lines on it, and then spew out various technical terms for it guaranteed to impress. Me, I always regarded technical analysis as drawing pictures in clouds. If any of those analysts were worth spit, they'd be working for a hedge fund, not the network.
  - karmakaze 2y ago
    > drawing pictures in clouds. Well phrased and it's how the stock market works, not only by technical analysts but everyone else playing: make a story in your head, place your bets, majority rules. Some even believe that's how reality works in general. Sometimes belief or need could be a factor[0]. [0] https://www.guinnessworldrecords.com/news/2012/9/norwegian-family-set-lottery-win-world-record-45097 https://www.guinnessworldrecords.com/news/2012/9/norwegian-f...
    - WalterBright 2y ago
      On a more long term basis, the stock market reflects the business reality. But in the short term, it's chaos.
      - karmakaze 2y ago
        The former is a belief. It always reflects the imagined realities of those investing--we assume that business reality catches up with them, and it mostly does but not always within a predictable time frame.
        WalterBright 2y ago
        > The former is a belief It's based on the Law of Supply & Demand, which is always in play.
        gloosx 2y ago
miohtama 2y ago
Because of Bitcoin volatility, fees are very insignificant compared to daily price movement and irrelevant in day trading.
- pakitan 2y ago
  1% is 1%. Giving it away for no reason is plain stupid, even if the trade makes you 1000% return.
neltnerb 2y ago
God help the regulators that need to determine if it's insider trading for the people training the LLM to know it will be biased in ways they can profit from when used in inappropriate ways like this. I suspect the answer will be that users should have known better... I am sad that some people will certainly assume it's unbiased analysis. Hopefully the LLM trainers didn't "accidentally" bias the model in weird ways that favor their employer or themselves... two of the three recommendations are a fund for investing in bitcoin and a company using blockchain to trace chemical supply chains. I look forward to seeing if the AIs can beat an index fund, or if they'll just invest in a thousand blockchain, NFT, and AI companies. I suspect a LLM has a high opinion of a company making AI given how many press releases they're summarizing.
csomar 2y ago
This assumes that both GBTC and BITB have the same price movements, volatility and liquidity. This is far from true and as a result you might end up with a higher alpha in GBTC despite the fees. I am not saying it is guaranteed, but the fee is one variable.

pavlov 2y ago

ChatGPT does a good job of imitating the average crypto influencer. They don’t know what they’re saying either, and 99% of crypto investors would be thrilled by the prospect of a “pivotal Phase 3 Bitcoin ETF trial” that will “drive trust value realization”. Sounds great, can’t miss out on that! The hallucinations are simply a mirror of a community that thrives on this nonsense. When nothing is real, you can’t blame the LLM for not figuring it out.

attentionmech 2y ago
This made me chuckle. You made a very interesting point that if LLMs are copying hallucinations those hallucinations are not infact hallucinations.
- mvdtnz 2y ago
  If you believe (as many HNers do, although certainly not me) that LLMs have intelligence and awareness then you necessarily must also believe that the LLM is lying (call it hallucinating if you want).
  - attentionmech 2y ago
    Reminds me of recent paper where they found LLMs are scheming to meet certain goals; And that is a scientific paper done by a big lab. Are you referring from that context? Words and their historical contexts aside, systems which are based on optimization can take actions which can appear like intermediate lying to us. When deepmind used to play those atari games - the agents started cheating but that was just optimisation wasn't it? similarly when a language based agent does a optimisation, what we might perceive it as is scheming/lying. I will start believing that LLM is self aware when a research paper from a top lab like Deepmind/Anthropic put such a paper in a peer reviewed journal. Otherwise, it's just matrix multiplication to me so far.
    - Terr_ 2y ago
      > [paper claimed] LLMs are scheming IMO a much better framing is that the system was able to autocomplete stories/play-scripts. The document was already set up to contain a character that was a smart computer program with coincidentally the same name. Then humans trick themselves into thinking the puppet-play is a conversation with the author.
  - sdwr 2y ago
    Intelligence is a prerequisite for lying, but its foundation is morality and agency. To lie, you have to know that you are not telling the truth, and arguably have to be able to held accountable for that action. It's easy to babble a series of untruths, but lying requires intention, which requires an entity that can be recognized as having intentions. I'd argue that ChatGPT's lack of a cohesive self prevents it from lying, no matter how many untruths it creates.
    - jhghikvhu 2y ago
      If you ask chatgpt to tell a story of a liar it is able to do so. So while it doesn't have a motivated self to lie for it can imagine a motivated other to project the lie on.
      - fatata123 2y ago
        [dead]
- karmakaze 2y ago
  This is the interesting part of the experiment. Since these LLMs are general and not specifically trained on historical (and current) stock prices and (business) news stories, it isn't a measure of how good they could be today.
  - attentionmech 2y ago
    My first through after seeing this post was that it's a real world eval. We are running out of evals lately (arc-agi test, then sudden jump on frontier math, etc). So it's good to have such real world tests which show how far we are.
- JohnMakin 2y ago
  This is a great insight and fascinating to me as well. What even is the solution though? It does seem like it follows logically though, since the earliest days of the internet huge swaths of wrong, fraudulent, or misleading info has plagued it and you’d usually have been wise to check your sources when trusting anything you read online. Then we had these models ingest the entire web, so we shouldn’t be surprised at how often it is confidently wrong.
  - attentionmech 2y ago
    I guess reasoning and healthy self-doubt to be built in system. Already the reasoning thing seems like 2025's candidate for what large labs will be zeroing down on.
- Terr_ 2y ago
  Simpler than that: It's all hallucinations, some of them just happen to be ones humans approve-of. It's kind of like a manufacturer of Ouija boards promising that they'll fix the "channeling the wrong spirits from beyond the mortal plane" problem. It falsely suggests that "normal" output is fundamentally different.
WalterBright 2y ago
When I'd watch the financial news on TV, they would always bring on the "technical analyst", show a graph of the stock price, and then hand-draw some lines on it, and then spew out various technical terms for it guaranteed to impress. Me, I always regarded technical analysis as drawing pictures in clouds. If any of those analysts were worth spit, they'd be working for a hedge fund, not the network.
- karmakaze 2y ago
  > drawing pictures in clouds. Well phrased and it's how the stock market works, not only by technical analysts but everyone else playing: make a story in your head, place your bets, majority rules. Some even believe that's how reality works in general. Sometimes belief or need could be a factor[0]. [0] https://www.guinnessworldrecords.com/news/2012/9/norwegian-family-set-lottery-win-world-record-45097 https://www.guinnessworldrecords.com/news/2012/9/norwegian-f...
  - WalterBright 2y ago
    On a more long term basis, the stock market reflects the business reality. But in the short term, it's chaos.
    - karmakaze 2y ago
      The former is a belief. It always reflects the imagined realities of those investing--we assume that business reality catches up with them, and it mostly does but not always within a predictable time frame.
      - WalterBright 2y ago
        > The former is a belief It's based on the Law of Supply & Demand, which is always in play.
        gloosx 2y ago

miohtama 2y ago

Because of Bitcoin volatility, fees are very insignificant compared to daily price movement and irrelevant in day trading.

pakitan 2y ago
1% is 1%. Giving it away for no reason is plain stupid, even if the trade makes you 1000% return.

neltnerb 2y ago

God help the regulators that need to determine if it's insider trading for the people training the LLM to know it will be biased in ways they can profit from when used in inappropriate ways like this. I suspect the answer will be that users should have known better... I am sad that some people will certainly assume it's unbiased analysis. Hopefully the LLM trainers didn't "accidentally" bias the model in weird ways that favor their employer or themselves... two of the three recommendations are a fund for investing in bitcoin and a company using blockchain to trace chemical supply chains. I look forward to seeing if the AIs can beat an index fund, or if they'll just invest in a thousand blockchain, NFT, and AI companies. I suspect a LLM has a high opinion of a company making AI given how many press releases they're summarizing.

csomar 2y ago

This assumes that both GBTC and BITB have the same price movements, volatility and liquidity. This is far from true and as a result you might end up with a higher alpha in GBTC despite the fees. I am not saying it is guaranteed, but the fee is one variable.