5 ms·

> not yet fixed, still being exploited It is actually fixed. Tweets have been showing up with their "Indexed On" date instead of the actual authoring date. The

> not yet fixed, still being exploited It is actually fixed. Tweets have been showing up with their "Indexed On" date instead of the actual authoring date. The thing that isn't implemented is showing both dates for tweets with separate dates.

deleted 2y ago
[deleted]
ChrisArchitect 2y ago
'fixed' or being worked on..... Still not cool and just an example of the 'building in public' approach that rubs me the wrong way.
- mort96 2y ago
  To me it sounds like it's fixed?
  - Retr0id 2y ago
    It's more of a temporary workaround iiuc, the real fix needs to happen in the UI. Posts have two timestamps, the "createdAt" timestamp which can be anything the author says it is, and the "indexedAt" timestamp, which is not an intrinsic property of the post but whenever the AppView server first saw it (which is relatively more trustworthy, but could be different across multiple AppView instances). The proper fix, imho, is to have the UI display the "createdAt" timestamp but warn (perhaps with some kind of alert icon) when there's a significant mismatch between createdAt and indexedAt. (An even more proper fix might involve a quorum of "witness" servers but that might be an exercise in scope creep)
    - pfraze 2y ago
      Everybody's kind of got it. We didn't publish a lot the past couple days because we didn't want to highlight the situation during the election. Our fault for not solving it sooner. Here's the full deal. Every record self-declares a createdAt which can't be fully trusted. Every appview records indexedAt which is == to "first seen at." We can always expect indexedAt to lag behind a "correct" createdAt, and in ideal conditions that lag is quite small. The server was previously blending the internal indexedAt with the declared createdAt by returning the min(createdAt, indexedAt). Blending a trusted-but-lagging timestamp with the declared timestamp is a technique that works for sorting under specific conditions; if you're doing reverse-chron ordering you take the min of both, and if you're doing chron ordering you take the max. We will continue to use the blended min() for ordering within reverse chron feeds, but the problem is that we've historically been sending that down to the client for rendering. We recently changed it to give the actual `indexedAt`, which is part of the full solution. Unfortunately in the weeks preceding that change, a bunch of tweet importers had gotten popular without us realizing it, and our change caused the imports to all show the import time instead of the create time. So, we reverted that for now. The plan is what retroid describes -- we turn back on that change to the server behavior, and then if `createdAt < indexedAt - 24hr` or so, we indicate in the UI that it's likely an import from an archive. That's the full longterm solution. A quorum witness model is definitely overdoing it for now, but we have discussed creating exports of our index times so that new appviews can take advantage of them when doing backfill.
    - mort96 2y ago
      Yes it's a temporary workaround which fixes the problem. It deserves further works but thanks to the workaround, the problem isn't there anymore. Your point?
      - CuriousCosmic 2y ago
        FWIW Retr0id is one of the big community bluesky/AT protocol developers and importantly the person who more or less figured this out (or at the very least brought it to the community's attention). Point being that they were just clarifying the situation as they are probably one of the most informed people on the situation (with the other reply in this subthread being a first party bluesky dev and therefore one of the other SMEs on the topic).
        mort96 2y ago
        Thanks for that context, I just assumed he was another "ChrisArchitect" like commenter who want to equate "there is still more work to do" with "the problem hasn't been mitigated and the solution is stuck in development hell".
    - dbbk 2y ago
      > The proper fix, imho, is to have the UI display the "createdAt" timestamp but warn (perhaps with some kind of alert icon) when there's a significant mismatch between createdAt and indexedAt. If the goal is to build something that normal people will understand, this is a failure.
      - Retr0id 2y ago
        I'm not proposing the literal words "createdAt" or "indexedAt" be used in the UI.
        dbbk 2y ago
        No, I mean, having two separate times for a post appearing at all.
    - corobo 2y ago
      Isn't that what labellers do? Eg this shows if a post has differing create/index dates https://bsky.app/profile/backdate.mozzius.dev https://bsky.app/profile/backdate.mozzius.dev
- ChrisArchitect 2y ago
  Example post in question, shows nothing regarding the date. It basically shouldn't exist https://bsky.app/profile/retr0.id/post/3k5l4vko6jp2p https://bsky.app/profile/retr0.id/post/3k5l4vko6jp2p
evbogue 2y ago
How does this fix the above issue on a decentralized network? What happens when a new index is created on another server?

deleted 2y ago

[deleted]

ChrisArchitect 2y ago

'fixed' or being worked on..... Still not cool and just an example of the 'building in public' approach that rubs me the wrong way.

mort96 2y ago
To me it sounds like it's fixed?
- Retr0id 2y ago
  It's more of a temporary workaround iiuc, the real fix needs to happen in the UI. Posts have two timestamps, the "createdAt" timestamp which can be anything the author says it is, and the "indexedAt" timestamp, which is not an intrinsic property of the post but whenever the AppView server first saw it (which is relatively more trustworthy, but could be different across multiple AppView instances). The proper fix, imho, is to have the UI display the "createdAt" timestamp but warn (perhaps with some kind of alert icon) when there's a significant mismatch between createdAt and indexedAt. (An even more proper fix might involve a quorum of "witness" servers but that might be an exercise in scope creep)
  - pfraze 2y ago
    Everybody's kind of got it. We didn't publish a lot the past couple days because we didn't want to highlight the situation during the election. Our fault for not solving it sooner. Here's the full deal. Every record self-declares a createdAt which can't be fully trusted. Every appview records indexedAt which is == to "first seen at." We can always expect indexedAt to lag behind a "correct" createdAt, and in ideal conditions that lag is quite small. The server was previously blending the internal indexedAt with the declared createdAt by returning the min(createdAt, indexedAt). Blending a trusted-but-lagging timestamp with the declared timestamp is a technique that works for sorting under specific conditions; if you're doing reverse-chron ordering you take the min of both, and if you're doing chron ordering you take the max. We will continue to use the blended min() for ordering within reverse chron feeds, but the problem is that we've historically been sending that down to the client for rendering. We recently changed it to give the actual `indexedAt`, which is part of the full solution. Unfortunately in the weeks preceding that change, a bunch of tweet importers had gotten popular without us realizing it, and our change caused the imports to all show the import time instead of the create time. So, we reverted that for now. The plan is what retroid describes -- we turn back on that change to the server behavior, and then if `createdAt < indexedAt - 24hr` or so, we indicate in the UI that it's likely an import from an archive. That's the full longterm solution. A quorum witness model is definitely overdoing it for now, but we have discussed creating exports of our index times so that new appviews can take advantage of them when doing backfill.
  - mort96 2y ago
    Yes it's a temporary workaround which fixes the problem. It deserves further works but thanks to the workaround, the problem isn't there anymore. Your point?
    - CuriousCosmic 2y ago
      FWIW Retr0id is one of the big community bluesky/AT protocol developers and importantly the person who more or less figured this out (or at the very least brought it to the community's attention). Point being that they were just clarifying the situation as they are probably one of the most informed people on the situation (with the other reply in this subthread being a first party bluesky dev and therefore one of the other SMEs on the topic).
      - mort96 2y ago
        Thanks for that context, I just assumed he was another "ChrisArchitect" like commenter who want to equate "there is still more work to do" with "the problem hasn't been mitigated and the solution is stuck in development hell".
  - dbbk 2y ago
    > The proper fix, imho, is to have the UI display the "createdAt" timestamp but warn (perhaps with some kind of alert icon) when there's a significant mismatch between createdAt and indexedAt. If the goal is to build something that normal people will understand, this is a failure.
    - Retr0id 2y ago
      I'm not proposing the literal words "createdAt" or "indexedAt" be used in the UI.
      - dbbk 2y ago
        No, I mean, having two separate times for a post appearing at all.
  - corobo 2y ago
    Isn't that what labellers do? Eg this shows if a post has differing create/index dates https://bsky.app/profile/backdate.mozzius.dev https://bsky.app/profile/backdate.mozzius.dev
ChrisArchitect 2y ago
Example post in question, shows nothing regarding the date. It basically shouldn't exist https://bsky.app/profile/retr0.id/post/3k5l4vko6jp2p https://bsky.app/profile/retr0.id/post/3k5l4vko6jp2p

evbogue 2y ago

How does this fix the above issue on a decentralized network? What happens when a new index is created on another server?