4 ms·

It's fascinating to see new UI primitives and concepts get invented in the LLM era. The sea of creativity makes it hard to even understand most of what each new

It's fascinating to see new UI primitives and concepts get invented in the LLM era. The sea of creativity makes it hard to even understand most of what each new app does, and nobody describes them well. When I went to the Hermes agent web page, I was left with zero clue about what it did or what it could do. It took a bit of digging to find the right part of the qm page that helped me grok what was going on. I've become attached to Orca (yc-backed) for managing coding sessions in the past week, but some sort of postgres session db is what's really lacking. So, maybe it's time to try qm.

j45 3mo ago
AI needs entirely new primitives in many areas.
- jesol 3mo ago
  I think we need a new area of study around UI/Agent connection. It can kinda be done with tools, but I'd we need much deeper primitives to allow the UI to inform the Agent and vice-versa. Right now we've just given up, replacing the UI with an Agent message view, but I think that's just because nobody is thinking about how the two can compliment each other I've been playing around with using a hidden markov model informed by a UI state event stream, with the end state fed into the Agent as a hint on each message turn. Then the Agent can make a tool call to add events to the HMM. This has been really interesting, but I haven't struck the right balance to make it actually feel good for the user yet
  - throwaway7783 3mo ago
    Isn't this the direction Claude on the web is going towards? It's a mix of message view on the left and specific interactive UI on the right panel. There is also MCP Apps spec now
- jychang 3mo ago
  It's hard to say what is needed now, vs needed in a few years. For example, many rumors say SSI has solved learning and retaining state. That would significantly change AI requirements.
  - deleted 3mo ago
    [deleted]
2001zhaozhao 3mo ago
It's really hard to focus on the unique aspect of your tool because it runs the risk that people will think it's unfamiliar and therefore not useful to them. That's why a lot of the tool marketing pages look the same even though the tools themselves might be different or innovative
- epistasis 2mo ago
  Isn't it a bigger risk for people to see the page, understand nothing, and think it's not useful to them? Having no information, after investing time to investigate, leaves me with a very sour taste and makes me unlikely to give it a fair shake a second time.
papascrubs 3mo ago
It is fascinating and I love to see it. Ultimately though, why not build your own? I think that in part is what we're beginning to see-- highly customized and personalized software. I take most of these as inspiration these days and just build my own. Nothing you can't hammer out with a few good Claude sessions.
- jauntywundrkind 3mo ago
  These new experiences are built atop layers and layers of bedrock libraries that we, generally, don't reinvent. Or we reinvent one or two. What's notable to me is that ux doesn't so far generally have this behavior. That as per this post people just build a new app, a new experience. I want to believe over time we'll have better composable & malleable ux experiences atop broader platforms for us. That over time the "go it alone" path has other worth ways to innovate that use a more substantial shared base. It's dangerous to go it alone, and doing so equipped with just our wooden sword and some courage and perhaps an LLM wisp is an amazing adventure, but I think the survival rate & impact would be much better if we had more general ux systems that supported better innovation atop them, and if less people did the pure "why not build your own" path.
  - duderific 3mo ago
    People are using the UX systems though, just that it's the agent that is gluing them together now. Under the hood, it's still React plus component libraries like shadcn, lucide, tailwind, axios and so on. The agents are so good at this, there's literally no point in not doing it that way.
    - jauntywundrkind 3mo ago
      These are all surface veneer but so so so much less than an app. The entire rest of the owl has to be redrawn, and all the guts inside the owl redesigned.
  - papascrubs 3mo ago
    I understand you take. And I definitely agree that we could use a lot better ux tools than we have right now. Maybe one of these prototypes will lead to that though. I'll look at it more as creative engineering. Not everything is meant to be distributed or consumed. But that doesn't mean that it's not worthwhile to try and prototype different flows and options. I have seen hundreds of harnesses, I've only used a couple-- but they've inspired me to build something that fits me better and works for me. If somebody comes out with something that helps unify the UX experience and provides better building blocks for me to work with, I'll definitely switch to that and rebuild my experience on top of that. But for now this is what we have. Admittedly this reflects my level of experience and my expertise too. I don't have a lot of experience working with building UX frameworks, so I adapt what I can.

j45 3mo ago

AI needs entirely new primitives in many areas.

jesol 3mo ago
I think we need a new area of study around UI/Agent connection. It can kinda be done with tools, but I'd we need much deeper primitives to allow the UI to inform the Agent and vice-versa. Right now we've just given up, replacing the UI with an Agent message view, but I think that's just because nobody is thinking about how the two can compliment each other I've been playing around with using a hidden markov model informed by a UI state event stream, with the end state fed into the Agent as a hint on each message turn. Then the Agent can make a tool call to add events to the HMM. This has been really interesting, but I haven't struck the right balance to make it actually feel good for the user yet
- throwaway7783 3mo ago
  Isn't this the direction Claude on the web is going towards? It's a mix of message view on the left and specific interactive UI on the right panel. There is also MCP Apps spec now
jychang 3mo ago
It's hard to say what is needed now, vs needed in a few years. For example, many rumors say SSI has solved learning and retaining state. That would significantly change AI requirements.
- deleted 3mo ago
  [deleted]

2001zhaozhao 3mo ago

It's really hard to focus on the unique aspect of your tool because it runs the risk that people will think it's unfamiliar and therefore not useful to them. That's why a lot of the tool marketing pages look the same even though the tools themselves might be different or innovative

epistasis 2mo ago
Isn't it a bigger risk for people to see the page, understand nothing, and think it's not useful to them? Having no information, after investing time to investigate, leaves me with a very sour taste and makes me unlikely to give it a fair shake a second time.

papascrubs 3mo ago

It is fascinating and I love to see it. Ultimately though, why not build your own? I think that in part is what we're beginning to see-- highly customized and personalized software. I take most of these as inspiration these days and just build my own. Nothing you can't hammer out with a few good Claude sessions.

jauntywundrkind 3mo ago
These new experiences are built atop layers and layers of bedrock libraries that we, generally, don't reinvent. Or we reinvent one or two. What's notable to me is that ux doesn't so far generally have this behavior. That as per this post people just build a new app, a new experience. I want to believe over time we'll have better composable & malleable ux experiences atop broader platforms for us. That over time the "go it alone" path has other worth ways to innovate that use a more substantial shared base. It's dangerous to go it alone, and doing so equipped with just our wooden sword and some courage and perhaps an LLM wisp is an amazing adventure, but I think the survival rate & impact would be much better if we had more general ux systems that supported better innovation atop them, and if less people did the pure "why not build your own" path.
- duderific 3mo ago
  People are using the UX systems though, just that it's the agent that is gluing them together now. Under the hood, it's still React plus component libraries like shadcn, lucide, tailwind, axios and so on. The agents are so good at this, there's literally no point in not doing it that way.
  - jauntywundrkind 3mo ago
    These are all surface veneer but so so so much less than an app. The entire rest of the owl has to be redrawn, and all the guts inside the owl redesigned.
- papascrubs 3mo ago
  I understand you take. And I definitely agree that we could use a lot better ux tools than we have right now. Maybe one of these prototypes will lead to that though. I'll look at it more as creative engineering. Not everything is meant to be distributed or consumed. But that doesn't mean that it's not worthwhile to try and prototype different flows and options. I have seen hundreds of harnesses, I've only used a couple-- but they've inspired me to build something that fits me better and works for me. If somebody comes out with something that helps unify the UX experience and provides better building blocks for me to work with, I'll definitely switch to that and rebuild my experience on top of that. But for now this is what we have. Admittedly this reflects my level of experience and my expertise too. I don't have a lot of experience working with building UX frameworks, so I adapt what I can.