7 ms·

We've been put in the vibe space

164 points by bertman 2y ago

loa_in_ 2y ago
I think the first diagram of the spaces is reasonable. I do imagine there's a whole lot more space around the included region that's still unexplored
bsenftner 2y ago
I'm dealing with the issues discussed in the article as I roll out project management software for attorneys that has deep LLM integration. I have four different types of LLM integration, but three of them use that same "familiar" chat request interface as their UI, and the fact that they are different interfaces for different purposes is very difficult for the end-users to reliably understand. There's the familiar "chat" interface that they are used to, and understand readily, which is used for legal R&D and new client on-boarding. But there are also traditional word processing and spreadsheet document authoring and editing, which also use that same "chat UI" - but that is not a conversational interface; requests to that interface cause modifications to the document, rewriting portions of the word processing or spreadsheet document. A third type of LLM integration is available that is a RAG type of interface for authored and uploaded documents, where the end-user is able to ask questions about the document with a generated subject matter expert in the document's topic (a word processor or spreadsheet). My fourth LLM integration has no interface, it makes an assessment of the end-user's familiarity with their goal, and adjusts the amount of help offered to the end-user. I am currently in the process of adding textual dressing around these three LLM request interfaces, but users are too used to operating interfaces on instinct, and the second and third type of LLM integrations where the LLM is co-authoring a document with the end-user is getting scant use, they simply don't realize it can co-author with them, and try to use it like a Q&A interface and not an assistant who will change the document for them.
- superb_dev 2y ago
  How visually distinct are the 3 interfaces?
  - bsenftner 2y ago
    LLM interfaces look the same, the are a multi-line text area the end-user can resize, very much like the comment fields here on HackerNews. I'm considering changing their background color and adding other visual distinctions to make clear their different purposes. At first, I placed a simple text field on the toolbars of the word processor and the spreadsheet interfaces, but my initial testers thought they were a duplicated find/replace function and not an open ended LLM request field, despite a label and popup tool text saying otherwise. So I moved them to be separate, independent resizable text areas. The end-users I have testing the interfaces realize they are LLM interfaces now, but are not yet realizing that the word processor and spreadsheet interfaces can co-author with them. They ask the LLMs how to use questions, rather than "would you make this change" requests. To address that, I'm adding a drawer of example videos next to the LLM text area, so they have use examples showing them how to use the interface. No amount of text seems to help, people just don't read it.
- afro88 2y ago
  You might be trying to solve problems that they don't really have (but think they have), or you're solving them in the wrong way. For example, is a RAG chatbot the right way to get the information they need from documents, or do they want executive summaries of the document from different points of view that they almost always need to consider it from? Is a chat assistant that you have to talk to to make modifications to a document the right choice? Or would it be better to have the user highlight the text they want changed which then gives them the option to direct how to change it with natural language?
  - shermantanktop 2y ago
    Sounds like this is third party product craziness.
  - bsenftner 2y ago
    >Or would it be better to have the user highlight the text they want changed which then gives them the option to direct how to change it with natural language? That's exactly what I have: the user selects what they want modified, uses the "chat text area" to request the change, the entire document is sent in (for prose consistency) with the selection identified and the change is performed. If no selection is, the change request is applied to the entire document in context-output-size aware chunks. The RAG interface is not the only document query interface, but is the one preferred for finding information inside documents and discussing that information in large documents. My RAG implementation is a bit more than "just RAG", I use the document to generate a subject matter expert in the subject matter of the document, and that generated agent is who the end-user converses, along with the retrieved document information. So, in a way, my RAG is with the document, and an agent generated from the document, along with a context sensitive "professional communicator agent" whose job is to fill in the context breaks between the different RAG snippets, which reduces the fractured nature of the RAG snippets, lending towards better, deeper, more nunaced answers.
  - ooterness 2y ago
    > Or would it be better to have the user highlight the text they want changed which then gives them the option to direct how to change it with natural language? Users could also highlight the text they want changed, then skip the middleman by typing in the new text themselves. Not sure how to monetize this.
- htrp 2y ago
  any link to mock-ups to more clearly differentiate the UX experiences?
- pjc50 2y ago
  > the second and third type of LLM integrations where the LLM is co-authoring a document with the end-user is getting scant use, they simply don't realize it can co-author with them This is a very new and unfamiliar paradigm, and you're going to have to "train" people in how it works.
marginalia_nu 2y ago
The main benefit of putting everything in one chat box that ambiguously does a bit of everything is the opportunity for the service to creatively misinterpret the query and present tangentially related sponsored content as a result. The clearer the design vocabulary you offer, the easier it is to unequivocally express what you want, the more obvious it becomes when you get something other than what you want. It's sort of a shame because we end up with these tools that are designed to inhibit and take away user agency rather than extend and advance it.
- RunSet 2y ago
  > It's sort of a shame because we end up with these tools that are designed to inhibit and take away user agency rather than extend and advance it. "Behold, we have improved the hammer by integrating an intelligent advertisement selection algorithm. Hence, to someone equipped with only this tool, the solution to all problems will seem to be money."
  - marginalia_nu 2y ago
    There's some Saphir-Whorf-adjacent things to be said here. If all you're able to express with clarity is purchase intent, how can you be anything other than a consumer?
    - roenxi 2y ago
      That seems to be a rhetorical question, but I'm going to pretend I didn't notice. You can purchase future income streams or a share of the revenue of an enterprise. Then you'd be an investor.
      - pineaux 2y ago
        "consumer" is sometimes used as codespeak left leaning circles meaning: someone who implicitly agrees with capitalism and uses it without much real criticism. So an "investor" of your type is just someone who is more of a consumer. As they consume not only food or housing for example, but they also consume investment opportunities.
langsoul-com 2y ago
We could have the AI search added as a click able option or distinctly separated from the normal search. That would be the best of both worlds, low friction AI search, but not only AI search.
- Spivak 2y ago
  Kagi I think strikes the balance well, if the query ends with "?" it signals that you want an AI response.
  - JohnFen 2y ago
    Wait, what? Did Kagi give a warning about this that I missed?
    - Spivak 2y ago
      https://help.kagi.com/kagi/ai/quick-answer.html https://help.kagi.com/kagi/ai/quick-answer.html
      - JohnFen 2y ago
        Ah, so they did. I missed it -- probably because I'm not interested in that so wouldn't have read the docs about it. They give an option to disable it completely! Once again, my affection for Kagi is rewarded.
ansible 2y ago
> And on Whatsapp, I want to talk to people on Whatsapp directly, without a chatbot in the middle. They're doing this with Skype too. It annoys me. I have a list of business contacts, and that's all I use it for. My friends are elsewhere. I don't want to talk about the weather or politics. I have specific people to talk or chat with for specific topics. Microsoft, you are already trying to be my friend via Bing and whatever that is built right into Windows 11 already. Rant finished. I guess I understand why they do that. If I only used Skype, and no other Microsoft products, how else would I discover all the wonderful experiences I could have with an LLM? And change is inevitable, and how much money do I give Microsoft for this particular service to stay the same, etc., etc., etc..
dangoodmanUT 2y ago
god tier title
Barrin92 2y ago
Good article. It's really obnoxious that so many websites now offer you probablistic results when you're actually searching for some well defined thing you asked for in precise terms. Getting some ML suggestion should be an optional fallback if I can't articulate what I want, not a first choice. Youtube isn't mentioned in the piece but it's also one of the worst offenders of this. They shove so much "do you want to see this again?" or "for you" in the middle of your results it's crazy. Luckily using a search operator like ` after:[date]` still gets rid of it. I've noticed having to use operators on more and more sites to basically force them to not hand me random content back.
morkalork 2y ago
I was using Miro to make some diagrams for work recently and it popped up with a notice about their new AI assistant. I asked if it could write me a poem about gorrillas in French and it happily obliged. It was a good vibe. I have no idea what that has to do with software architecture though.
ryandrake 2y ago
> But I don’t want to ask Meta AI about the best history podcasts or ask about k-pop, I want to find the video of the trained raven I saw a couple weeks ago. I want to perform social keyword search. And on Whatsapp, I want to talk to people on Whatsapp directly, without a chatbot in the middle. It's getting increasingly evident that tech companies have stopped caring about what their users actually want to do, and instead are nudging/funneling them into doing what they want their users to do. For a while it was voice assistants. "Use Siri!!" "Use Assistant!" "Use Alexa!" "Use Cortana!" "No, really, please use it!" "We're begging you." "OK, we're making it more prominent so you accidentally use it." "We're now removing the other options so you have to use it!" I expect the same cycle with AI. "PLEASE, USERS, you have to use this AI product! It is the current fashion, after all. We pay $billions for GPUs and the salaries of hundreds of engineers, and it's expensive. You simply must use it for everything!"
- rml 2y ago
  I find this to be true in many areas of life, esp. in recent years. My perfect trail running shoe was made in 2006. My perfect car was made in 2008. Those products are no longer made, and the new ones are not "better", just "different". (Often worse tbh) We get to buy the product someone is willing to make, not the product we want to buy. The product is often not "better" for the user, but for the company making it. And as a user you can just FEEL it dripping from every new thing. It's not appealing at all to me. There must be an economics term that describes this. I've taken to calling it "supply side rules everything around me"
  - sph 2y ago
    I'm holding on to my iPhone 12 mini and BOSE QC 35 II for dear life. It is rare these days to find a product that just works for you, and you always know that it's gonna be replaced by a crappier, more expensive model as soon as it breaks.
  - shostack 2y ago
    A thousand times this. I distinctly recall going to a Columbia store to get a new basic fleece jacket years ago and all they had were jackets with that stupid breast zipper pocket which I didn't want. It clearly seemed like the new style as anything without it was nowhere to be found on the floor. I begged the sales person who dug one up in a box in the back room without the pocket. My guess is slapping an extra pocket on there let them justify increasing the cost and margins because "you're getting more!"
greasegum 2y ago
Not discussed in the article, but I think relevant, is the breakdown of search as a useful tool even on such venerated platforms as eBay, which has become similar to Facebook Marketplace in its vibes-based algorithm. Laser-focused search precision is a losing business model, apparently, and not the way of the future.
adfm 2y ago
I'm reminded of the toaster from Red Dwarf. Toast, anyone?

164 points by bertman 2y ago

I think the first diagram of the spaces is reasonable. I do imagine there's a whole lot more space around the included region that's still unexplored

bsenftner 2y ago

I'm dealing with the issues discussed in the article as I roll out project management software for attorneys that has deep LLM integration. I have four different types of LLM integration, but three of them use that same "familiar" chat request interface as their UI, and the fact that they are different interfaces for different purposes is very difficult for the end-users to reliably understand. There's the familiar "chat" interface that they are used to, and understand readily, which is used for legal R&D and new client on-boarding. But there are also traditional word processing and spreadsheet document authoring and editing, which also use that same "chat UI" - but that is not a conversational interface; requests to that interface cause modifications to the document, rewriting portions of the word processing or spreadsheet document. A third type of LLM integration is available that is a RAG type of interface for authored and uploaded documents, where the end-user is able to ask questions about the document with a generated subject matter expert in the document's topic (a word processor or spreadsheet). My fourth LLM integration has no interface, it makes an assessment of the end-user's familiarity with their goal, and adjusts the amount of help offered to the end-user. I am currently in the process of adding textual dressing around these three LLM request interfaces, but users are too used to operating interfaces on instinct, and the second and third type of LLM integrations where the LLM is co-authoring a document with the end-user is getting scant use, they simply don't realize it can co-author with them, and try to use it like a Q&A interface and not an assistant who will change the document for them.

superb_dev 2y ago
How visually distinct are the 3 interfaces?
- bsenftner 2y ago
  LLM interfaces look the same, the are a multi-line text area the end-user can resize, very much like the comment fields here on HackerNews. I'm considering changing their background color and adding other visual distinctions to make clear their different purposes. At first, I placed a simple text field on the toolbars of the word processor and the spreadsheet interfaces, but my initial testers thought they were a duplicated find/replace function and not an open ended LLM request field, despite a label and popup tool text saying otherwise. So I moved them to be separate, independent resizable text areas. The end-users I have testing the interfaces realize they are LLM interfaces now, but are not yet realizing that the word processor and spreadsheet interfaces can co-author with them. They ask the LLMs how to use questions, rather than "would you make this change" requests. To address that, I'm adding a drawer of example videos next to the LLM text area, so they have use examples showing them how to use the interface. No amount of text seems to help, people just don't read it.
afro88 2y ago
You might be trying to solve problems that they don't really have (but think they have), or you're solving them in the wrong way. For example, is a RAG chatbot the right way to get the information they need from documents, or do they want executive summaries of the document from different points of view that they almost always need to consider it from? Is a chat assistant that you have to talk to to make modifications to a document the right choice? Or would it be better to have the user highlight the text they want changed which then gives them the option to direct how to change it with natural language?
- shermantanktop 2y ago
  Sounds like this is third party product craziness.
- bsenftner 2y ago
  >Or would it be better to have the user highlight the text they want changed which then gives them the option to direct how to change it with natural language? That's exactly what I have: the user selects what they want modified, uses the "chat text area" to request the change, the entire document is sent in (for prose consistency) with the selection identified and the change is performed. If no selection is, the change request is applied to the entire document in context-output-size aware chunks. The RAG interface is not the only document query interface, but is the one preferred for finding information inside documents and discussing that information in large documents. My RAG implementation is a bit more than "just RAG", I use the document to generate a subject matter expert in the subject matter of the document, and that generated agent is who the end-user converses, along with the retrieved document information. So, in a way, my RAG is with the document, and an agent generated from the document, along with a context sensitive "professional communicator agent" whose job is to fill in the context breaks between the different RAG snippets, which reduces the fractured nature of the RAG snippets, lending towards better, deeper, more nunaced answers.
- ooterness 2y ago
  > Or would it be better to have the user highlight the text they want changed which then gives them the option to direct how to change it with natural language? Users could also highlight the text they want changed, then skip the middleman by typing in the new text themselves. Not sure how to monetize this.
htrp 2y ago
any link to mock-ups to more clearly differentiate the UX experiences?
pjc50 2y ago
> the second and third type of LLM integrations where the LLM is co-authoring a document with the end-user is getting scant use, they simply don't realize it can co-author with them This is a very new and unfamiliar paradigm, and you're going to have to "train" people in how it works.

marginalia_nu 2y ago

The main benefit of putting everything in one chat box that ambiguously does a bit of everything is the opportunity for the service to creatively misinterpret the query and present tangentially related sponsored content as a result. The clearer the design vocabulary you offer, the easier it is to unequivocally express what you want, the more obvious it becomes when you get something other than what you want. It's sort of a shame because we end up with these tools that are designed to inhibit and take away user agency rather than extend and advance it.

RunSet 2y ago
> It's sort of a shame because we end up with these tools that are designed to inhibit and take away user agency rather than extend and advance it. "Behold, we have improved the hammer by integrating an intelligent advertisement selection algorithm. Hence, to someone equipped with only this tool, the solution to all problems will seem to be money."
- marginalia_nu 2y ago
  There's some Saphir-Whorf-adjacent things to be said here. If all you're able to express with clarity is purchase intent, how can you be anything other than a consumer?
  - roenxi 2y ago
    That seems to be a rhetorical question, but I'm going to pretend I didn't notice. You can purchase future income streams or a share of the revenue of an enterprise. Then you'd be an investor.
    - pineaux 2y ago
      "consumer" is sometimes used as codespeak left leaning circles meaning: someone who implicitly agrees with capitalism and uses it without much real criticism. So an "investor" of your type is just someone who is more of a consumer. As they consume not only food or housing for example, but they also consume investment opportunities.

langsoul-com 2y ago

We could have the AI search added as a click able option or distinctly separated from the normal search. That would be the best of both worlds, low friction AI search, but not only AI search.

Spivak 2y ago
Kagi I think strikes the balance well, if the query ends with "?" it signals that you want an AI response.
- JohnFen 2y ago
  Wait, what? Did Kagi give a warning about this that I missed?
  - Spivak 2y ago
    https://help.kagi.com/kagi/ai/quick-answer.html https://help.kagi.com/kagi/ai/quick-answer.html
    - JohnFen 2y ago
      Ah, so they did. I missed it -- probably because I'm not interested in that so wouldn't have read the docs about it. They give an option to disable it completely! Once again, my affection for Kagi is rewarded.

ansible 2y ago

> And on Whatsapp, I want to talk to people on Whatsapp directly, without a chatbot in the middle. They're doing this with Skype too. It annoys me. I have a list of business contacts, and that's all I use it for. My friends are elsewhere. I don't want to talk about the weather or politics. I have specific people to talk or chat with for specific topics. Microsoft, you are already trying to be my friend via Bing and whatever that is built right into Windows 11 already. Rant finished. I guess I understand why they do that. If I only used Skype, and no other Microsoft products, how else would I discover all the wonderful experiences I could have with an LLM? And change is inevitable, and how much money do I give Microsoft for this particular service to stay the same, etc., etc., etc..

dangoodmanUT 2y ago

god tier title

Barrin92 2y ago

Good article. It's really obnoxious that so many websites now offer you probablistic results when you're actually searching for some well defined thing you asked for in precise terms. Getting some ML suggestion should be an optional fallback if I can't articulate what I want, not a first choice. Youtube isn't mentioned in the piece but it's also one of the worst offenders of this. They shove so much "do you want to see this again?" or "for you" in the middle of your results it's crazy. Luckily using a search operator like ` after:[date]` still gets rid of it. I've noticed having to use operators on more and more sites to basically force them to not hand me random content back.

morkalork 2y ago

I was using Miro to make some diagrams for work recently and it popped up with a notice about their new AI assistant. I asked if it could write me a poem about gorrillas in French and it happily obliged. It was a good vibe. I have no idea what that has to do with software architecture though.

ryandrake 2y ago

> But I don’t want to ask Meta AI about the best history podcasts or ask about k-pop, I want to find the video of the trained raven I saw a couple weeks ago. I want to perform social keyword search. And on Whatsapp, I want to talk to people on Whatsapp directly, without a chatbot in the middle. It's getting increasingly evident that tech companies have stopped caring about what their users actually want to do, and instead are nudging/funneling them into doing what they want their users to do. For a while it was voice assistants. "Use Siri!!" "Use Assistant!" "Use Alexa!" "Use Cortana!" "No, really, please use it!" "We're begging you." "OK, we're making it more prominent so you accidentally use it." "We're now removing the other options so you have to use it!" I expect the same cycle with AI. "PLEASE, USERS, you have to use this AI product! It is the current fashion, after all. We pay $billions for GPUs and the salaries of hundreds of engineers, and it's expensive. You simply must use it for everything!"

rml 2y ago
I find this to be true in many areas of life, esp. in recent years. My perfect trail running shoe was made in 2006. My perfect car was made in 2008. Those products are no longer made, and the new ones are not "better", just "different". (Often worse tbh) We get to buy the product someone is willing to make, not the product we want to buy. The product is often not "better" for the user, but for the company making it. And as a user you can just FEEL it dripping from every new thing. It's not appealing at all to me. There must be an economics term that describes this. I've taken to calling it "supply side rules everything around me"
- sph 2y ago
  I'm holding on to my iPhone 12 mini and BOSE QC 35 II for dear life. It is rare these days to find a product that just works for you, and you always know that it's gonna be replaced by a crappier, more expensive model as soon as it breaks.
- shostack 2y ago
  A thousand times this. I distinctly recall going to a Columbia store to get a new basic fleece jacket years ago and all they had were jackets with that stupid breast zipper pocket which I didn't want. It clearly seemed like the new style as anything without it was nowhere to be found on the floor. I begged the sales person who dug one up in a box in the back room without the pocket. My guess is slapping an extra pocket on there let them justify increasing the cost and margins because "you're getting more!"

greasegum 2y ago

Not discussed in the article, but I think relevant, is the breakdown of search as a useful tool even on such venerated platforms as eBay, which has become similar to Facebook Marketplace in its vibes-based algorithm. Laser-focused search precision is a losing business model, apparently, and not the way of the future.

adfm 2y ago

I'm reminded of the toaster from Red Dwarf. Toast, anyone?