3 ms·

The next thing in user interfaces is combining LLMs with GUIs to create a flexible interface that isn't fully defined in advance but works with the user to adap

The next thing in user interfaces is combining LLMs with GUIs to create a flexible interface that isn't fully defined in advance but works with the user to adapt to their needs. This is a total departure from the pixel-copying design methods we have today and there is no tooling yet for designing such an interface. In a few years time, we'll expect all applications to have LLM inputs and full natural language understanding in addition to their menus and buttons, just like we expect all software today to make use of the internet where before, none did.

johneth 2y ago
> combining LLMs with GUIs to create a flexible interface that isn't fully defined in advance but works with the user to adapt to their needs I doubt this. Or, I doubt that it'll catch on. Unlearnable UIs, as in those that 'adapt' to a user's needs (meaning change unpredictably) will just confuse and put off users. LLMs being wrong at least enough of the time to make you not trust their output also makes me doubt this style of interface.
- dtagames 2y ago
  Nothing says it has to be unlearnable or change at whim. Imagine your bank's website allowing you to say "show me what I earned from Wilco last year" and have it make the UI selections to achieve this. We will absolutely see these type of interfaces. Imagine how much Photoshop could be improved if you could simply ask to surface the tool you need instead of rearranging the interface yourself and hunting things down. Just like we expect apps now to be able to use the net, not just info on our own machines. You can already ask ChatGPT about many interfaces in popular software and it will give you step by step instructions for nearly any scenario. Soon, apps will integrate the understanding with the button and menu selections and offer to do it for you.

3 ms·

The next thing in user interfaces is combining LLMs with GUIs to create a flexible interface that isn't fully defined in advance but works with the user to adap

by dtagames 2y ago

johneth 2y ago
> combining LLMs with GUIs to create a flexible interface that isn't fully defined in advance but works with the user to adapt to their needs I doubt this. Or, I doubt that it'll catch on. Unlearnable UIs, as in those that 'adapt' to a user's needs (meaning change unpredictably) will just confuse and put off users. LLMs being wrong at least enough of the time to make you not trust their output also makes me doubt this style of interface.
- dtagames 2y ago
  Nothing says it has to be unlearnable or change at whim. Imagine your bank's website allowing you to say "show me what I earned from Wilco last year" and have it make the UI selections to achieve this. We will absolutely see these type of interfaces. Imagine how much Photoshop could be improved if you could simply ask to surface the tool you need instead of rearranging the interface yourself and hunting things down. Just like we expect apps now to be able to use the net, not just info on our own machines. You can already ask ChatGPT about many interfaces in popular software and it will give you step by step instructions for nearly any scenario. Soon, apps will integrate the understanding with the button and menu selections and offer to do it for you.