3 ms·

My team is in the agentic orchestrator phase. I like this software factory pattern in concept, but our biggest challenges in development are acceptance testing

by bicx 15d ago

My team is in the agentic orchestrator phase. I like this software factory pattern in concept, but our biggest challenges in development are acceptance testing of anything UI-related. Mobile app testing in particular is still a huge bottleneck that requires a human. AI models really suck at identifying poor usability and jank, particularly because they only typically process snapshots of the app from an instance in time. I know there are traditional testing frameworks that can detect jitter and frame drop to a certain level. We could potentially start having agents build that in. If we had concrete designs and specs on every project, that would also be helpful, but in a fast-moving startup, that gets delegated to the builders. That puts a human back in the loop every time. Curious to hear what anyone else does to fully adopt a software factory pattern.

nkrisc 15d ago
If your UI is created for humans to use, you should still have humans involved in testing it.
sroerick 15d ago
I totally agree with this. Agents seem tremendously bad at UI to me. Maybe it's just because I am a back end guy. Right now I'm working on a declarative UI framework which can help me along here. My thought is that if I sacrifice a little control for sane primitives, that will make that spec /build loop easier. I think ClayUI is a really interesting "reduced instruction set" for UI. I don't know that immediate mode UI is the right call for anything web related (that's how you get React lol) but his reduced primitive layer is very interesting to me
- flir 14d ago
  > Right now I'm working on a declarative UI framework which can help me along here After your first para, I was about to suggest exactly that (well, maybe not writing your own). I find that frameworks (both front-end and back-end) constrain the LLM's choices and result in both sensible defaults and improved consistency. Of course, you will immediately hit the problem all frameworks have: customer requirements that the framework components don't quite meet. Still, great for RAD.
  - sroerick 14d ago
    Can you elaborate on what frameworks you've used? and how they've helped
deleted 14d ago
[deleted]

by bicx 15d ago

nkrisc 15d ago

If your UI is created for humans to use, you should still have humans involved in testing it.

sroerick 15d ago

I totally agree with this. Agents seem tremendously bad at UI to me. Maybe it's just because I am a back end guy. Right now I'm working on a declarative UI framework which can help me along here. My thought is that if I sacrifice a little control for sane primitives, that will make that spec /build loop easier. I think ClayUI is a really interesting "reduced instruction set" for UI. I don't know that immediate mode UI is the right call for anything web related (that's how you get React lol) but his reduced primitive layer is very interesting to me

flir 14d ago
> Right now I'm working on a declarative UI framework which can help me along here After your first para, I was about to suggest exactly that (well, maybe not writing your own). I find that frameworks (both front-end and back-end) constrain the LLM's choices and result in both sensible defaults and improved consistency. Of course, you will immediately hit the problem all frameworks have: customer requirements that the framework components don't quite meet. Still, great for RAD.
- sroerick 14d ago
  Can you elaborate on what frameworks you've used? and how they've helped

deleted 14d ago

[deleted]