alexhans

7 ms·

61.
by alexhans 8mo ago
I think a key ingredient here is accountabilty and liability. If there's a mistake, you can't blame the computer. Who is the human accountable at the end of it all? If there's liability, who pays for it? That's where def
62.
by alexhans 8mo ago
That's a decent practice from the lens of reducing blast radius. It becomes harder when you start thinking about unattended systems that don't have you in the loop. One problem I'm finding discussion about automation or semi
63.
by alexhans 8mo ago
I'd be happy to. One thing that is tough is knowing what will resonate with the audience and not being too simple or too complex. What do you think would resonate with you or with the audience you're thinking about? That repo als
64.
by alexhans 8mo ago
I wrote https://ai-evals.io (community site) to make the concept approachable no matter what tools you choose to use. You can learn about them evaluating that site https://github.com/Alexhans/eval-ception a
65.
by alexhans 8mo ago
This fun exercise might actually be extremely insightful as a educational vehicle around AI and intent. It can also help combat the excessive emphasis on any "end to end" demo on twitter which doesn't really correspond to a d
66.
by alexhans 8mo ago
I think the prototype thing is absolutely true but breaks down like all prototypes at the level of collaborating, sharing and evolving while handling entropy throug simplicity UNLESS you know what you're doing or the agent steers you w
67.
by alexhans 8mo ago
We won't be able to be sure of 100% with LLMs but maybe proper engineering around evals get us to an acceptable level of quality based on the blast radius/safety profile. I'd also argue that we should be pushing towards trace
68.
by alexhans 8mo ago
Nice to see you here (Just reached out on bluesky over sandboxing - gandolin). I follow your work and agree and am hoping that you and others who have well earned audiences based on awesome open source work, can help with the advocacy on me
69.
by alexhans 8mo ago
It doesn't disappear but it does easy up some instances of fighting configuration, documentation, syntax or even comparing three approaches that are similar but you don't know their full effect. I think it's a very fun space,
70.
by alexhans 8mo ago
Thanks! Very useful.
71.
by alexhans 8mo ago
I am pushing for this judicious Eval [1] Driven Development where tech/non tech users use intent when coding to minimally design some known aspects and try to build simply and using common standards across the team (that should be in t
72.
by alexhans 8mo ago
This is fantastic. How long have you used it? Can you tell us more? I'll probably buy one of these in the near future once I can research a bit more and I have to decide where to buy it (if in USA on travel, or ship from wherever they
73.
by alexhans 8mo ago
I'm having trouble parsing exactly what you're seeking but maybe this post is worth a look https://www.nicolasbustamante.com/p/lessons-from-building-ai... Specially the part around temporal. If you have drif
74.
by alexhans 8mo ago
Thanks for the insights. You don't change your priorities based on my opinion but adding an RSS should be quite easy as it's just an auto-generated XML. I don't have any concrete answers but some things to consider: - Are you
75.
by alexhans 8mo ago
It does. Great. Thanks for the quick update.
76.
by alexhans 8mo ago
Thanks for the insight. This resonates with the same explanation I tried to give around non determinism tasks that feed into each other [1]. If you don't break the cycle with determinism or, in your case, CPCs, you get to a stage wher
77.
by alexhans 8mo ago
> Don't let your dog run errand and use a good leash. I think the key part is who are you talking to. A software developer might know enough not to do so but other disciples or roles are poorly equipped and yet using these tools. S
78.
by alexhans 8mo ago
I thought of doing a similar LLM in a AI evals teaching site to tell users to interact through it but was concerned with inducing users into a prompt injection friendly pattern.
79.
by alexhans 8mo ago
I've naturally done this a lot and suggested that other people prompt this way. I can see how a "ready made" solution with this behaviour could be interesting. The compliance parts are good to make clear considerating one seg
80.
by alexhans 8mo ago
Point taken. I was mistakenly assuming a conversational agent experience. I love the idea of showing how easy prompt injection or data exfiltration could be in a safe environment for the user and will definitely keep an eye out on any good
81.
by alexhans 8mo ago
Great endeavour. Land of lisp is still one of my favourite alternative programming books. One thing about the sample. Is there a chance to get a glimpse of a random chapter you like a lot? Most of it seems to be foreword/acknowledgeme
82.
by alexhans 8mo ago
The website is great as a concept but I guess it mimics an increasingly rare one off interaction without feedback. I understand the cost and technical constraints but wouldn't an exposed interface allow repeated calls from different en
83.
by alexhans 8mo ago
I had a similar experience trying to get feedback on my attempt to help different role families adopt AI evals as a common language (hands on tutorial or tool comparison). https://news.ycombinator.com/item?id=47026263 I att
84.
by alexhans 8mo ago
This is great. I really want to find simple secure defaults when I share people how to eval [1] and bwrap / srt still feel somewhat cumbersome if you think about non tech roles. Do you have any information on estimated overhead? Inform
85.
by alexhans 8mo ago
Isn't the title editorialised? Probably for clicks? I think that most of the adoption around Agent Skills would have a focus on ease of use, standarization and context management and not correctness. My own thoughts on how to approach
86.
by alexhans 8mo ago
It really depends on the type of agent and problem but your path resonates as a way to slowly build more complexity as you start understanding the uncertain space. I've written about it in my blog with the intent of helping non tech u
87.
by alexhans 8mo ago
Beautiful. Have you considered adding a "replay certain timeline" feature so that users get the feel of the throughput and emergence much like Gource [1] did for git? [1] https://gource.io/
88.
by alexhans 8mo ago
Fair enough. I think I'm trying to say the same thing you're saying. It's tough to navigate that world without a lot of investment in understanding the internals which, most people wouldn't do (rational behaviour for &q
89.
by alexhans 8mo ago
Nice article. I think a key part of the conversation is getting people to start thinking in terms of evals [1] and observability but it's been quite tough to combat the hype of "but X magic product just solves what you mentioned a
90.
by alexhans 8mo ago
Git is a beautiful piece of software but it does expose complexity in a very by programmers for programmers kind of way. I've successfully gotten many non tech roles to use git but there's usually a lot missed in the nuances and p

More ›