5 ms·

Have a Coherent AI Policy

70 points by ai_critic 5mo ago

lprimeisafk 5mo ago
When I see "in the year of our Lord" I immediately tune out the writer. Almost as bad as "Unreasonable Effectiveness"
- antonvs 5mo ago
  Ikr. It should be “in the year 2026 of the Common Era.”
- deleted 5mo ago
  [deleted]
__MatrixMan__ 5mo ago
For another type of incoherent policy: don't restrict your employees to 2025 models and then accuse them of being sticks in the mud when they say the models are inadequate.
mikestorrent 5mo ago
> You must understand what your AI generated code does Absolutely yes. > You must be able to do your job if your AI tooling disappears Absolutely not. Look, I'm an alright programmer. Not good, far from great. Interpreted languages work for me; add all that strong typing and compilation and it starts to go beyond what I'm interested in. Nonetheless, pre-AI, I have shipped many very functional, production-grade applications for many companies. Now, I can write stuff in Go, and Rust, and it's fantastic. So much faster. The AI likes the strong typing, the test-ability, predictability, it all makes total sense. I'm using this stuff all the time, but I have not learned any Go; I'm too busy focusing on the parts the AI cannot do for me, like real requirements gathering, architecture, fit and finish, engaging stakeholders, etc. that still require the human touch. Maybe I could have learned some Go using that time, but at the end of the day my employer is paying me for results, not for my edification! There are now huge parts of my job I cannot do without AI. Sure, it's like 800-1200 bucks a month of extra cost; ok; but with that extra low-5-figs a year of cost I am a much better employee in terms of my capabilities. It's easily delivering ROI for me, and therefore for my employer. Instead of sitting around wishing I had a Go developer to ask for help implementing a simple feature in a Terraform provider, I can just fork it and add what I need, try to submit it upstream for inclusion, etc. and the lack of language specific skills is no longer holding me back. Take away the tool and I can't do that part of the job anymore, sorry. I can still do a lot, but slower, and honestly it would feel like going from a car back to walking, now; walking's fun, I do it recreationally for the sheer joy, but when there's hundreds of kilometres to cover in a short amount of time, the car is clearly the correct choice. So too is it with AI: we've invented the car for computers and only a fool would pretend he can do everything the same without it now.
- sublinear 5mo ago
  > only a fool would pretend he can do everything the same without it now Unless you're working in a coding sweatshop, I don't see why you need AI to do what people have been doing for decades just fine without breaking a sweat. What are you working on?
  - potsandpans 5mo ago
    > Unless you're working in a coding sweatshop You are obviously unaware of what the silicon valley companies are asking for and commiting to.
    - applfanboysbgon 5mo ago
      The same shit they've always been asking for, judging by what OpenAI and Anthropic are pumping out surrounding their models: bloated, buggy Electron apps that consume gigabytes of memory to display fucking <1kb of text. We are not witnessing better software, even from the people who have unlimited capital and unlimited access to frontier models and are true believers in its potential to replace engineers.
      - pixl97 5mo ago
        Better software means nearly nothing at the end of the day. The software that gets used is the metric that really matters. You can write 'perfect' software that runs at 100% efficient and never makes a mistake, but if no one ever downloads it and uses it, you've just engaged in a bout of intellectual masturbation. And honestly I've seen the 'write better software' people complain for years as Microsoft just absolutely financially decimates them. And yea, Microsoft loves writing bloated electron crap. And "one of these days Alice, people are going to rise up and use less bloated software and Microsoft is going to die", lol, just fucking kidding, people will never do that.
        applfanboysbgon 5mo ago
        Bun and uv were better software than their alternatives and gained massive traction quickly, leading to them selling out for a big payday. Better software has to overcome a massive marketing advantage, ecosystem capture, and inertia, but people are absolutely interested in using things that aren't buggy bloated bullshit where they have a choice. I don't really know why you brought Microsoft up. I don't know anyone who thinks writing better software can displace Windows. Windows has absolute ecosystem capture, notably on the hardware front -- you can't write a better OS even if you want to because hardware vendors simply won't work with you, and even if you did write a better OS you have to contend with not having 30 years of software developed for it. Computers are increasingly falling into the domain of professionals (with smartphones displacing casual consumer usage), who need Photoshop, Excel, and all of the rest of professional tools and would put up with an inferior OS out of necessity, because the tools are more important than the OS. Even then, Windows is an excellent piece of software, technically. It is handicapped by explicit anti-consumer decisions shoving things users don't want into it, but the kernel is hands down better than Linux's kernel, and the userspace is superior on technical merits if not user-friendliness merits. Windows has been going downhill in more recent years, but the gap between it and Linux is still massive. Really, I don't know how Microsoft is your go-to example, when they are actually a producer of excellent software. Excel, the modern .NET ecosystem, C#, and Typescript are top-tier, and VSCode is perhaps the only software I've used that justifies being an Electron application, because it actually exposes the capability to completely customise every aspect of it and extend it with sandboxed extensions. To the extent people have grievances with Microsoft, it is largely because of deliberate monopolistic practices rather than the technical quality of their software. I guess you are conceding that LLMs can't write good software, but are suggesting that good software doesn't matter. I think it does matter very much. In cases of monopoly control, people will begrudgingly use bad software, but they won't be loyal and you will bleed users over time as the frustrations build. But I think most critically, in cases where you don't already have monopoly control, nobody will use your bad software. This is why we haven't seen any vibe coded applications really taking the world by storm despite all the LLM hype. OpenAI and Anthropic can make you use their bad software because it's the gate to a useful proprietary tool, which is their real attraction, but Random Startup #482942 cannot make you use their bad software. Creating good software doesn't guarantee that Random Startup #482942 will succeed, given other market factors, but creating bad software guarantees they won't succeed.
riffic 5mo ago
DORA.dev (DevOps Research And Assessment) also point to having a clearly communicated stance concerning AI to be a foundational capability. https://dora.dev/capabilities/clear-and-communicated-ai-stance/ https://dora.dev/capabilities/clear-and-communicated-ai-stan...
ktallett 5mo ago
Academia is the place with the least coherent policy. In the few institutions I'm aware of the AI rules for, the guide is usually 3 lines long and it is basically we don't promote usage of it, which is a meaningless phrase. Therefore you end up with students who are not supposed to use it unless they are international masters students who require it because of language barriers, and in that scenario, it is basically allow them to use it however they like even if it makes a mockery of the rigour of a degree. Lecturers can use it as and when they wish, then you get researchers who either use it endlessly or not at all. Then upper management who use it instead of using their own brain.
ngriffiths 5mo ago
As a fun exercise replace AI with "junior" and "junior" with "mid-level." It holds up pretty well, as a manager you have responsibility for the work your team does and "make everyone put in more hours for no reason" is dumb. Maybe it comes across a bit neglecting of the "juniors" (in particular, it doesn't show any desire for figuring out ways for AI/"the juniors" to grow their responsibilities in a sustainable way). Imagine reading that version as someone who doesn't know how big companies work. "But then they'll just fire all the mid-level managers, since they don't do any of the actual work!" Haha, boy would you be wrong.
djoldman 5mo ago
Is anyone actually at a company that is purposely trying to use a ton of tokens? It gets expensive really fast.
- denkmoon 5mo ago
  For about 2 months, then I assume our fearless leaders saw the bill and wet themselves. Since then, Opus is off limits lol.
- jwilliams 5mo ago
  I've personally had people talk about token leaderboards at their work. Amazon and Meta did have ones, but I'd take with a decent grain of salt. We all know it's such an insanely gameable metric you'd be insane to actually use it...
  - djoldman 5mo ago
    I came across a comedy clip where the employees are fighting over how many billion tokens they were using and assumed it was a joke.
    - adonovan 5mo ago
      It seems quite common for the infrastructure teams to put up a dashboard just to keep a sense of what is going on, but it is then misinterpreted as a “leaderboard” and encourages the most prolific users to find creative ways to squander more to stay the “winner”. Management is slightly disappointed by the waste but also happy that staff are engaging with their future replacements.
  - etrautmann 5mo ago
    many of these leaderboards were built internally by curious ICs and taken down by leadership (for obvious reasons).
- bormaj 5mo ago
  Yes, but also this particular company has the means to justify the expense. I think there's enough opportunity at scale in the industry to really change the business landscape. Aside from that (and assuming a large enough sample size) I think it's a safe experiment to at least bet on finding profitable use cases. In 1-2 years, after this experiment runs its course, not everyone will have "unlimited" usage.
jwilliams 5mo ago
> It’s Still Your Code... AI maximalists will read this section and scoff. They’re already vibe coding everything and have little to no idea what the generated code looks like. This frames the argument like a dichotomy. And to be honest, using the Social Media "vibe-coding" as a strawman risks anchoring against something that's a mirage. There are plenty of good engineers getting good results whilst accepting code-ownership as a continuum. > If Claude goes down tomorrow, can you still do your job? This is a valid counterpoint, but doing software is already a tricky set of dependencies. The answer here isn't automatically "you need to be able to do everything". It could simply be also use Codex. I think the overall point is well made, I just don't agree with the absolute framing. There are things you can hand over AI safely. Even if you start small and increment it'll have a decent impact.
- daemin 5mo ago
  Who's name do you put down on a code commit? Yours, or the LLM? If it's yours then it is your code and you are responsible for knowing all about it. If it's the LLM then what do we even need you for?
  - jwilliams 5mo ago
    I've worked on plenty of code where the original author is long gone. Not sure that holds water at all.
    - daemin 5mo ago
      So an LLM is an author that writes code and immediately leaves.
  - hperrin 5mo ago
    Lol, just cause your name is on it doesn’t mean it’s your code. You didn’t write it. You don’t own the copyrights. You couldn’t sue someone if they stole it. It’s not your code. Yeah, I get that it means “you’re responsible for it”, but the phrasing irks me.
- lmm 5mo ago
willio58 5mo ago
> You must be able to do your job if your AI tooling disappears While I agree with the sentiment I fear this won’t last long. I already find myself, when Claude goes down for 15 minutes due to whatever, kind of throwing my hands up and taking a walk assuming it’ll be up by the time I get back. Usually it is. If it went away for good I’d be able to code, but would I want to? I’d be kind of bummed in a way. Which is odd because I used to tout myself as someone who like programming but I think what I’ve discovered is I like building.
- xboxnolifes 5mo ago
  I'd still want to code without AI if it's permanently gone. But if it's gone for 10 minutes? Sure, I'll switch to something else.
- ray_v 5mo ago
  If your spaceship is broken, you'd still be able to fly back to the moon on your own just like our forefathers did! /s It feels like the arguments here are very "status quo biased" - Ok, sure - I guess for some shops the code is the source of truth for the org and you really have to be prepared/equipped to deeply reason around the code and these so-called-patterns. Really it seems to me that the era of "agentic coding" has uncovered some uncomfortable truths. This is purely speculative, but it seems like most shops maintain little-to-no specification documentation for defining the code behavior - the code IS the documentation and that feels dangerous from a software engineering / architectural statdpoint.
hperrin 5mo ago
https://sciactive.com/human-contribution-policy/#Reasoning https://sciactive.com/human-contribution-policy/#Reasoning
theuniverseson 5mo ago
[flagged]

70 points by ai_critic 5mo ago

lprimeisafk 5mo ago

When I see "in the year of our Lord" I immediately tune out the writer. Almost as bad as "Unreasonable Effectiveness"

antonvs 5mo ago
Ikr. It should be “in the year 2026 of the Common Era.”
deleted 5mo ago
[deleted]

__MatrixMan__ 5mo ago

For another type of incoherent policy: don't restrict your employees to 2025 models and then accuse them of being sticks in the mud when they say the models are inadequate.

mikestorrent 5mo ago

> You must understand what your AI generated code does Absolutely yes. > You must be able to do your job if your AI tooling disappears Absolutely not. Look, I'm an alright programmer. Not good, far from great. Interpreted languages work for me; add all that strong typing and compilation and it starts to go beyond what I'm interested in. Nonetheless, pre-AI, I have shipped many very functional, production-grade applications for many companies. Now, I can write stuff in Go, and Rust, and it's fantastic. So much faster. The AI likes the strong typing, the test-ability, predictability, it all makes total sense. I'm using this stuff all the time, but I have not learned any Go; I'm too busy focusing on the parts the AI cannot do for me, like real requirements gathering, architecture, fit and finish, engaging stakeholders, etc. that still require the human touch. Maybe I could have learned some Go using that time, but at the end of the day my employer is paying me for results, not for my edification! There are now huge parts of my job I cannot do without AI. Sure, it's like 800-1200 bucks a month of extra cost; ok; but with that extra low-5-figs a year of cost I am a much better employee in terms of my capabilities. It's easily delivering ROI for me, and therefore for my employer. Instead of sitting around wishing I had a Go developer to ask for help implementing a simple feature in a Terraform provider, I can just fork it and add what I need, try to submit it upstream for inclusion, etc. and the lack of language specific skills is no longer holding me back. Take away the tool and I can't do that part of the job anymore, sorry. I can still do a lot, but slower, and honestly it would feel like going from a car back to walking, now; walking's fun, I do it recreationally for the sheer joy, but when there's hundreds of kilometres to cover in a short amount of time, the car is clearly the correct choice. So too is it with AI: we've invented the car for computers and only a fool would pretend he can do everything the same without it now.

sublinear 5mo ago
> only a fool would pretend he can do everything the same without it now Unless you're working in a coding sweatshop, I don't see why you need AI to do what people have been doing for decades just fine without breaking a sweat. What are you working on?
- potsandpans 5mo ago
  > Unless you're working in a coding sweatshop You are obviously unaware of what the silicon valley companies are asking for and commiting to.
  - applfanboysbgon 5mo ago
    The same shit they've always been asking for, judging by what OpenAI and Anthropic are pumping out surrounding their models: bloated, buggy Electron apps that consume gigabytes of memory to display fucking <1kb of text. We are not witnessing better software, even from the people who have unlimited capital and unlimited access to frontier models and are true believers in its potential to replace engineers.
    - pixl97 5mo ago
      Better software means nearly nothing at the end of the day. The software that gets used is the metric that really matters. You can write 'perfect' software that runs at 100% efficient and never makes a mistake, but if no one ever downloads it and uses it, you've just engaged in a bout of intellectual masturbation. And honestly I've seen the 'write better software' people complain for years as Microsoft just absolutely financially decimates them. And yea, Microsoft loves writing bloated electron crap. And "one of these days Alice, people are going to rise up and use less bloated software and Microsoft is going to die", lol, just fucking kidding, people will never do that.
      - applfanboysbgon 5mo ago
        Bun and uv were better software than their alternatives and gained massive traction quickly, leading to them selling out for a big payday. Better software has to overcome a massive marketing advantage, ecosystem capture, and inertia, but people are absolutely interested in using things that aren't buggy bloated bullshit where they have a choice. I don't really know why you brought Microsoft up. I don't know anyone who thinks writing better software can displace Windows. Windows has absolute ecosystem capture, notably on the hardware front -- you can't write a better OS even if you want to because hardware vendors simply won't work with you, and even if you did write a better OS you have to contend with not having 30 years of software developed for it. Computers are increasingly falling into the domain of professionals (with smartphones displacing casual consumer usage), who need Photoshop, Excel, and all of the rest of professional tools and would put up with an inferior OS out of necessity, because the tools are more important than the OS. Even then, Windows is an excellent piece of software, technically. It is handicapped by explicit anti-consumer decisions shoving things users don't want into it, but the kernel is hands down better than Linux's kernel, and the userspace is superior on technical merits if not user-friendliness merits. Windows has been going downhill in more recent years, but the gap between it and Linux is still massive. Really, I don't know how Microsoft is your go-to example, when they are actually a producer of excellent software. Excel, the modern .NET ecosystem, C#, and Typescript are top-tier, and VSCode is perhaps the only software I've used that justifies being an Electron application, because it actually exposes the capability to completely customise every aspect of it and extend it with sandboxed extensions. To the extent people have grievances with Microsoft, it is largely because of deliberate monopolistic practices rather than the technical quality of their software. I guess you are conceding that LLMs can't write good software, but are suggesting that good software doesn't matter. I think it does matter very much. In cases of monopoly control, people will begrudgingly use bad software, but they won't be loyal and you will bleed users over time as the frustrations build. But I think most critically, in cases where you don't already have monopoly control, nobody will use your bad software. This is why we haven't seen any vibe coded applications really taking the world by storm despite all the LLM hype. OpenAI and Anthropic can make you use their bad software because it's the gate to a useful proprietary tool, which is their real attraction, but Random Startup #482942 cannot make you use their bad software. Creating good software doesn't guarantee that Random Startup #482942 will succeed, given other market factors, but creating bad software guarantees they won't succeed.

riffic 5mo ago

DORA.dev (DevOps Research And Assessment) also point to having a clearly communicated stance concerning AI to be a foundational capability. https://dora.dev/capabilities/clear-and-communicated-ai-stance/ https://dora.dev/capabilities/clear-and-communicated-ai-stan...

ktallett 5mo ago

Academia is the place with the least coherent policy. In the few institutions I'm aware of the AI rules for, the guide is usually 3 lines long and it is basically we don't promote usage of it, which is a meaningless phrase. Therefore you end up with students who are not supposed to use it unless they are international masters students who require it because of language barriers, and in that scenario, it is basically allow them to use it however they like even if it makes a mockery of the rigour of a degree. Lecturers can use it as and when they wish, then you get researchers who either use it endlessly or not at all. Then upper management who use it instead of using their own brain.

ngriffiths 5mo ago

As a fun exercise replace AI with "junior" and "junior" with "mid-level." It holds up pretty well, as a manager you have responsibility for the work your team does and "make everyone put in more hours for no reason" is dumb. Maybe it comes across a bit neglecting of the "juniors" (in particular, it doesn't show any desire for figuring out ways for AI/"the juniors" to grow their responsibilities in a sustainable way). Imagine reading that version as someone who doesn't know how big companies work. "But then they'll just fire all the mid-level managers, since they don't do any of the actual work!" Haha, boy would you be wrong.

djoldman 5mo ago

Is anyone actually at a company that is purposely trying to use a ton of tokens? It gets expensive really fast.

denkmoon 5mo ago
For about 2 months, then I assume our fearless leaders saw the bill and wet themselves. Since then, Opus is off limits lol.
jwilliams 5mo ago
I've personally had people talk about token leaderboards at their work. Amazon and Meta did have ones, but I'd take with a decent grain of salt. We all know it's such an insanely gameable metric you'd be insane to actually use it...
- djoldman 5mo ago
  I came across a comedy clip where the employees are fighting over how many billion tokens they were using and assumed it was a joke.
  - adonovan 5mo ago
    It seems quite common for the infrastructure teams to put up a dashboard just to keep a sense of what is going on, but it is then misinterpreted as a “leaderboard” and encourages the most prolific users to find creative ways to squander more to stay the “winner”. Management is slightly disappointed by the waste but also happy that staff are engaging with their future replacements.
- etrautmann 5mo ago
  many of these leaderboards were built internally by curious ICs and taken down by leadership (for obvious reasons).
bormaj 5mo ago
Yes, but also this particular company has the means to justify the expense. I think there's enough opportunity at scale in the industry to really change the business landscape. Aside from that (and assuming a large enough sample size) I think it's a safe experiment to at least bet on finding profitable use cases. In 1-2 years, after this experiment runs its course, not everyone will have "unlimited" usage.

jwilliams 5mo ago

> It’s Still Your Code... AI maximalists will read this section and scoff. They’re already vibe coding everything and have little to no idea what the generated code looks like. This frames the argument like a dichotomy. And to be honest, using the Social Media "vibe-coding" as a strawman risks anchoring against something that's a mirage. There are plenty of good engineers getting good results whilst accepting code-ownership as a continuum. > If Claude goes down tomorrow, can you still do your job? This is a valid counterpoint, but doing software is already a tricky set of dependencies. The answer here isn't automatically "you need to be able to do everything". It could simply be also use Codex. I think the overall point is well made, I just don't agree with the absolute framing. There are things you can hand over AI safely. Even if you start small and increment it'll have a decent impact.

daemin 5mo ago
Who's name do you put down on a code commit? Yours, or the LLM? If it's yours then it is your code and you are responsible for knowing all about it. If it's the LLM then what do we even need you for?
- jwilliams 5mo ago
  I've worked on plenty of code where the original author is long gone. Not sure that holds water at all.
  - daemin 5mo ago
    So an LLM is an author that writes code and immediately leaves.
- hperrin 5mo ago
  Lol, just cause your name is on it doesn’t mean it’s your code. You didn’t write it. You don’t own the copyrights. You couldn’t sue someone if they stole it. It’s not your code. Yeah, I get that it means “you’re responsible for it”, but the phrasing irks me.
lmm 5mo ago

willio58 5mo ago

> You must be able to do your job if your AI tooling disappears While I agree with the sentiment I fear this won’t last long. I already find myself, when Claude goes down for 15 minutes due to whatever, kind of throwing my hands up and taking a walk assuming it’ll be up by the time I get back. Usually it is. If it went away for good I’d be able to code, but would I want to? I’d be kind of bummed in a way. Which is odd because I used to tout myself as someone who like programming but I think what I’ve discovered is I like building.

xboxnolifes 5mo ago
I'd still want to code without AI if it's permanently gone. But if it's gone for 10 minutes? Sure, I'll switch to something else.
ray_v 5mo ago
If your spaceship is broken, you'd still be able to fly back to the moon on your own just like our forefathers did! /s It feels like the arguments here are very "status quo biased" - Ok, sure - I guess for some shops the code is the source of truth for the org and you really have to be prepared/equipped to deeply reason around the code and these so-called-patterns. Really it seems to me that the era of "agentic coding" has uncovered some uncomfortable truths. This is purely speculative, but it seems like most shops maintain little-to-no specification documentation for defining the code behavior - the code IS the documentation and that feels dangerous from a software engineering / architectural statdpoint.

hperrin 5mo ago

https://sciactive.com/human-contribution-policy/#Reasoning https://sciactive.com/human-contribution-policy/#Reasoning

theuniverseson 5mo ago

[flagged]