4 ms·

I read the entire post, and I think this company has no expertise at all. So I'd rather they just used AI writing instead.At least Frontier model AI doesn't mak

by jdw64 3mo ago

I read the entire post, and I think this company has no expertise at all. So I'd rather they just used AI writing instead.At least Frontier model AI doesn't make such overblown claims. They proudly claim that every AI project they've observed over the past year and a half had a 0% success rate, and that they've rejected all AI implementation work. While this is evidence that the market is crazy, at its core, it's a painful confession that they have no engineering expertise to implement and control modern AI architectures like RAG, Agentic Workflow, and context window optimization to meet business requirements. I find it fascinating how they're packaging that. It's basically saying, 'We're behind the times.' There are already products that have achieved results by using AI as part of their development process, yet lumping all different types of AI usage into a single failure category is not only inaccurate but also misleading. Same goes for the Snowflake Cortex anecdote. Even a freelancer like me can explain technical limitations and distinguish between what's possible and what's not, especially when clients are eager. There's no engineering analysis in this entire post about why AI fails. No mention of technical bottlenecks like vector DB retrieval quality degradation or prompt injection failures. I've also worked on RAG for a specific company. For internal knowledge chatbots, it often fails depending on document collection rates and chunking. But none of that is mentioned. So I understand that AI projects and related things are bad. But there's no analysis of why. For example, regarding Snowflake, I'm not sure, but did they discuss accuracy in terms of what query set or what ground truth they were using? You're consultants, aren't you? Honestly, I don't understand why people are excited about this. I'd rather they just used AI. TIt's not about whether human writing is good or bad. It's that this kind of writing feels like a deception of the reader. When making overgeneralizations, there's a basic minimum standard required. Saying that making token usage a KPI makes it hard for employees to report is just an 'obvious' fact that's already appeared in far too many essays. Wake up. You're 'consultants.' Consultants are supposed to provide metrics and directions, but all you're doing is shouting into an echo chamber and asking for agreement. If a significant portion of corporate AI investments are shoddy, you could at least propose specific metrics like document collection rates or user evaluation scores using the very skills you claim to have. I really don't get it. Just use AI. I wish the OP had used AI. Let me be realistic.

robwilliams 3mo ago
The article is not a technical overview of AI's failures, it's about the organizational failures that lead to things like token leaderboards. I would argue that this article goes far beyond the "obvious" fact of token leaderboards and into the deeper problems faced by both vendors and companies trying to measure the efficacy of AI. How would AI writing help in any way?
- jdw64 3mo ago
  Honestly, it's just shouting into an echo chamber. They call it organizational failure, but it's really just 'come on, my side.' A consultant who does that is untrustworthy. It means they have no metrics. Same goes for their 0% claim. It's overgeneralization, isn't it? So why did they fail, what went wrong, and how should these AI projects be analyzed—and by what metrics? They skip all that and just throw out a single KPI. What insight does that offer? Honestly, there are a lot of people on HN who just shout that AI is useless because their own jobs are at risk. That's an undeniable fact. In my country, Korea, we say 'like a pheasant hiding its head in the ground.' It means denying reality and lacking self-awareness. I've also seen many failed AI projects due to a lack of proper evaluation of AI capabilities. I don't want to deny that premise. But if you're going to make that argument, you need to describe why they failed. There's none of that here, just this: 'We're going to opt out because it looks like AI projects will fail.' They package it as being cautious, but when you peel back the wrapper, it really means: 'We can't keep up with the AI trend. And we can't tell what works and what doesn't.' Sure, AI has a bubble. No one can deny that. But separate from that, it is useful. Do you really think millions of consumers would stick around for something useless? Social problems are arising, and people are losing jobs because of it. Let's think the other way around. It doesn't make sense for people to lose jobs over something that's truly useless and wrong. People are losing jobs because it's useful and valuable. So wouldn't it be more accurate to say that the problem is 'over-application' rather than the thing itself being bad? In other words, we could say that it's being valued more than it's worth. But this argument just says it's bad, period, with no nuance. I can't trust someone who talks like that. AI writing, if you give it a red-team prompt, can at least refine the surface of counterarguments to your claim. But this—from overgeneralization to extremes—is a textbook example of bad human writing

by jdw64 3mo ago

robwilliams 3mo ago

The article is not a technical overview of AI's failures, it's about the organizational failures that lead to things like token leaderboards. I would argue that this article goes far beyond the "obvious" fact of token leaderboards and into the deeper problems faced by both vendors and companies trying to measure the efficacy of AI. How would AI writing help in any way?

jdw64 3mo ago
Honestly, it's just shouting into an echo chamber. They call it organizational failure, but it's really just 'come on, my side.' A consultant who does that is untrustworthy. It means they have no metrics. Same goes for their 0% claim. It's overgeneralization, isn't it? So why did they fail, what went wrong, and how should these AI projects be analyzed—and by what metrics? They skip all that and just throw out a single KPI. What insight does that offer? Honestly, there are a lot of people on HN who just shout that AI is useless because their own jobs are at risk. That's an undeniable fact. In my country, Korea, we say 'like a pheasant hiding its head in the ground.' It means denying reality and lacking self-awareness. I've also seen many failed AI projects due to a lack of proper evaluation of AI capabilities. I don't want to deny that premise. But if you're going to make that argument, you need to describe why they failed. There's none of that here, just this: 'We're going to opt out because it looks like AI projects will fail.' They package it as being cautious, but when you peel back the wrapper, it really means: 'We can't keep up with the AI trend. And we can't tell what works and what doesn't.' Sure, AI has a bubble. No one can deny that. But separate from that, it is useful. Do you really think millions of consumers would stick around for something useless? Social problems are arising, and people are losing jobs because of it. Let's think the other way around. It doesn't make sense for people to lose jobs over something that's truly useless and wrong. People are losing jobs because it's useful and valuable. So wouldn't it be more accurate to say that the problem is 'over-application' rather than the thing itself being bad? In other words, we could say that it's being valued more than it's worth. But this argument just says it's bad, period, with no nuance. I can't trust someone who talks like that. AI writing, if you give it a red-team prompt, can at least refine the surface of counterarguments to your claim. But this—from overgeneralization to extremes—is a textbook example of bad human writing