4 ms·
It's amazing that it took months to figure this out. "Well we thought that if engineers are told to maximize costs through AI use, to consume as much as possibl
by chihuahua 4mo ago
It's amazing that it took months to figure this out. "Well we thought that if engineers are told to maximize costs through AI use, to consume as much as possible of a resource that costs us money, then obviously good things will happen. Imagine my surprise when it didn't turn out that way."
Imagine if engineers were ranked based on their AWS spend. People allocate VMs and fill databases with terabytes of random bits, to get to the top of the AWS leaderboard. If you don't do this, you're ranked at the bottom, and good luck at the next review cycle. Who could have expected that this is not the road to success?
- solenoid0937 4mo agoYou say "amazing that it took months to figure this out" as if the answer to the question is obvious. But it's not. Some FAANGs are doing amazing things with unlimited tokens. Other companies have no clue what to do with tokens, they've just told their engineers to max them. It really depends on how you're using the tokens. If you're just using them for Codex and Claude Code - yeah, tokenmaxxing is incredibly dumb.
- steveBK123 4mo ago> Some FAANGs are doing amazing things with unlimited tokens. Others have no clue what to do with tokens. Unlimited tokens is different from “use AI a lot or we will fire you, and we are counting token consumption as usage”. Obviously the latter is stupid and yet it was done in many places.
- deleted 4mo ago[deleted]
- SpicyLemonZest 4mo agoI'm not convinced it actually was done in many places, although I understand why in a bad job market people don't trust that it isn't happening in secret. Every time I've heard of a token leaderboard or such it's come with a denial that the company is using it as an employee performance metric.
- steveBK123 4mo ago> it's come with a denial that the company is using it as an employee performance metric. Surely the anonymous employee feedback polls are totally anonymous too. BigCorp loves you, is family, and would never harm you.
- morpheuskafka 4mo ago> But it's not. Some FAANGs are doing amazing things with unlimited tokens Giving someone unlimited access to a resources is not the same as directing or incentivizing them to use it for the sake of using it which is what the parent comment criticized. As for the other FAANGs, Meta and Google have (not good but still) frontier models of their own, so they are very different from a company paying API costs per token.
- SecretDreams 4mo agoShow me some fang that have made nice outwards facing products through a fully embraced AI workflow? AI is an accelerator that engineers should know and have access to, but it's not something that should have mandated usage and quotas around. It's also absolutely dangerous for young engineers and the like - it fundamentally denies you of the "learning" aspect. I'm now seeing in interviews young graduates being given AI tasks to complete and they come back with a correct solution and no concept of how it is working. You learn and reinforce learning by DOING and reading in depth. High level summaries don't teach anything and are the kinds of things only VPs care about. So, unless the intention in the future is for everyone to be a VP using AI to do the work, we need some middle ground here and some real thought around implementation of these tools or there's going to be a generational canyon gap of knowledge between being able to "say" and being able to "do".
- fsloth 4mo ago> Some FAANGs are doing amazing things with unlimited tokens. Would love to know what things!
- arkadiytehgraet 4mo agoOP (solenoid0937) is an unfounded AI-hype peddler and an Anthropic shill (check their comment history), do not expect them to provide an actual example of their wild claims.
- dgellow 4mo agoI checked their history and they seem alright ¯\_(ツ)_/¯
- deleted 4mo ago[deleted]
- arkadiytehgraet 4mo agoHere, just a few examples over the last 9 days, in no particular order of exaggeration magnitude: Unsubstantiated claims of 3x-10x productivity at allegedly a FAANG: https://news.ycombinator.com/item?id=48174666 https://news.ycombinator.com/item?id=48174666 More claims of the same, still unsubstantiated: https://news.ycombinator.com/item?id=48173995 https://news.ycombinator.com/item?id=48173995 Even more claims of the same, still nothing of substance: https://news.ycombinator.com/item?id=48173975 https://news.ycombinator.com/item?id=48173975 Outlandish claim of using 300$ in tokens in 1 hour at allegedly a FAANG: https://news.ycombinator.com/item?id=48158438 https://news.ycombinator.com/item?id=48158438 Ostensibly great things with unlimited tokens at allegedly a FAANG: https://news.ycombinator.com/item?id=48269185 https://news.ycombinator.com/item?id=48269185 Complaints about HN "not getting" AI: https://news.ycombinator.com/item?id=48245673 https://news.ycombinator.com/item?id=48245673 An attempt to validate vibe-coding in production: https://news.ycombinator.com/item?id=48243651 https://news.ycombinator.com/item?id=48243651 Whitewashing Claude degradation: https://news.ycombinator.com/item?id=48211638 https://news.ycombinator.com/item?id=48211638 More Claude degradation whitewashing: https://news.ycombinator.com/item?id=48245784 https://news.ycombinator.com/item?id=48245784 Whitewashing Anthropic: https://news.ycombinator.com/item?id=48199951 https://news.ycombinator.com/item?id=48199951 Promises of coming AI revolution: https://news.ycombinator.com/item?id=48188104 https://news.ycombinator.com/item?id=48188104 All of this serves to hype up the LLM technology with absolutely outlandish claims, while also propping up Anthropic online.
- saghm 4mo agoIn other words, people who are productive get more done when you scale up what they're already doing, and people who aren't productive will not magically become productive when you scale up what they're already doing. That's incredibly obvious, because we've seen how this plays out repeatedly in so many different ways (lines of code, commits, tickets closed, etc.), and it has nothing to do with tokens or even programming, but just how trying to manage people works.
- dgellow 4mo agoWhere can I see those amazing things done by FAANGs?
- solenoid0937 4mo agoJoin one!
- dgellow 4mo agoAnything more concrete?
- arkadiytehgraet 4mo agoYou got baited by an LLM-hype peddler and Anthropic shill, do not expect anything of substance. See https://news.ycombinator.com/item?id=48272695 https://news.ycombinator.com/item?id=48272695
- roxolotl 4mo agoThe inability of leaders to understand Goodhart’s Law is always a sight to behold. They see a number go up and pat themselves on the back for how well their employees are making it go up without ever wondering if the thing they care about is happening.
- pmg101 4mo agoThis is one explanation, sure. Isn't it more likely that they simply don't in fact care about the "thing they care about", only the metric? They can plot the metric on a chart and receive praise, so that's what they're interested in.
- roxolotl 4mo agoThat’s an even worse characterization of them isn’t it? They don’t even care about the end result just the metric. That would take them from clueless to malicious.
- pmg101 4mo agoI wouldn't describe it as malice. If your job is to make the line go up, you make the line go up, and are rewarded for doing so, then you have done your job.
- this_user 4mo agoThe point of this was always to explore what is possible with AI as quickly as possible. Obviously, there is going to be a lot of waste, but the 5-10% of employees who are truly thinking about it and discovering novel applications are what you are truly after. Because right now, you effectively have a giant, as of yet poorly explored space of potential uses. Anyone who can find the actually valuable portions of the space early has a potentially huge competitive advantage. Even if the result of the experiment is the negative that AI is actually mostly not that useful, that is still extremely useful information in a time of great uncertainty regarding outcomes. The bottom line is that this approach may be expensive, but if you have the money to burn, it's far from the worst strategy if you are trying to position yourself correctly for the future.
- adrianN 4mo agoWhat’s the huge advantage though? Adopting workflows that give big productivity gains is relatively easy even for big corporations. It’s only an advantage if you can keep it secret. OTOH maybe we’re in for a future of patenting prompts.
- uejfiweun 4mo agoThe thing I don't get though, is that most people just don't have that much work they need to do. I can use AI to pretty easily get my work done just via the regular chat interfaces. But because of the tokenmaxxing metrics that leadership tracks, I end up just having the AI deliberate for hours on random things just so that I can boost my token numbers. I think tokenmaxxing for the end goal you described is only realistic when the engineers are truly buried under a backlog of work.
- smoofles 4mo agoNot being buried under a backlog of work is one aspect, and the other is that the sheer _urgency_ of these efforts makes it look like companies like Uber could be displaced in a year or two by someone who gets lucky with AI use. Which absolutely isn’t the case. Even if someone would manage to overtake a market leader on tech merit alone, within 1-2 years, thanks to AI, markets don’t swing on such short notices. The fake urgency is absolutely psychotic.
- saghm 4mo agoSomeday maybe Goodhart's Law will be intuitive to people making decisions like this, but not any time soon I guess
- dgellow 4mo ago> It's amazing that it took months to figure this out We aren’t there yet, so far it is just a COO questioning the investment
- davnicwil 4mo agoI think unfortunately it's not about what seems obvious, or even what seems more likely, but about what seems retrospectively justifiable regardless of outcome. The incentive structure of this type of decision is 'absolutely under no circumstances existentially mess up'. Ostensibly with respect to the organisation, but in actual reality much more so with respect to the individual(s) involved in the decision. If everyone else is doing something that kind of obviously makes no sense, and you decide to break from the crowd by instead doing what does make sense, then there's a pretty solid chance of gaining a temporary edge while reality resolves the truth. But those gains probably won't matter all that much for the organisation, or indeed your position within it. It's a solid chance of an unimportant gain. However on the other hand, there's a tail risk that something very unexpected happens and the thing everyone's doing that makes no sense actually turns out to make sense - sometimes even for entirely unpredictable incidental reasons - and then, well, you're in trouble. Not necessarily 'you' the organisation.. they'll likely be able to catch up and it won't matter that much. But for 'you' personally, the decision maker, it's very much not good. As a bonus, in the much more likely scenario that the thing that makes no sense turns out to indeed make no sense, you're in the same boat as everyone else, there's no relative loss, and most importantly you don't stick out as someone who did something as risky as to go against the prevailing, albeit pretty clearly nonsensical, sentiment. So basically, game theory tells you pretty quickly to just go with the thing that makes no sense if you're optimising for some (weighted) cross of what's best for the organisation and yourself as the decision maker.