5 ms·
I hope people realize that tools like caveman are mostly joke/prank projects - almost the entirety of the context spent is in file reads (for input) and reasoni
by Tiberium 6mo ago
I hope people realize that tools like caveman are mostly joke/prank projects - almost the entirety of the context spent is in file reads (for input) and reasoning (in output), you will barely save even 1% with such a tool, and might actually confuse the model more or have it reason for more tokens because it'll have to formulate its respone in the way that satisfies the requirements.
- make3 6mo agoI wonder if you can have it reason in caveman
- 0123456789ABCDE 6mo agowould you be surprised if this is what happens when you ask it to write like one? folks could have just asked for _austere reasoning notes_ instead of "write like you suffer from arrested development"
- Sohcahtoa82 6mo ago> "write like you suffer from arrested development" My first thought was that this would mean that my life is being narrated by Ron Howard.
- acedTrex 6mo agoYou really think the 33k people that starred a 40 line markdown file realize that?
- verdverm 6mo agoStars are more akin to bookmarks and likes these days, as opposed to a show of support or "I use this"
- zbrozek 6mo agoI use them like bookmarks.
- LPisGood 6mo agoI use them as likes
- giraffe_lady 6mo agoI intentionally throw some weird ones on there just in case anyone is actually ever checking them. Gotta keep interviewers guessing.
- andersa 6mo agoYou mean the 33k bots that created a nearly linear stars/day graph? There's a dip in the middle, but it was very blatant at the start (and now)
- pdntspa 6mo ago[flagged]
- embedding-shape 6mo ago> I hope people realize that tools like caveman are mostly joke/prank projects This seems to be a common thread in the LLM ecosystem; someone starts a project for shits and giggles, makes it public, most people get the joke, others think it's serious, author eventually tries to turn the joke project into a VC-funded business, some people are standing watching with the jaws open, the world moves on.
- simonw 6mo agoI was convinced https://github.com/memvid/memvid https://github.com/memvid/memvid was a joke until it turned out it wasn't.
- embedding-shape 6mo agoTo be fair, most of us looked at GPT1 and GPT2 as fun and unserious jokes, until it started putting together sentences that actually read like real text, I remember laughing with a group of friends about some early generated texts. Little did we know.
- Alifatisk 6mo agoAre there any public records I can see from GPT1 and GPT2 output and how it was marketed?
- walthamstow 6mo agoI don't think it was marketed as such, they were research projects. GPT-3 was the first to be sold via API
- deleted 6mo ago[deleted]
- embedding-shape 6mo agoHN submissions have a bunch of examples in them, but worth remembering they were released as "Look at this somewhat cool and potentially useful stuff" rather than what we see today, LLMs marketed as tools. https://news.ycombinator.com/item?id=21454273 https://news.ycombinator.com/item?id=21454273 / https://news.ycombinator.com/item?id=19830042 https://news.ycombinator.com/item?id=19830042 - OpenAI Releases Largest GPT-2 Text Generation Model HN search for GPT between 2018-2020, lots of results, lots of discussions: https://hn.algolia.com/?dateEnd=1577836800&dateRange=custom&dateStart=1514764800&page=0&prefix=false&query=GPT&sort=byPopularity&type=story https://hn.algolia.com/?dateEnd=1577836800&dateRange=custom&...
- egorfine 6mo agoThey are indeed impractical in agentic coding. However in deep research-like products you can have a pass with LLM to compress web page text into caveman speak, thus hugely compressing tokens.
- claytongulick 6mo agoI don't understand how this would work without a huge loss in resolution or "cognitive" ability. Prediction works based on the attention mechanism, and current humans don't speak like cavemen - so how could you expect a useful token chain from data that isn't trained on speech like that? I get the concept of transformers, but this isn't doing a 1:1 transform from english to french or whatever, you're fundamentally unable to represent certain concepts effectively in caveman etc... or am I missing something?
- egorfine 6mo agoGood catch actually. Okay maybe not exactly caveman dialect, but text compression using LLM is definitely possible to save on tokens in deep research.
- ieie3366 6mo agoAll LLMs also effectively work by ”larping” a role. You steer it towards larping a caveman and well.. let’s just say they weren’t known for their high iq
- DiogenesKynikos 6mo agoThis is why ancient Chinese scholar mode (also extremely terse) is better.
- Hikikomori 6mo agoModern humans were also cavemen.
- roughly 6mo agoFun fact: Neanderthals actually had larger brains than Homo Sapiens! Modern humans are thought to have outcompeted them by working better together in larger groups, but in terms of actual individual intelligence, Neanderthals may have had us beat. Similarly, humans have been undergoing a process of self-domestication over the last couple millenia that have resulted in physiological changes that include a smaller brain size - again, our advantage over our wilder forebearers remains that we're better in larger social groups than they were and are better at shared symbolic reasoning and synchronized activity, not necessarily that our brains are more capable. (No, none of this changes that if you make an LLM larp a caveman it's gonna act stupid, you're right about that.)
- adwn 6mo agoI thought we were way past the "bigger brain means more intelligence" stage of neuroscience?
- nomel 6mo agoAll data shows there's a moderate correlation.
- waffletower 6mo agoEven neuronal density is simplistic, and the dimension of size alone doesn't consider that.
- stingraycharles 6mo agoWhile the caveman stuff is obviously not serious, there is a lot of legit research in this area. Which means yes, you can actually influence this quite a bit. Read the paper “Compressed Chain of Thought” for example, it shows it’s really easy to make significant reductions in reasoning tokens without affecting output quality. There is not too much research into this (about 5 papers in total), but with that it’s possible to reduce output tokens by about 60%. Given that output is an incredibly significant part of the total costs, this is important. https://arxiv.org/abs/2412.13171 https://arxiv.org/abs/2412.13171
- ACCount37 6mo agoSome labs do it internally because RLVR is very token-expensive. But it degrades CoT readability even more than normal RL pressure does. It isn't free either - by default, models learn to offload some of their internal computation into the "filler" tokens. So reducing raw token count always cuts into reasoning capacity somewhat. Getting closer to "compute optimal" while reducing token use isn't an easy task.
- stingraycharles 6mo agoYeah the readability suffers, but as long as the actual output (ie the non-CoT part) stays unaffected it’s reasonably fine. I work on a few agentic open source tools and the interesting thing is that once I implemented these things, the overall feedback was a performance improvement rather than performance reduction, as the LLM would spend much less time on generating tokens. I didn’t implement it fully, just a few basic things like “reduce prose while thinking, don’t repeat your thoughts” etc would already yield massive improvements.
- AdamN 6mo agoYeah you could easily imagine stenography like inputs and outputs for rapid iteration loops. It's also true that in social media people already want faster-to-read snippets that drop grammar so the desire for density is already there for human authors/readers.
- altruios 6mo ago
- bensyverson 6mo agoExactly. The model is exquisitely sensitive to language. The idea that you would encourage it to think like a caveman to save a few tokens is hilarious but extremely counter-productive if you care about the quality of its reasoning.
- andai 6mo agoDoes this imply that if you train it on Gwern style output, the quality will improve?
- gwern 6mo agoUnfortunately, that is an oversimplification for a highly RLed/chatbot trained LLM like Claude-4.7-opus. It may have started life as a base model (where prompting it with correctly spelled prompts, or text from 'gwern', would - and did with davinci GPT-3! - improve quality), but that was eons ago. The chatbots are largely invariant to that kind of prompt trickery, and just try to do their best every time. This is why those meme tricks about tips or bribery or my-grandmother-will-die stop working.
- deleted 6mo ago[deleted]
- Waterluvian 6mo agoHelp me understand: I get that the file reading can be a lot. But I also expand the box to see its “reasoning” and there’s a ton of natural language going on there.
- reacharavindh 6mo agoThis specific form may be a joke, but token conscious work is becoming more and more relevant.. Look at https://github.com/AgusRdz/chop https://github.com/AgusRdz/chop And https://github.com/toon-format/toon https://github.com/toon-format/toon
- alex7o 6mo agoAlso https://github.com/rtk-ai/rtk https://github.com/rtk-ai/rtk but some people see that changing how commands output stuff can confuse some models
- micromacrofoot 6mo agoI mean we had a shoe company pivot to AI and raise their stock value by 300%, how can we even know anymore
- bombcar 6mo agoLemonade and blockchain rides again! Or was it ice tea?
- addandsubtract 6mo agoWe started out with oobabooga, so caveman is the next logical evolution on the road to AGI.
- causal 6mo agoOutput tokens are more expensive
- sidrag22 6mo agoI hesitated 100% when i saw caveman gaining steam, changing something like this absolutely changes the behaviour of the models responses, simply including like a "lmao" or something casual in any reply will change the tone entirely into a more relaxed style like ya whatever type mode. I think a lot of people echo my same criticism, I would assume that the major LLM providers are the actual winners of that repo getting popular as well, for the same reason you stated. > you will barely save even 1% with such a tool For the end user, this doesnt make a huge impact, in fact it potentially hurts if it means that you are getting less serious replies from the model itself. However as with any minor change across a ton of users, this is significant savings for the providers. I still think just keeping the model capable of easily finding what it needs without having to comb through a lot of files for no reason, is the best current method to save tokens. it takes some upfront tokens potentially if you are delegating that work to the agent to keep those navigation files up to date, but it pays dividends when future sessions your context window is smaller and only the proper portions of the project need to be loaded into that window.
- SEJeff 6mo agoI believe tools like graphify cut down the tokens in thinking dramatically. It makes a knowledge graph and dumps it into markdown that is honestly awesome. Then it has stubs that pretend to be some tools like grep that read from the knowledge graph first so it does less work. Easy to setup and use too. I like it. https://graphify.net/ https://graphify.net/
- sambellll 6mo agoSomeone should make an MCP that parses every non-code file before it hits claude to turn it into caveman talk
- xnx 6mo agoThere's a tremendous amount of superstition around LLMs. Remember when "prompt engineering" "best practices" were to say you were offering a tip or some other nonsense?