3 ms·
Say I want to ask a model about a code base 500k tokens in size. The questions would not be to generate code, but rather to help understand it in a way that req
by WhitneyLand 2y ago
Say I want to ask a model about a code base 500k tokens in size. The questions would not be to generate code, but rather to help understand it in a way that require the model to globally reason about everything at once.
How might performance compare between:
1.) Using a model like Gemini to load it all into context at once
2.) Using one of the various summarization systems/embedding/RAG etc.
3.) Fine tuning the whole code base into Gpt-4o
- spullara 2y ago1 then 2. Never 3.
- WhitneyLand 2y agoInteresting why would you say that? There’s not been much opportunity previously to easily fine tune a Gpt 4 class model. I haven’t seen anything written up on this being tried.
- Liquix 2y agotools like aider [0] maintain a large context with minimal token usage by reading a repo's .git folder. impressive results [0] https://github.com/paul-gauthier/aider https://github.com/paul-gauthier/aider
- trees101 2y agoalso, aider has added experimental support for prompt caching: https://github.com/paul-gauthier/aider/issues/1086#issuecomment-2297256228 https://github.com/paul-gauthier/aider/issues/1086#issuecomm... I really like the control you get with aider over the LLM context. You can /add or /drop source code, markdown notes. You can /clear the chat. /tokens shows you the context and the cost, you can see what each prompt will cost you. I find aider best used in conjunction with a git diff view in VSCode, I run aider with --no-auto-commits and then manually review each time in VSCode. I'm keen to learn any AI coding workflows if anyone has any links. I've benefitted greatly from tips such as using type hints and documentation for the LLM's benefit.
- machiaweliczny 2y agoI have 5 lines shell to load git into single file and using google AI studio. Tested it to simplify dreamcoder implementation but results are ~ so far