Time to try my 100k token reality check test: Here is a tarball of a golang repository. Please add a module that does XXX. Claude 1 did not like this request.

Time to try my 100k token reality check test: Here is a tarball of a golang repository. Please add a module that does XXX. Claude 1 did not like this request. Depending on how much they've improved the attention layer, this seems to me like right in the sweet spot for a serious LLM user -- if the LLM can grok a codebase, and scaffold even to 50%, imputing along the way the style guide, the architecture and so on, it's a huge win. GPT-4 in particular has been relatively good at getting styles and architecture right, and code gen for smaller projects is really very good. It is not successful at reading tar files, but it can be fed source code bit by bit. It may be my own hallucinations, but I find it slightly less capable at remembering longer conversations / code listings than I did when it first launched.