> 12 months ago, if I fed a list of ~800 poems with about ~250k tokens to an LLM and asked it to summarize this huge collection, they would be completely blind

> 12 months ago, if I fed a list of ~800 poems with about ~250k tokens to an LLM and asked it to summarize this huge collection, they would be completely blind to some poems and were prone to hallucinating not simply verses but full-blown poems. for the past week claude code has been routinely ignoring CLAUDE.md and every single instruction in it. I have to manually prompt it every time. As I was vibe coding the notes MCP mentioned in the article [1] I was also testing it with claude. At one point it just forgot that MCPs exist. It was literally this: > add note to mcp Calling mcp:add_note_to_project > add note to mcp Running find mcp.ex ... Interrupted by user ... > add note to mcp Running <convoluted code generation command with mcp in it> We have no objective way of measuring performance and behavior of LLMs [1] https://github.com/dmitriid/mcp_notes https://github.com/dmitriid/mcp_notes