SparkyMcUnicorn

10 ms·

91.
by SparkyMcUnicorn 1y ago
Gemini 2.5 Pro is the highest ranking model on the aider benchmarks leaderboard. For benchmarks, either Gemini writes code that adheres to the required edit format, builds successfully, and passes unit tests, or it doesn't. I primarily
92.
by SparkyMcUnicorn 1y ago
My link is to the benchmark results _over time_. The main leaderboard page that you linked to is updated quite frequently, but it doesn't contain multiple benchmarks for the same exact model.
93.
by SparkyMcUnicorn 1y ago
Aider has one, but it hasn't been updated in months. People kept claiming models were getting worse, but the results proved that they weren't.
94.
by SparkyMcUnicorn 1y ago
The Aider discord community has proposed and disproven the theory that 2.5 Pro became worse, several times, through many benchmark runs. It had a few bugs here or there when they pushed updates, but it didn't get worse.
95.
by SparkyMcUnicorn 1y ago
It's a bit dated now, but it would be cool if people submitted PRs for this one: https://aider.chat/docs/leaderboards/by-release-date.html
96.
by SparkyMcUnicorn 1y ago
30-A3B is a really impressive model. I throw tasks at it running locally to save on API costs, and it's possibly better than anything we had a year or so ago from closed source providers. For programming tasks, I'd rank it higher
97.
by SparkyMcUnicorn 1y ago
My guess is that Orbstack might switch to using this, and it'll just be a more competitive space with better open source options popping up. People still want the nice UI/UX, and this is just a Swift package.
98.
by SparkyMcUnicorn 1y ago
Just close the loop and give it direct access to your console logs in chrome and node, then it can do the "full manual debugging" on its own. It's not perfect, and it's not exactly cheap, but it works.
99.
by SparkyMcUnicorn 1y ago
Install Roo Code (or Cline) in VS Code, plug in your API key(s), and start using them. It's that simple.
100.
by SparkyMcUnicorn 1y ago
My most recent flow is very similar, but I use AiderDesk[0] instead of Prompt Tower for easier creation/editing of plan files. AiderDesk lets you save snapshots of a point in time, so I create "presets" to restore sets of con
101.
by SparkyMcUnicorn 1y ago
Definitely concerning, although I'm having trouble finding anything in the codebase to support this. This paper even seems to contradict aspects of the project's no tracking stance. If someone told me this paper was for a differen
102.
by SparkyMcUnicorn 1y ago
Packages can get hijacked too.
103.
by SparkyMcUnicorn 1y ago
100% sure. Tested in the Anthropic workbench[0] to double check and got the same result. The web interface has a prompt that defines a cutoff date and who's president[1]. [0] https://console.anthropic.com/workbench [1]
104.
by SparkyMcUnicorn 1y ago
Interesting. It's claiming different knowledge cutoff dates depending on the question asked. "Who is president?" gives a "April 2024" date.
105.
by SparkyMcUnicorn 1y ago
Although I believe it, I wish there was some observability into what data is included here. Both Sonnet and Opus 4 say Joe Biden is president and claim their knowledge cutoff is "April 2024".
106.
by SparkyMcUnicorn 1y ago
Yep! I'm aware of the tradeoffs here. For home assistant specifically, there's two options if you want to stay on the path of first-class support. Run it bare metal or in a VM. Going a different path isn't a bad choice, or ev
107.
by SparkyMcUnicorn 1y ago
One-click updates work from the dashboard? I don't think they would. Of course you don't have to go the proxmox route, but it is an easy route. I have a proxmox cluster, so moving things between machines, high availability, and ba
108.
by SparkyMcUnicorn 1y ago
Proxmox adds very little overhead. I'm running dozens of things alongside HAOS. The OS is the path of least resistance and gives you the best experience for low maintenance. https://community-scripts.github.io/ProxmoxVE
109.
by SparkyMcUnicorn 1y ago
If you choose your hardware wisely, it's incredibly reliable. Most of my house is automated and I all I've done with it in the past year is add a few devices. I will admit that low (or zero) maintenance is easier to accomplish if
110.
by SparkyMcUnicorn 1y ago
I did this a few months ago for a test home assistant setup and had the opposite experience. I forget exactly what I did, but it involved editing a file with the wifi settings and dropping my public ssh key on the SD card before installatio
111.
by SparkyMcUnicorn 1y ago
You have that backwards. The leaderboard results have the thinking model as the architect. In this case, o3 is the architect and 4.1 is the editor.
112.
by SparkyMcUnicorn 1y ago
It's also missing the documentation part. Without additional context, method/type definitions with a short description will only go so far. Cherry picking a tiny example, this wouldn't capture the fact that cloudflare durable
113.
by SparkyMcUnicorn 1y ago
MLC[0] indicates that it can run models in the 8B range on iOS, but 1-3B sounds more reasonable to me. [0] https://llm.mlc.ai/docs/deploy/ios.html#bring-your-own-model
114.
by SparkyMcUnicorn 1y ago
Maybe because it's not exactly NAT64, even though it has the same goal?
115.
by SparkyMcUnicorn 1y ago
Anyone know of any coding agents that support video inputs? Web chat interfaces are great, but copy/paste gets old fast.
116.
by SparkyMcUnicorn 1y ago
I ran them again several times to make sure the results were fair. My previous runs also had a different 30B model loaded in the background that I forgot about. LM Studio is an easy way to use both mlx and llama.cpp anemll [0]: ~9.3 tok&#x2
117.
by SparkyMcUnicorn 1y ago
M4 Max with 128GB of memory.
118.
by SparkyMcUnicorn 1y ago
The 1B model works on iPhones[0]. See my other comments. anemll appears to use less memory. [0] https://huggingface.co/anemll/anemll-llama-3.2-1B-iOSv2.0
119.
by SparkyMcUnicorn 1y ago
See my other comment for results. mlx is much faster, but anemll appeared to use only 500MB of memory compared to the 8GB mlx used.
120.
by SparkyMcUnicorn 1y ago
I ran R1-8B for both anemll[0] and mlx[1][2] models on an M4 Max. Prompt: "Tell me a long story about the origins of 42 being the answer." anemll: 9.3 tok/sec, ~500MB of memory used. mlx 8bit: 31.33 tok/sec, ~8.5GB of me

More ›