jermaustin1

4 ms·

31.
by jermaustin1 1mo ago
I've had some rough time getting LM-Studio properly configured for multi-card. It exists, but I feel like it is kind of buggy. I will disable a card and it will still load the model into it. Sometimes it will split the model even thoug
32.
by jermaustin1 1mo ago
Because concurrency with a single "accessible" card quickly diminishes. I have dual 3090s, and on Qwen 3.6 35B A3B at 80k a single card with max concurrent set to 4 will get 70-80tps single request, 50-60 TOTAL tps with 2, 45-50 w
33.
by jermaustin1 1mo ago
Same. I have bad "factual" memory, but very good "conceptual" memory. I might not remember exactly what someone told me, but I probably gathered a keyword, and a feeling and idea of what they told me. Almost like it take
34.
by jermaustin1 1mo ago
> I understand that such models can be used by malicious actors, but it’s fair to have it publicly available I feel like there should be some mechanism to prove you own the code/app/site/whatever and it will remove the gua
35.
by jermaustin1 2mo ago
But you most likely have a passport, and most (western/european) countries have "automatic" visas to the UK, so for the most part, training from London to Edinburgh isn't much easier than Paris to London. Sure there is n
36.
by jermaustin1 2mo ago
As a songwriter, drum hitter, and acoustic strummer, I am not good at drums (can play on beat, though), guitar (can strum all the cowboy chords, though), or singing (no caveat on this one). So sometimes when I want to hear what my song COUL
37.
by jermaustin1 2mo ago
Somewhat hand rolled, somewhat claude coded. Back in 2023 I started my own C# LLM library for doing tool calls and structured output, and over the years it has morphed bigger and bigger, and that is the backbone of almost all of my LLM-base
38.
by jermaustin1 2mo ago
I'll go over my repo, and see if it is hiding any API keys and maybe make it public. The issue I have is it relies on a nuget package that also isn't live (its in my local nuget feed). I'm not sure what all is needed to make
39.
by jermaustin1 2mo ago
It is on a single 3090, and that seems to be where it averages out. I'll get 85tps on turn 0, but then it settles down to low 70s within a few turns, but holds steady at that. My issue currently is KV Cache, because I can't keep e
40.
by jermaustin1 2mo ago
I'd love an HBO "Succession" style dramatization of this. I've always found family businesses so interesting. My first job was working directly for a perpetually "near-retirement" 70 year old man, that never st
41.
by jermaustin1 2mo ago
I've been using Qwen3.6 35B A3B, and with reasoning turned on, I'd say 2/3 (give or take) of the tokens for a response are thinking tokens. Which at 70+ tps locally, that isn't that awful. I run an 80k context across 4-1
42.
by jermaustin1 2mo ago
I liked this. I agreed with most of the suggestions (primarily because I knew them already), so I know the "algorithm" is working. There were a couple of groups I had never heard before, so double success!
43.
by jermaustin1 2mo ago
And I do `docker compose up` now with a caddy server in front of docker, so I can just add a new site (sub domain or directory) that reverse proxies to that docker container's port. And the beauty is, that I have a skill for it for cla
44.
by jermaustin1 2mo ago
I have found that for some personal prose-related projects, QWEN 3.6 35B A3B is an amazing model even quantized down to 4 bits. I actually find it's "writing" style as a GM for an LLM-powered solo text adventure game, better
45.
by jermaustin1 2mo ago
> Pax Americana could've gone on forever. Hubris is the undoing of all empires. And hubris is basically entropy for humanity.
46.
by jermaustin1 2mo ago
My guy, your account is nearly 10 years old, but your karma is only 36, and you are active on here... there is a reason. You are derisive, and abrasive, and your comments (specifically the ones that get downvoted to hell) are low-effort, an
47.
by jermaustin1 2mo ago
> it's less clear why those are so much more expensive now. It's basically, because everything else has gotten more expensive. Labor + fuel + wear and tear on equipment + raw materials. Inflation of a single thing, will have ef
48.
by jermaustin1 2mo ago
So we had our central HVAC burn up, and I bought a mini-split heat pump to replace it, because the HVAC companies around me don't do mini-splits. Apparently they also don't just remove old systems unless they are replacing them. S
49.
by jermaustin1 2mo ago
There are a few Data Centers going up within a couple hours of me, it has pulled all of the trades people away from us. I've called around to every electrician I can find online, or been recommended by neighbors, they are all out of to
50.
by jermaustin1 2mo ago
Sorry, I was doing conservative napkin math, better safe than burnt to a crisp, as grandpa NEVER said... I'm currently having to rewire a lot of the house that he built and I bought so he could retire. There have been a lot of electric
51.
by jermaustin1 2mo ago
That's why they said "depending on country" In North America we are on 120V, making a standard 15A outlet only 1500W max, and something like 1200W sustained. To use higher wattage appliances, we have to upgrade our outlets to
52.
by jermaustin1 2mo ago
> many books are pretty low quality. Even low-quality books have words people might not know. I often find people who don't read books (physical or eBook) have a much lower vocabulary, and they typically don't value vocabulary,
53.
by jermaustin1 2mo ago
Texans (generally) consider themselves Texans firsts. Similar to born and raised New Yorkers (NYC). That said, I really meant it tongue-in-cheek that my accent was too thick, and doesn't work well for Siri's STT model. ChatGPT&#x2
54.
by jermaustin1 3mo ago
As a Texan first, American second, I sympathize with this statement. Siri can't understand me probably 25% of the time. I use STT for iMessage while in the car, and half the time it will take 3+ times to either get it right or me give
55.
by jermaustin1 3mo ago
There are properties on the API call you can pass for specific providers, so you test which providers you like the output, then add them to the list in ranked order if you want one by default, then to fall back to the other. There might be
56.
by jermaustin1 3mo ago
More than 3 decades. The original world wide web had them. Link "circles"[1] fell out of favor for SEO around 2015 or so, but before that, they had been a primary driver of Page Rank forever. I worked in SEO from 2008 until 2015,
57.
by jermaustin1 3mo ago
> You'd see a lot more 1200sqft new builds if it was more economically viable and developers could make a profit on a $125k home instead of needing $400k. You do see them, but they are factory built. I could have a $75k manufactured
58.
by jermaustin1 3mo ago
> without a strong guarantee you'd get it all back? Mortgages are backed by the asset you are buying. $400k for a house is backed by the house. On top of that, almost every bank can sell off those mortgages and get the cash back imm
59.
by jermaustin1 3mo ago
Even if new stock were built, new homes in Houston start around $300k for a 3/2 configuration, but those are bought up nearly instantly because not many are built, and the remaining stock are $400-700k. I grew up with 6 family members
60.
by jermaustin1 3mo ago
I have 2 autoimmune disorders/diseases, and I have spent the last 2 decades managing them. One of them (thyroid) is relatively easy, take my synthetic thyroid hormones ever day, and check my levels every 3 months to make sure I'm

More ›