claytonjy

9 ms·

91.
by claytonjy 3y ago
I have successfully transitioned an ML/AI team of seasoned researchers away from conda and to poetry. Some also use pyenv, I suspect a lot don't bother but may get bitten eventually. It's definitely a learning curve, but it t
92.
by claytonjy 3y ago
pyenv only manages python versions, while poetry manages dependencies and virtual environments. They are complimentary, but do not overlap.
93.
by claytonjy 3y ago
can you say more about the network issues with thousands of k8s nodes? I'm regularly running 2-3000 nodes in a GKE cluster, majority have GPUs, is this something I need to be worrying about?
94.
by claytonjy 3y ago
That is a satirical twitter account.
95.
by claytonjy 3y ago
Even GPT-4 has been hit or miss for me. I think it's less about the model and more about the data. I'm intentionally aiming to write my home-manager stuff as modern as possible (e.g. flakes, recent home-manager syntax) but most of
96.
by claytonjy 3y ago
Doesn't this work pretty well everywhere else now, too? Torch at least is easy to pip install, as is poetry if you skip all the versions that were mispackaged. It's unclear to me what the benefit of conda is these days, other than
97.
by claytonjy 3y ago
another comment in here mentioned Zipline Reloaded¹, a fork from a guy who wrote a book using Zipline. Last commit 6 months ago, much better than 3 years from the original. ¹: https://github.com/stefan-jansen/zipline-re
98.
by claytonjy 3y ago
is your strategy more profitable than indexing? or are you aiming for something else, like beta-neutrality?
99.
by claytonjy 3y ago
related, has anyone had success moving from Slurm to Kubernetes for a physical (non-cloud) cluster primarily used for training large models on lots of GPUs?
100.
by claytonjy 3y ago
they said they finish work in 1-2 hours
101.
by claytonjy 3y ago
what kind of role are you in and at what kind of company that there is so little for you to do? Do you never meet with coworkers or bosses? I think it's awesome some people have found such a place, more power to them/you, but havi
102.
by claytonjy 3y ago
They offer Obsidian Publish for $8/mo: https://obsidian.md/publish
103.
by claytonjy 3y ago
And even if they do pay, larger companies often insist on net-60 or even net-90...so you have to wait potentially months after delivering work to get paid. For a small or new shop that alone can put you on the ropes, especially since the
104.
by claytonjy 3y ago
huh, I didn't even know openvino supported anything but CPUs! TIL
105.
by claytonjy 3y ago
Anecdotally I've found ctranslate2 to be even faster than insanely-fast-whisper. On an L4, using ctranslate2 with a batch size as low as 4 beats all their benchmarks except the A100 with flash attention 2. It's a shame faster-whis
106.
by claytonjy 3y ago
To have a good comparison I think we'd need to run the insanely-fast-whisper code on a 4090. I bet it handily beats both the benchmarks in OP, though you'll need a much smaller batch size than 24. You can beat these benchmarks on
107.
by claytonjy 3y ago
Can confirm. I worked at a small statistics consultancy where the owner did some expert testimony, and you have to reveal what you charge. The opposing side was often paying 500+/hr for shoddy work! I think we've seen stories of I
108.
by claytonjy 3y ago
One of the notable tricks the various LLM serving frameworks provide is a special approaches to batching, e g. continuous, persistent, or in-flight batching depending on the inference framework. At some level they each allow you to start a
109.
by claytonjy 3y ago
I would not consider metrics explorer to be a particularly good product. For queries where the query builder isn't good enough and you write PromQL instead, they don't even let you alias the legend; instead you must see the entire
110.
by claytonjy 3y ago
Even in the HBO show Succession, these things take a season, not an episode
111.
by claytonjy 3y ago
I definitely believe he can raise a lot of money quickly, but I'm not sure where he'll get the talent, at least the core modeling talent. That's Ilya's lane, and I get the sense that group are the true believers in the o
112.
by claytonjy 3y ago
Metrics, probably? The article calls out logs, metrics, and traces as the 3 pillars of observability.
113.
by claytonjy 3y ago
Their Ultimate plan includes access to various top-tier LLMs for different uses, including GPT4 https://help.kagi.com/kagi/ai/assistant.html
114.
by claytonjy 3y ago
I used DDG for years and often reached for "!g", yet a year or so into using Kagi and I don't think I've ever needed to fall back to Google. Any particular patterns to the searches you prefer Google for?
115.
by claytonjy 3y ago
Definitely poetry; pipenv is not actively maintained and poetry is it's spiritual successor.
116.
by claytonjy 3y ago
I appreciate the pushback and the links! The third chart in the apartment list link ("Homeownership has been in decline for decades") is exactly the plot I wanted. That chart makes the gap not look so dramatic as either your numbe
117.
by claytonjy 3y ago
Aren't age-adjusted homeownership rates roughly the same for millennials as it was for baby boomers? I won't argue that it's harder and more expensive, it sure is, but millennials seem to prioritize it enough to do whatever i
118.
by claytonjy 3y ago
I built an ML pipeline in pachyderm years ago, maybe 2016, and loved using it then. But my data was small-to-medium, and I stuck with CSVs because back then you couldn't get many of the benefits with a binary format. I think it's
119.
by claytonjy 3y ago
which things do you do with HF, and which with WandB? HF seems like "GitHub for models and datasets", it has a cool brand and everyone in ML uses it in some capacity. But when it comes to _company_ needs like private datasets&#x2F
120.
by claytonjy 3y ago
I don't think that is obvious. If your use case demands lowest latency at any cost, you might run batch size 1. I believe replit's new code model (announced about a month ago) runs at batch 1 in prod, for example, because code com

More ›