Palmik

15 ms·

271.
by Palmik 2y ago
How is this different from the Coolify hosted cloud, apart from the fact that you are the co-founder of Coherence and not Coolify. I've used neither solution, but just at a glance, right now I'd bet on Coolify -- it has more permi
272.
by Palmik 2y ago
The article is interesting, but it's not clear how Lago fixes the issue. As far as I know, Lago starts in the low thousands per month. If you go with the self hosted route, you might also just process Stripe's API (backfill using
273.
by Palmik 2y ago
Pricing calculator was easy to find https://www.stackit.de/en/pricing/cloud-services/iaas/stacki... Menu -> Compute Engine -> Pricing
274.
by Palmik 2y ago
Would love to hear how you went about doing things at Framer!
275.
by Palmik 2y ago
Quite shady indeed: https://www.saashub.com/theres-an-ai-alternatives -- see the icon, it's almost the same as https://theresanaiforthat.com/
276.
by Palmik 2y ago
The name is a prefix of the most popular website in the niche: thereisanaiforthat.com. Maybe some shady SEO tactic? There are tons of these. Many of them are quite fishy -- they often reach out to you offering you to list your tool for a fe
277.
by Palmik 2y ago
For a reference, that's at least 40% more than what H100 sxm would cost if you are willing to reserve for a month (so not apples to apples). H100 will also be much faster, especially if you are willing to use fp8. Maybe 3-4x
278.
Google hires CEO and cofounder of Character AI(fortune.com)
1 points by Palmik 2y ago|1 comments
279.
by Palmik 2y ago
I don't think these analogies work. Meta provides open source code to modify the the weights (fine tune the model). In this context, fine-tuning the model is better converted to being able to modify the code of the game.
280.
by Palmik 2y ago
In the LLM world there are many open source solutions to find tuning, maybe the best one being from Meta: https://github.com/pytorch/torchtune In terms of inference and interface (since you mentioned comfy) there are m
281.
by Palmik 2y ago
It's much better than sharing a binary artifact of regular software, since the weights can be and are easily and frequently modified by fine tuning the model. This means you can modify the "binary artifact" to your needs, sim
282.
by Palmik 2y ago
They specifically call out fp8 aware training and TensoRT LLM is really good (efficient) with fp8 inference on H100 and other hopper cards. It's possible that they run the 7b natively in fp16 as smaller models suffer more from even &qu
283.
by Palmik 2y ago
People arguing that this is "just extension" are ignoring the fact that extensions have special priviledges compared to websites, and you would not want all websites to have the full power of arbitrary extension. If it's &quo
284.
Google Search Console not updated for >90 hours(twitter.com)
3 points by Palmik 2y ago|0 comments
285.
by Palmik 2y ago
It has fp8 support. Not sure whether fp8 on MI300x is supported by vLLM yet. Also, many of these comparisons use vLLM for both setups, but for Nvidia you can and should use TensorRT-LLM which tends to do quite a bit better than vLLM at high
286.
Notion Sites(notion.so)
1 points by Palmik 2y ago|0 comments
287.
by Palmik 2y ago
Congrats on the traffic, I hope you get a high conversion rate :) I assume you do more than one GPT call per search, e.g. rewriting user's query into multiple search queries, summarization of the results, etc.?
288.
by Palmik 2y ago
It would be great to have real world inference benchmarks for LLMs. These aren't it. That means e.g. 8xH100 with TensorRT-LLM / vLLM vs 8xMI300X with vLLM running many concurrent requests with reasonable # of input and output toke
289.
by Palmik 2y ago
Google Search is ad-backed site. Especially for highly commercial queries. They just prefer internet users to consume their ads, rather than the ads of the content creators.
290.
by Palmik 2y ago
I don't think it's that low. I use neovim embedded in vscode as do many others: https://github.com/vscode-neovim/vscode-neovim
291.
by Palmik 2y ago
Speculative decoding requires you to load the smaller model into memory and run inference on it.
292.
by Palmik 2y ago
RunPod is one of the providers that has some of the best on demand pricing (there are places where you can get H100 PCIe for less, but they typically don't have any capacity) without contracts and doesn't own hardware (or didn&#x2
293.
by Palmik 2y ago
MI250 and MI300 are competitive for inference and training. The problem is that there's no cloud with simple registration that offers these for reasonable on demand price. This means that basically no one in the open source AI communit
294.
by Palmik 2y ago
Are there any plans to support your on demand cloud in the future? In the past few months there are basically no GPUs available. My understanding is that your reserved cloud starts at 64 GPUs (8 pods) and with quite a long reservation time
295.
by Palmik 2y ago
It's horrible horrible pricing! Their on demand price for A100 is what gets you H100 sxm in other places.
296.
by Palmik 2y ago
I really don't get this. Most of these low cost providers actually re-rent GPUs from Azure, AWS or GCP (*), yet they offer much better on-demand pricing (as low as $3.8/hr for H100 sxm and $2.5/hr for PCIe). And it's a f
297.
by Palmik 2y ago
Seems like ground.news is just very bad, I wouldn't rely on it. It was very easy to find other sources: https://www.nytimes.com/2016/07/12/upshot/surprising-new-evi... Note that the work is from 201
298.
by Palmik 2y ago
It's really amazing that they are releasing the results of all of that compute (the Llama models). Much more efficient and environmentally friendly than keeping things closed like Google or OpenAI!
299.
by Palmik 2y ago
It's not unlikely that the Llama 3 405B that's still training will outperform the best OpenAI model of today. The Llama 3 70B model in English already beats all models from Google, from Anthropic and it's only bested by the l
300.
by Palmik 3y ago
Yes, it's quite common, see for example: https://mozilla.vc/ https://www.cff.org/about-us/our-venture-philanthropy-model

More ›