borzunov

13 ms·

31.
by borzunov 4y ago
Sure! People who disconnect for a while (not necessarily predictably) won't be penalized - it's okay if you suddenly decide to use your GPU for something else, then get back to running a server.
32.
by borzunov 4y ago
A Petals dev here. At the moment, we're working on a centralized incentive system, no blockchain involved. It will award points if someone is running a server that consistently stays online and returns correct results. Then, users will
33.
by borzunov 4y ago
I think this is a relevant link for you: https://huggingface.co/transformers/v4.9.0/parallelism.html For large LMs, people usually use tensor-parallelism (TP) or pipeline-parallelism (PP). TP involves lots of comm
34.
by borzunov 4y ago
Really large (GPT-3-sized) language models have much more parameters than diffusion models, so it's difficult to load them locally unless you have a server with 8x 3090/3x A100 GPUs. Petals is the only way to fine-tune and inferen
35.
by borzunov 4y ago
You can switch the chat bot ( http://chat.petals.ml ) to the "few-shot mode" and provide a couple of "task description & code" examples. Then you can add a new task description and it'll respond with c
36.
by borzunov 4y ago
A Petals dev here. Indeed, the public swarm should not be used for any kind of sensitive data (we have warnings about that in the instructions). If someone wants to process such data, we recommend to set up a private swarm among the orgs th
37.
by borzunov 4y ago
A Petals dev here. It is not real-time, but we think the speed of ~1 token/sec may be enough for some interactive apps such as chat bots (especially, if you show tokens to a user once they are generated). You can try one at http:&#x2F
38.
by borzunov 4y ago
Offloading is another popular method for running large LMs when you don't have the GPU memory to fit the entire model. Imagine you have an A100 GPU with 80 GB memory and want to generate text with BLOOM, a 70-block transformer model wi
39.
Petals is creating a free, distributed network for running text-generating AI(techcrunch.com)
3 points by borzunov 4y ago|1 comments
40.
by borzunov 4y ago
See technical details in the paper: https://arxiv.org/abs/2209.01188
41.
Show HN: Run and fine-tune 175B+ LMs in Colab using a P2P network of GPUs(github.com)
3 points by borzunov 4y ago|0 comments
42.
by borzunov 4y ago
I'm not sure it's a problem since newer versions of CPython usually introduce only minor updates to bytecode ops.
43.
by borzunov 4y ago
A while ago, I made a similar tool for Pocketbook eInk readers specifically: https://github.com/borzunov/remoteink
44.
by borzunov 4y ago
Yeah, I've remembered this episode as well!
45.
by borzunov 5y ago
I do the same in my module that enables increment operators in Python by patching the bytecode for two unary plus/minus ops: https://github.com/borzunov/plusplus
46.
Bytecode hack that enables increment operators in Python(github.com)
2 points by borzunov 5y ago|0 comments
47.
Show HN: Enable ++x and --x expressions in Python(github.com)
4 points by borzunov 5y ago|1 comments
48.
Show HN: Turn Pocketbook e-ink reader into computer monitor(github.com)
7 points by borzunov 5y ago|1 comments
49.
by borzunov 5y ago
Hi, I'm the author of remoteink. Recently, the idea of using an e-ink device for programming/writing texts/showing dashboards/etc turned out to be popular among HN users [1, 2, 3], so I've decided to share my tool f
50.
by borzunov 5y ago
Years ago, I've made a program turning Pocketbook e-ink readers into computer monitors with some extra features: https://github.com/borzunov/remoteink The update speed is quite good (comfortable for writing texts
51.
by borzunov 6y ago
Reminded me of the tool turning Pocketbook E-Ink readers into computer monitors, which I made several years ago: https://github.com/borzunov/remoteink This could also be used for the dashboards if connected to a remo
52.
Show HN: Exploit farm for attack-defense CTF competitions(github.com)
1 points by borzunov 7y ago|0 comments
53.
Show HN: Extend built-in Python collections with LINQ-for-objects style methods(github.com)
15 points by borzunov 11y ago|0 comments