2 ms·

I just used it, went through the whole installation (took like 1 hour approx). Long but straightforward. If I put my computer to sleep will it continue? I star

I just used it, went through the whole installation (took like 1 hour approx). Long but straightforward. If I put my computer to sleep will it continue? I started the server (very curiously I was running oLlama in the same prot slotserve uses by default, instead of switching it which I know you can do, I just ditched oLlama, perhaps an insight for you) and built a small html hello world served via Python. The thing pointed me to the localhost link, nice! As an early user, my advice is to focus on efficiency. The efficiency of the installation but more importantly, the efficiency of running the thing. 8.1GB per slotserve process is a lot! Is that in your control? Also, I've seen an interest of certain kinds of programmers for open-weight models. "We all know agree that LLMs for coding are very useful but we're giving money to a small set of big, evil corporations. They're Trump donors. I heard it's bad for the environment because it uses water". If it's local and open-weight, this could be marketed this way I think. Finally, what's the actual, real use case for slotserve?

carloslfu 1mo ago
thanks! > ditched oLlama" yeah! this is interesting. > 8.1GB per slotserve process is a lot! Is that in your control? yes, it is hard, but I agree the smaller the better. I'll work on that > If it's local and open-weight, this could be marketed this way I think. I like this! > what's the actual, real use case for slotserve? I'm working rn on an app on top of it that closes the loop and is a fully local AI app, an experiment. I'll publish it as soon as it is usable! > built a small html hello world served via Python What did you use as a harness here?
- baristaGeek 1mo ago
  For the harness... just the shell. No client library. Does that answer your question?
  - carloslfu 1mo ago
    thanks! in part, I was wondering how you got the code into files. I guess you copy pasted it inside a file, am I right?

carloslfu 1mo ago

thanks! > ditched oLlama" yeah! this is interesting. > 8.1GB per slotserve process is a lot! Is that in your control? yes, it is hard, but I agree the smaller the better. I'll work on that > If it's local and open-weight, this could be marketed this way I think. I like this! > what's the actual, real use case for slotserve? I'm working rn on an app on top of it that closes the loop and is a fully local AI app, an experiment. I'll publish it as soon as it is usable! > built a small html hello world served via Python What did you use as a harness here?

baristaGeek 1mo ago
For the harness... just the shell. No client library. Does that answer your question?
- carloslfu 1mo ago
  thanks! in part, I was wondering how you got the code into files. I guess you copy pasted it inside a file, am I right?