mikeravkine

11 ms·

31.
by mikeravkine 3y ago
This is exactly what happened when I tried this approach personally: on a non trivial codebase, the tests were just as likely to be wrong as the code itself and during "debugging" the wrong would converge until the test passed but
32.
by mikeravkine 3y ago
TensorDock Marketplace is another option: https://marketplace.tensordock.com/order_list It's unique in that you can set your own prices, it's a true spot marketplace.. I've grabbed 2x3090 for $0.02/hr be
33.
by mikeravkine 3y ago
Sorry where are these .50c many core servers you speak of exactly?
34.
by mikeravkine 3y ago
Outliers only begin to appear around 3B parameters (as per the original LLM.int8 paper) so unfortunately not consumer GPU in an afternoon kinda stuff to prove you've managed to suppress them.
35.
by mikeravkine 3y ago
Maybe mention that GGML is under the hood. I see you've forked the library, did you pick up the GQA changes needed for llama2-70b? Also seems like you are picking quants on people's behalf, which ones?
36.
by mikeravkine 3y ago
Llama2 7B and 13B GGML are up and work with existing llama.cpp no changes needed! The 70B does require a llama.cpp change, but I'm sure it won't take long.
37.
by mikeravkine 3y ago
No, no you can't! When the admin borks the server that holds your data, it's just gone.
38.
by mikeravkine 3y ago
The GGLLM fork seems to be the leading falcon winner for now [1] It comes with its own variant of the GGML sub format "ggcv1" but there's quants available on HF [2] Although if you have a GPU I'd go with the newly releas
39.
by mikeravkine 3y ago
This smells very badly of quantization - the extra commentary is a failure mode I observe frequently when dropping down from FP16 down to 4 bits.
40.
by mikeravkine 3y ago
This is so terrible, almost hilariously so: https://heypi.com/s/gf72UPDDacbLwxTHEQLzg
41.
by mikeravkine 3y ago
Errors are so common with these endpoints, not having retry out of the box just makes it harder to hack on which I think is against the spirit?
42.
by mikeravkine 3y ago
What's your position on cannabis photo/review content? Your ToS mentions Australian law so going to guess I am just as unwelcome here as everywhere else
43.
by mikeravkine 3y ago
Its a pretty terrible model, I wouldn't use this for anything at all. Vicuna 1.1 outperforms it in all my tests.
44.
by mikeravkine 3y ago
I have been burned by Fediverse so badly I will never go back. I joined a Mastodon instance with 3k+ other users and made several hundred posts.. until one day when login failed. Turns out the admin lied about a lot of things (like backups
45.
by mikeravkine 3y ago
This is exactly how the serverless guys "get you": Low traffic is nearly free but you pay for it on the slope of the scale cost ramp.
46.
by mikeravkine 3y ago
The search plugin is bad. I have done some exploratory work [1] on specializing search agents to understand the query and result syntax and use that to structure results but I started with an explicitly structured data source to prove the
47.
by mikeravkine 3y ago
I've been working on a project evaluating the Python and Javascripr programing performance of open-licensed large language models [1] but don't currently have the resources to run the larger 65B ones. Any chance you'd be able
48.
by mikeravkine 3y ago
4bit Quantized versions that run on an A100-40G or 2x3090/4090 24GB: https://huggingface.co/TheBloke/falcon-40b-instruct-GPTQ Inference is very slow right now but it works!
49.
by mikeravkine 3y ago
This looks great! Sharing your Open AI key is probably a bad idea however.
50.
Cannabis – A state of the art about the millenary plant (2023)(sciencedirect.com)
1 points by mikeravkine 3y ago|0 comments
51.
Can AI Code? A self-evaluating junior developer interview for AI models(github.com)
1 points by mikeravkine 3y ago|2 comments
52.
by mikeravkine 3y ago
Here's a project I've been working on for the last couple of weeks that compares baseline programming performance across multiple LLMs. ChatGPT is king, which is unsurprising. What is surprising however is that several smaller mod