convexstrictly

10 ms·

91.
SqueezeLLM: Dense-and-Sparse Quantization(arxiv.org)
5 points by convexstrictly 3y ago|1 comments
92.
by convexstrictly 3y ago
"We start from the surprising finding that certain attention heads have a clear activation distribution difference for true and false statements. Probing at these points yields upwards of 83% accuracy on TruthfulQA while zero-shot gene
93.
Inference-Time Intervention: Eliciting Truthful Answers from a Language Model(arxiv.org)
4 points by convexstrictly 3y ago|1 comments
94.
by convexstrictly 3y ago
"Orca surpasses ... Vicuna-13B by more than 100% in complex zero-shot reasoning benchmarks like Big-Bench Hard (BBH) and 42% on AGIEval. ... reaches parity with ChatGPT on the BBH benchmark and shows competitive performance in ... SAT,
95.
Orca: Progressive Learning from Complex Explanation Traces of GPT-4(arxiv.org)
4 points by convexstrictly 3y ago|1 comments
96.
by convexstrictly 3y ago
"We present SpQR, which allows lossless LLM inference at 4.75 bits with a 15% speedup. You can run a 33B LLM on a single 24GB GPU fully lossless. SpQR works by isolating sensitive weights with higher precision and roughly doubles impro
97.
SpQR: A Sparse-Quantized Representation for Near-Lossless LLM Weight Compression(arxiv.org)
8 points by convexstrictly 3y ago|2 comments
98.
by convexstrictly 3y ago
The Brainformer building block is designed using neural architecture search. "Brainformer consistently outperforms the state-of-the-art dense and sparse Transformers, in terms of both quality and efficiency. A Brainformer model with 8
99.
by convexstrictly 3y ago
Relevant codebase is https://github.com/getcursor/human-eval
100.
Azure GPT 3.5 completion endpoint bumps HumanEval from <50% to 74%(twitter.com)
1 points by convexstrictly 3y ago|1 comments
101.
by convexstrictly 3y ago
They claim average 1.45x speedup and maximum 1.7x speedup over GPTQ. High-level idea: About 1% of the weights contribute greatly to quantization error. So skip the quantization of these weights.
102.
by convexstrictly 3y ago
For fast inference, the HuggingFace cofounder, Thom Wolf recommends their text-generation-inference library https://github.com/huggingface/text-generation-inference https://twitter.com/Thom_Wolf/st
103.
by convexstrictly 3y ago
Agree. The Stable Diffusion Open RAIL M license: "You agree not to use the Model or Derivatives of the Model ... To defame, disparage or otherwise harass others" Does "disparage" have a settled meaning in law? Is a paro
104.
by convexstrictly 3y ago
Research in the LLM space is moving so fast, I doubt any architecture will "stick".
105.
by convexstrictly 3y ago
Do you consider the Unity model "sleazy"? But I agree that it was poor form to call it open-source at the time. Sadly, it seems to be standard practice in the LLM space to release models with all kinds of restrictions on use and
106.
by convexstrictly 3y ago
Yann LeCun: "No. But it's not because we don't want to. It's because of complicated legal issues." https://twitter.com/ylecun/status/1651782621540524032
107.
by convexstrictly 3y ago
https://huggingface.co/tiiuae/falcon-40b
108.
Falcon 40B LLM (which beats Llama) now Apache 2.0(twitter.com)
435 points by convexstrictly 3y ago|141 comments
109.
by convexstrictly 3y ago
Closed beta runs for a week. 4 bit floats. Quantization on top of quantization. Finetune Alpaca 7B in ~3 hours on an A40 GPU/RTX3090. Paging of optimizer states. Uses LoRA. Integrated with HuggingFace framework
110.
Tim Dettmers: QLoRA finetunes a 65B model on a single 48 GB GPU(twitter.com)
3 points by convexstrictly 3y ago|1 comments
111.
by convexstrictly 3y ago
Video: https://www.youtube.com/watch?v=iI8e3kU11eU