renonce

10 ms·

61.
by renonce 2y ago
Anything that is not settled for several years, like papers published last year or so. Like RingAttention, quantization/pruning, rotary embedding, distillation, RLHF, L2 regularization, multimodal, MoE etc.
62.
by renonce 2y ago
Yeah but the other side of the coin is that they only explain the very basic concepts that are already settled for several years, not any of these "latest trends"
63.
by renonce 2y ago
Try out RGP lens. These are the sort of contact lenses that you wear before sleep and take off the morning. It maintains the shape of your eye, providing sharper vision in the morning with just the naked eye. I wear it since I'm 11 y
64.
by renonce 2y ago
Only if you are a 45-week old baby
65.
by renonce 2y ago
Yeah but it’s not representing a known number - I would rather say it is a proof that the busy beaver function for BLC at 49 bits is higher than Graham’s number. The fact that it represents known numbers concisely is more interesting to me.
66.
by renonce 2y ago
One invention that drew me to this topic was Binary Lambda Calculus, invented by John Tromp (“Tromp” as in Tromp-Taylor Rules). It’s just a direct binary encoding of a lambda calculus term, but it gives you a concrete and concise representa
67.
by renonce 2y ago
Essentially the algorithm is about pruning parameters on the fly and making the weight matrix sparse by setting least significant weights to zero, where "least significant" is defined in terms of the rank of the absolute value o
68.
by renonce 3y ago
Quantum teleportation does not allow you to transfer an amount of information faster than speed of light. Say you have two labs with two entangled particles 300000km away such that when one particle is “up” the other will be “down”. When la
69.
by renonce 3y ago
Based on your argument, the reason for legalization of weed is that the current police system has totally failed and decided the only way forward is to give up policing it? That would make this move seem more of a failure than a victory…
70.
by renonce 3y ago
I don’t think zero knowledge systems are practical at the moment. It will take over around 8 orders of magnitude more compute and memory to produce a ZKP proof of generic computation like compilation. Even 2 orders of magnitude is barely ac
71.
by renonce 3y ago
While I don't think it's practical to expect every paper author to write in the most consise and elegant style possible, one thing I do wonder is the lack of a "comment section" for papers: why isn't there a central
72.
by renonce 3y ago
While I don’t expect the company to do such PR themselves, from my experience in China it is indeed the case that many major public figures or products began their explosive growth from negative comments like this
73.
by renonce 3y ago
I understand the definitions of such crypto algorithms but have no idea about differential cryptanalysis. Can someone explain how attacks like this are constructed, and why it took 8 years to advance cryptanalysis by 3 rounds? What insight
74.
by renonce 3y ago
I'm not sure what this is for - can't I use the API of the underlying models? Using Supervision means I need to learn a new API for using existing CV tools and I don't see how it reduces enough complexity to justify that extr
75.
by renonce 3y ago
Competitors are catching up and one day the chat API providers will race the prices to bottom
76.
by renonce 3y ago
Depends on whether Apple’s own browser engine requires JIT. I think the point is to ensure fair competition.
77.
by renonce 3y ago
Highlight from this paper: > Our empirical investigations show the effectiveness of our methods, which allow us to estimate the embedding size of OpenAI’s gpt-3.5-turbo to be about 4,096. Discussion of the other paper is at https:/
78.
Logits of API-Protected LLMs Leak Proprietary Information(arxiv.org)
1 points by renonce 3y ago|1 comments
79.
by renonce 3y ago
> the researchers examined the relations between participants’ self-reported engagement in both types of BSing and their ratings of how profound, truthful, or accurate they found pseudo-profound and pseudo-scientific statements and fake
80.
by renonce 3y ago
Sounds like an interesting direction of research. There is a tool called mergekit and the models produced by merging different models is called frankenmerge. Search for “frankenmerge” and you can find a lot of interesting results and models
81.
by renonce 3y ago
> The obvious alternative to gradient descent here would be Bayes Formula If you know a little about the math behind gradient descent you can see that an embedding layer followed by a softmax layer gives you exactly the best Bayes estima
82.
by renonce 3y ago
At least in theory we can achieve incremental learning by training from scratch every time we got some new training data. There are drawbacks with this approach such as inconsistent performance for different training runs and significantly
83.
Former US Surgeon General got a $5k bill for dehydration(businessinsider.com)
3 points by renonce 3y ago|0 comments
84.
by renonce 3y ago
It’s stealing the last layer (softmax head), not an arbitrary part, also it targets “production models whose APIs expose full logprobs, or a logit bias”. Not all language model APIs have these features and this characterizes what APIs can b
85.
by renonce 3y ago
You dont need to wait for that one day. Right now I’m using cloud servers from hetzner and store data in Backblaze S3 and train models on Lambda or even vast.ai.
86.
by renonce 3y ago
I learned the easiest way to do d-dimension sampling in Foundations of Data Science: see https://news.ycombinator.com/item?id=34575637 or https://www.cs.cornell.edu/jeh/book.pdf?file=book.pdf I don&#x2
87.
by renonce 3y ago
Two places where the project is very impressive: 1. Emulating a desktop with windows so smoothly on both desktop and mobile. This webapp is much snappier than most webapps these days. 2. A storage and file explorer with a friendly API for t
88.
by renonce 3y ago
> Founders and employees would be paid only if the acquisition exceeded $559M I think the problem with the article is that it emphasized the absolute acquisition value and omitted the fact that the company raised $400M. Considering the a
89.
by renonce 3y ago
Here is my shiny new super business plan for a startup that will profit thousands from a supposedly non-paying customer: 1. Offer “free static website” with lots of templates and guides to help you build one 2. The first 100GB is free and b
90.
by renonce 3y ago
So is this GPT for images? They take a generative model and apply finetuning via LoRA on some downstream task such as surface normal and conclude that these models intrinsically learn these representations, and find that they do better than

More ›