DavidSJ

10 ms·

121.
by DavidSJ 3y ago
Sorry, how is this any safer for the journalist? If their phone is compromised in a way such that someone can login and control their Signal app, their chat logs are already compromised. I’m just saying there should be the ability to export
122.
by DavidSJ 3y ago
Personally, I refuse to financially support Signal so long as they're still holding my chat logs hostage on my old iPhone and seem not at all concerned about solving this problem, which has existed for years. There was (and still is, s
123.
by DavidSJ 3y ago
I note that while E is more common than A if we're counting letters appearing anywhere in a word, A is substantially more common than E if we only count first letters of words: $ egrep -o . /usr/share/dict/wor
124.
by DavidSJ 3y ago
Oops, I dropped the factor of 6, so it’s actually 150x what’s typical.
125.
by DavidSJ 3y ago
Oops, I dropped the factor of 6, so it’s actually 626M tokens (306k sequences), which is about 150x what’s typical.
126.
by DavidSJ 3y ago
Thank you, I appreciate this reply!
127.
by DavidSJ 3y ago
Whether you count tokens or sequences, it’s about 25x the usual batch size. My guess is it makes for a fancy benchmark but isn’t actually useful. Would be interested in being proven otherwise.
128.
by DavidSJ 3y ago
You can measure it either way, and you’ll see it both ways in the literature. In this case it doesn’t matter much how you measure since 2048 is a typical pretraining sequence length. 300k sequences per batch is huge compared to typical batc
129.
by DavidSJ 3y ago
Question for rwitten or anyone else involved in this project: I see a per-device batch size of 6 for the 16B model. With 256x199 = 50944 TPUs and a sequence length of 2048, this works out to 104M tokens per batch. This is much larger than t
130.
by DavidSJ 3y ago
A uniform distribution over 30528 tokens is just under 15 bits of information per token, whereas a vocabulary size of ~60000 would be just under 16 bits per token. In practice it's not uniform, but this shows that they're in the s
131.
by DavidSJ 3y ago
Neither does GPT-4 or other sparse mixtures of experts, such as e.g. switch transformers [1]. [1] https://arxiv.org/abs/2101.03961
132.
by DavidSJ 3y ago
Meanwhile, ever since my upgrade to iOS 16, I'm asked for my passcode every time I attempt an iTunes backup. This is apparently by design, in order to address a security vulnerability: https://imazing.com/blog/ios-
133.
by DavidSJ 3y ago
SpaceX gives permission for their publicly-shared media such as pictures and videos to be freely re-shared. But it does not share most of its technology.
134.
by DavidSJ 3y ago
Yes, and it can be simpler to calculate it, but it tends to have worse theoretical properties. In particular, with log likelihood we can sum the scores for a sequence of conditional probability estimates and we get the same score as if it w
135.
by DavidSJ 3y ago
You can still determine the log likelihood of your predictions across the whole set of questions, even if you only have a single sampled response for each question.
136.
by DavidSJ 3y ago
I added that word because at least with non-bespoke backups, you have probably thousands of users each day testing restoration out of necessity, and word might get around if the method failed to restore. But nevertheless, one should still t
137.
by DavidSJ 3y ago
My snapshots are encrypted by the original computer (this is cool because the NAS can’t read them!). So I also needed to restore the encryption “wrapper key” to be able to use the backups. Not gonna lie, it was pretty terrifying until I ha
138.
by DavidSJ 3y ago
Indeed, netscape.com has been running a variant of this software for a long time: http://web.archive.org/web/20090604104519/http://netscape.ao...
139.
by DavidSJ 3y ago
> Half using it less than before isn't surprising if the other half are using it same or more than before. There's a breakdown of responses in a chart included in the article: - 6.7% stopped usage - 24% significantly decreased
140.
by DavidSJ 3y ago
I think that's consistent with their comment. They said it's still possible to expect profit over the long-term, you just won't have an edge over the market as a whole.
141.
by DavidSJ 3y ago
> <angle-sign> tau = 1 Who knew that if you go around in a circle you get back where you started?
142.
by DavidSJ 3y ago
To address the problem they discuss at the end with defining Θ = e^2πi, they could instead define Θ(x) = e^2πix, the circular analog to the exponential function exp (which is really more fundamental than exp(1) = e anyways).
143.
by DavidSJ 3y ago
Hmm, the universe is over 10^17 seconds old and the key space is apparently only a few orders of magnitude larger, so this seems wrong.
144.
by DavidSJ 3y ago
The attention is sparse in a single attention head in a single layer, but the model still has a way to route information where it needs it given multiple attention heads and multiple layers. Similarly, your computer is not physically connec
145.
by DavidSJ 3y ago
I haven't ever decided to fall into a black hole so I'm not speaking from experience, but I think Bob would only see Alice's (N+1)st photon after he himself has crossed the event horizon.
146.
by DavidSJ 3y ago
But this study doesn't say the opposite. It fails to show an effect. That's different from proving the absence of an effect. This was my first thought, jaded as I am from bad scientific reporting (such as the linked article) whi
147.
by DavidSJ 3y ago
seq2seq was indeed a big deal.
148.
by DavidSJ 3y ago
> … the publication of key research like LSTMs in 2014 … Minor nitpick, but LSTMs date to 1997 and were not invented by Google. [1] [1] Hochreiter and Schmidhuber (1997). Long short-term memory. https://ieeexplore.ieee.org&#x2
149.
by DavidSJ 3y ago
Didn't look to me like FTS. I think that was aerodynamic forces once the altitude got low enough.
150.
by DavidSJ 3y ago
The term you were looking for appears to be tacit collusion: https://en.wikipedia.org/wiki/Tacit_collusion Me: What's the official economics term for passive collusion, when players in a market avoid competing wit

More ›