thesz

6 ms·

31.
by thesz 3mo ago
Nice to see a pretty advanced language at frontpage of HN! From what I understand, GRIN does some parts of supercompilation [1] during optimization process. Supercompilation can prove equivalence of functional programs [2] modulo terminatio
32.
by thesz 3mo ago
Haskell: https://ghc.gitlab.haskell.org/ghc/doc/users_guide/runghc.ht...
33.
by thesz 3mo ago
> The tokens it "predicts" aren't sampled from any naturally occurring distribution; the model's output is the result of an optimisation process that rewarded behaviour that was useful, and that's fundamenta
34.
by thesz 3mo ago
Bipolar disorder patients were able to adhere to ketogenic diet with great outcomes [1] (diet adherence for bipolar disorder was 95%). [1] https://www.youtube.com/watch?v=PBsjhpphowI - 18:20 and on.
35.
by thesz 3mo ago
WaveNet, which was sound-generating autoencoder for text-to-speech, one of the first viable few, had a separate "control" channel by design. Otherwise, it worked much like contemporary LLMs - there was a window of (autogenerated
36.
by thesz 3mo ago
Yours' "not as efficient" in [2] means that, sometimes, ADAM "does not work." Look at figure 2, ADAM literally does not work in the case of "true model."
37.
by thesz 3mo ago
ADAM does not work on simple convex problems [1]. [1] https://parameterfree.com/2020/12/06/neural-network-maybe-evolved-to-make-adam-the-best-optimizer/ [2] https://arxiv.org/pdf/
38.
by thesz 3mo ago
Can we use backspace key?
39.
by thesz 4mo ago
I do not know. I am here to point out that HIIT is almost a resistance training. ;) Here's meta review of resistance training for endurance running: https://pmc.ncbi.nlm.nih.gov/articles/PMC11258194/ Here&#x2
40.
by thesz 4mo ago
> You say that like it's a bad thing? I am saying that politicians will get around the sunsetting rule, if any, that's first. Constitution, then, will serve as what regular law does right now, without anything that make
41.
by thesz 4mo ago
HIIT is a borderline strength training. Consider Tabata protocol. It is supermaximal effort protocol, participants are required to exert maximum effort repeatedly. The duration of active phase of Tabata is 20 seconds, half of approximately
42.
by thesz 4mo ago
> Every law should have an automatic sunset period of 1-10 years that requires it to pass the entire legislative process again, or at least both full chambers + signing. Then you will have a lot of Constitution amendments. That
43.
by thesz 4mo ago
Adversarial inputs at the level of the government somewhat fascinate me. (aeons old blackmail tactic requires more work than the ability to select which of two data batches is more aligned with adversarial behavior)
44.
by thesz 4mo ago
https://en.wikipedia.org/wiki/Mixture_of_experts#Sparsely-ga... "The sparsely-gated MoE layer,[21] published by researchers from Google Brain, uses feedforward networks as experts, and linear-softmax gating. Simil
45.
by thesz 4mo ago
Slopkies.
46.
by thesz 4mo ago
> What kind of models are you including here? Logistic regression, non-linear logistic regression, simple neural networks, etc. Not the code as in "bunch of ifs" or decision trees. The truth tables? Predict zeroth bit i
47.
by thesz 4mo ago
> doesn't that mean we're approaching optimality? No. Transformers are Markov chains [1]. Somewhere around this fascinating site [2] I read that stateful models have an advantage. Author provided an example, a state mac
48.
by thesz 4mo ago
My comment in the previous discussion of that paper: https://news.ycombinator.com/item?id=48014197 Authors used LTL (linear temporal logic) to express, basically, non-reduced non-ordered binary decision diagrams. Or just bi
49.
by thesz 5mo ago
Most probably you do not want two of your SELECT statements to have the same seed during the execution. This necessitates some state, most probably, persisted one. Hence, imperativeness.
50.
by thesz 5mo ago
While it is true on the schema level, it is not true at the query level. The most frequently used kind of join is a general relation. [1] https://github.com/agirish/tpcds/blob/master/query1.sql Query 1 f
51.
by thesz 5mo ago
Imperative languages such as C/C++ specify "microtransactions" - an ordering over memory accesses (including (de)allocations) within some statement or group of statements. Compilers are free to rearrange these accesses if the
52.
by thesz 5mo ago
For all practical purposes, it very much is, consider random number generation.
53.
by thesz 5mo ago
Some of them, with notable exception of perfect squares of primes, can be expressed by products of different combinations of factors. E.g., 6^2 = (2 2 3) 3 = 2 (2 3 3). One of ~22 (ln(2^32)) perfect squares will be a square of perfect prime
54.
by thesz 5mo ago
PSP/TSP have everything to do with the discussion.
55.
by thesz 5mo ago
> gradient descent isn't good at combinatorial optimisation. If you convolve your problem with sufficiently wide Gaussian, you can use gradient descent. The approach is called Natural Evolution Strategies [1]. [1] https:&#x
56.
by thesz 5mo ago
Multiply "inference + backwards pass (~2x inference cost) + activations (vram overhead)" by batch size (thousands) to get to the actual RAM and compute cost. Optimizer like ADAM adds only two or three model-sized overhead. And las
57.
by thesz 5mo ago
The difference between training and inference is 1) one have to keep intermediate results for backward pass in training and 2) computation for training double because of the backward pass. Training is also done over batches, which increase
58.
by thesz 5mo ago
English is not my native language. PSP/TSP (Personal and Team Software Processes) and RUP (Rational's Uniform Process, to a lesser extent) are very valuable approaches to software engineering. Please, read about them, they are ver
59.
by thesz 5mo ago
LLMs are Markov Chains [1]. "Emergent abilities" of LLMs can be explained by decrease of perplexity in text prediction [2]. [1] https://arxiv.org/abs/2410.02724 [2] https://arxiv.org/abs&#x
60.
by thesz 5mo ago
> central concept or software engineering is architecture patterns. Both RUP and PSP/TSP do stand on the ground of defect prevention. All sorts of defects, from incorrect sets of requirements to memory corruption. Architectu

More ›