pona-a

17 ms·

301.
Learning Malbolge(lutter.cc)
2 points by pona-a 2y ago|0 comments
302.
by pona-a 2y ago
That does make me optimistic. "Stealing back" our stolen data does in the end a free model make — unless the current, very... unprecedented US admin decides distributing unauthorized distilled models carries a prison sentence. But
303.
by pona-a 2y ago
Gemini was looking quite nice the last time I've been there.
304.
by pona-a 2y ago
> DOJ leadership has put all Public Integrity Section lawyers into a room with 1 hour to decide who will dismiss Adams indictment or else all will be fired. Sending them strength to stand by their oath, which is to support the Constitut
305.
by pona-a 2y ago
Open benchmarks are vulnerable to saturation. I think benchmarks should have an embargo periodic, until which only 3% of the question-answer pairs is released, with an explicit warning not to use it 3 months after being released.
306.
by pona-a 2y ago
We can still see the motion is forced, the cursor takes non-direct paths, actively doges away from targets, requiring active steering to stay put long enough to press them. While there are few misclicks, it feels like a product of active ef
307.
by pona-a 2y ago
The original model, aside from its programming mistakes, also misremembered the doubling formula. I hoped to see that solved, which it was, as well as maybe a more general performance boost from recovering some distillation loss.
308.
by pona-a 2y ago
I tested it on basic long addition problems. It frequently misplaced the decimal signs, used unnecessary reasoning tokens (like restating previously done steps) and overall seemed only marginally more reliable than the base DeepSeek 1.5B. O
309.
by pona-a 2y ago
It really depends. After going on a month-long trip, Google was serving me ads in German, as well as offering Google Photos printing expecting a German address with the price in Euros, all despite us never entering Germany, let alone stayin
310.
by pona-a 2y ago
Why would it have to be one or the other? Yes, it's been proven LLMs do create world models, how good they are is a separate matter. There still could be goal misalignment, especially when it comes to RLHF. If the model has in its inte
311.
by pona-a 2y ago
> AI solving international math olympiad problems is not intelligence But couldn't it be overfitting? LLMs are very good at deriving patterns, many of which humans simply can't tell apart from noise. With a few billion paramete
312.
by pona-a 2y ago
But don't words have a fixed size embedding? Causal models create a sequence of attended word embeddings. It's only about 300 dimensions per one word, so it seems counterintuitive you can compress an entire reasoning chain into su
313.
by pona-a 2y ago
Previous discussion: [41 comments, 166 points] https://news.ycombinator.com/item?id=42919597
314.
by pona-a 2y ago
If chain of thought acts as a scratch buffer by providing the model more temporary "layers" to process the text, I wonder if making this buffer a separate context with its own separate FNN and attention would make sense; in essenc
315.
by pona-a 2y ago
My mind went to some horrible production accident. The first episode aired only 24 yeas after asbestos snow, after all.
316.
by pona-a 2y ago
I tested some cases in Misguided Attention[0]: while many cases now pass, others fail all the same. Given the amount of contamination and the difficulty of finding sufficiently original problems of this nature, I defer to a 20:80 ratio of g
317.
by pona-a 2y ago
https://old.reddit.com/r/OpenAI/comments/1ieonxv/comment/ma9...
318.
by pona-a 2y ago
To be honest, the heuristics here are kind of bad... The first two paragraphs from Wikipedia page for Viterbi algorithm return as AI generated, while several samples from TinyStories, a GPT-4 generated dataset of children stories for traini
319.
by pona-a 2y ago
Hyper + Shift + -
320.
by pona-a 2y ago
I am trilingual. Sometimes, Google would auto-translate their docs into the local language, despite my browser and account language being set to English. I hate this. Monolingual people may not fully grasp how much languages differ in the e
321.
by pona-a 2y ago
https://archive.ph/TOVrp
322.
Trump revokes Lyndon B. Johnson's landmark civil rights executive order(axios.com)
9 points by pona-a 2y ago|3 comments
323.
by pona-a 2y ago
Can you enlighten me of a state where none of parent's points apply? I'd be glad to be educated.
324.
ExpShield: Safeguarding Web Text from Unauthorized Language Modeling(arxiv.org)
2 points by pona-a 2y ago|0 comments
325.
by pona-a 2y ago
How about a naive Bayesian Bag of Words? Just find/scrape/generate with an LLM a large enough corpus of products/services, build the term frequency matrix, calculate class priors and P(term|class) and inference with straightf
326.
by pona-a 2y ago
It feels like a Markov chain isn't adversarial enough. Maybe you can use an open-weights model, assuming that all LLMs converge on similar representations, and use beam-search with inverted probability and repetition penalty or just GP
327.
by pona-a 2y ago
Perhaps CoT and the like may be limited by this. If your model is cooked and does not adequately represent less immediately useful predictions, even if you slap a more global probability maximization mechanism, you can't extract knowle
328.
by pona-a 2y ago
> But there are differences between the ratio and the widely used unemployment rate, which Trump used in citing the percentage. > The unemployment rate reflects the number of jobless people who are actively seeking work as a percentag
329.
by pona-a 2y ago
They will just relabel what is political. Union organizing? A bill on internet censorship? Anything mildly inconvenient to Meta or its shareholders? That's politics, you said you don't want to see any politics, didn't you? Th
330.
by pona-a 2y ago
Grammarly at least used to be deterministic and fast. Its edits are consistent and its mistakes are easier to spot. That alone is enough of a value proposition over LLMs, especially since their tech required years of computational linguisti

More ›