jampekka

37 ms·

721.
by jampekka 2y ago
Transformer encoders are not really popular anymore, and all the top LLMs are decoder-only architectures. But encoder models like BERT are used for some tasks. In any case, self-attention and MLP is the crux of Transformer blocks, be they i
722.
by jampekka 2y ago
Artists can be paid with exposure, teachers with the self-learning opportunity ;) More seriously, teaching is usually the best way to learn.
723.
by jampekka 2y ago
In civil law countries verdicts should not depend on an individual judge's feefees.
724.
by jampekka 2y ago
Judges are very much "brainwashed" too, and by design. The judges should apply the law, and the same case should ideally lead to the same verdict regardless of the judge. With the caveat that this applies to sane legal systems, an
725.
by jampekka 2y ago
"Knowing" something is different from formalizing it into a mathematical model. Almost all papers are about relatively minor findings at best. Publish and perish is real though. As are hype and bad popularization of research.
726.
by jampekka 2y ago
A friend of mine just defended his law PhD and in the introductory lectio said that (even) current LLMs would likely give better verdicts than human judges. Law isn't really a cognitively such demanding task as walking a dog or waiting
727.
by jampekka 2y ago
> Looks interesting, but I'm skeptical that a book can feasibly stay up to date with the speed of development. The basic structure of the base models has not really changed since the first GPT launched in 2018. You still have to und
728.
by jampekka 2y ago
Early models had a transreflective displays, newer models sadly don't.
729.
by jampekka 2y ago
Amazfit watches have done this for since 2016 or so. And cheaply.
730.
by jampekka 2y ago
Abliteration can be done with off-the-github scripts. It's not perfect, but works very well for the heaviest censorship. Male/female dynamics may be in the corpus too, and even the reality may famously have some perceived biases.
731.
by jampekka 2y ago
At least with open models you can relatively easily uncensor them quite well. Or even finetune your own.
732.
by jampekka 2y ago
Something like Mann-Whitney U would be safer. But it's indeed quite obvious from the samples that the difference is robust.
733.
by jampekka 2y ago
That there's a probability of 4.20e-05 that the observed difference of that large would happen by chance due to observation noise if there would be no real difference (given your assumptions about the data generating process holds).
734.
by jampekka 2y ago
The startup ecosystem in which the goal is to get bought to be killed by the monopolies would help us how anyway?
735.
by jampekka 2y ago
Indeed. At least as long as the living conditions are tolerable (for them), most people don't really care about things like censorship or surveillance or propaganda, no matter the system. The ones inventing the VPNs are a small minorit
736.
by jampekka 2y ago
In short yes. That's how the raw base models trained to replicate the internet are turned into chatbots in general. Making it to refuse to talk about some things is technically no different. There are multiple ways to do this: humans r
737.
by jampekka 2y ago
My guess would be that most Chinese even support the censorship at least to an extent for its stabilizing effect etc. CCP has quite a high approval rating in China even when it's polled more confidentially. https://dornsife.
738.
by jampekka 2y ago
There's both. With the web interface it clearly has stopwords or similar. If you run it locally and ask about e.g. Tienanmen square, the cultural revolution or Winnie-the-Pooh in China, it gives a canned response to talk about somethin
739.
by jampekka 2y ago
Pelosi didn't question the election results or call for their overturning. It was about the claims about foreign influence, which was later used to build the narrative for impeachment.
740.
by jampekka 2y ago
I didn't agree with the Trump ban and a lot of other "content moderation" (which affected the left wing too). But in the case of Trump he was banned due to the incitement of violence/revolt in the Capitol Hill incident a
741.
by jampekka 2y ago
Authors or their institutions don't get a penny from paywalled papers either. Oftentimes they have to pay to publish. Authors choose the titles, sometimes reviewers (not paid a dime either) can demand changes to the title, and the acad
742.
by jampekka 2y ago
Attention is all you need!
743.
by jampekka 2y ago
Good that China has caught up US in tech, so at least us Europeans can escape the US oligarchical censorship. Not looking great for those left behind the great firewall of Muskerberg. When will PG fall in line?
744.
by jampekka 2y ago
Interestingly people, especially poor people, were better nourished during the WW 2 rationing than before the war. Also e.g. universal healthcare was establised post-war. Is number of war ships, or billionaires, a good measure for a country
745.
by jampekka 2y ago
Good catch, indeed a measurement matrix is needed if the state and measurement are of different dimensions or require a (linear) transformation. For that use Y = H*z where H is the measurement matrix and z is the observation vector. For UKF
746.
by jampekka 2y ago
Yes.
747.
by jampekka 2y ago
This is a very educational and intuitive way of putting it, but to nitpick the Kalman filter is a very special case of this (it assumes a linear ruleset and Gaussian uncertainties in the sensor readings). What you're describing is in g
748.
by jampekka 2y ago
> Truly appreciate the power of linear approximations by going through algebra, appreciate the tricks of calculus, marvel at the inherent tradeoffs of knowledge with estimator theory, and see the joy of the central limit theorem being tr
749.
by jampekka 2y ago
Sympy can manipulate Gaussian distributions symbolically, but the Gaussian is more or less the only distribution that can be really symbolically manipulated. Though I'm not sure Sympy can handle the conditional (Bayesian posterior) dis
750.
by jampekka 2y ago
FWIW, I think I understand Kalman filters quite well, but the linked PDF is hard for me to follow, and I'd really struggle to understand it if I didn't already know what it's saying. I think the lesson there is that the Kalma

More ›