10 ms·
Probabilistic Artificial Intelligence
- jacob019 2y agoThis is great. Is it available as a printed book?
- falcor84 2y agoFrom a brief search I see that it isn't (or it least not yet), but seeing how well-formatted the pdf is, and the fact that it's CC-licensed, you could print it yourself, or perhaps talk with them to organize a batch. Though I personally prefer to read these sorts of books directly from pdf, and am grateful to them for sharing it on arxiv.
- mnky9800n 2y agoI wonder if one could organize an arXiv print service that binds and prints and ships with a unique cover and such. Also it should use LLMs and the blockchain. But this would be nice there are a number of papers and such that if you could submit an arXiv link to a print service I would probably buy a copy. I wonder why no one does it.
- woolion 2y agoAren't you describing Lulu but for the very niche case of arxiv publications that are small books but not published as books? I think you could do it in a weekend with their API.
- mnky9800n 2y agoyes thats what i thought after the post. haha.
- ivansavz 2y agoIf anyone is interested in trying this, here is some Python starter code you might find useful: https://github.com/minireference/lulu-api-client?tab=readme-ov-file#usage https://github.com/minireference/lulu-api-client?tab=readme-... This worked four years ago when the API was still launched, but there might have been changes since, so no guarantees. Most ArXiv PDFs are probably lulu-printable out of the box, but to make a general solution, one would probably need to do some pre-processing with ghostscript (gs), e.g. embed all fonts and flatten images (no transparency).
- mnky9800n 2y agoalso, they require that you have both the interior contents as a pdf and a pdf of the cover, so it will need to either auto-generate or offer the option to have a custom cover art.
- madcaptenor 2y agoI wonder if they're aiming for it to be a book. Hubotter describes it on his web page as "notes on Probabilistic AI".
- au8er 2y agoThe "book" is accompanying studying material for the course Probabilistic AI at ETH Zurich. Essentially each chapter is the material covered in one lecture (3hrs). Source: I did the course
- madcaptenor 2y agoI think this is where a lot of textbooks come from.
- esafak 2y agoI do not think so. I am asking the author for confirmation.
- adrc 2y agoThere's no printed version. Btw I took this course at ETHZ last year (a course with this title and whose script is this document. Pretty nice course and pretty nice course notes, happy to see that the authors decided to share it outside of the course website now!
- abhgh 2y agoI came across this a few days ago, and my excuse to give it a a serious look is that Andreas Krause has some deep and interesting research in Gaussian Processes and Bandits [1]. [1] https://scholar.google.com/scholar?start=10&q=andreas+krause&hl=en&as_sdt=0,5 https://scholar.google.com/scholar?start=10&q=andreas+krause...
- trostaft 2y agoIt's Krause, he's one of the biggest researchers in the field. At least based on the other work of his I've read, he's a good writer too. This ought to be a worth while read.
- brador 2y agoInteresting separation and distinction between noisy inputs, noisy processing and noisy chains.
- thisisauserid 2y agoGemini 2.0 Experimental 02-05 sees this as "only" 107K tokens. Handy if you want help breaking this down. https://aistudio.google.com https://aistudio.google.com 'Laplace Approximation is a "quick and dirty" way to turn a complex probability distribution into a simple Gaussian (bell curve). It works by finding the highest point (mode) and matching the curvature at that point. It's fast and easy, but it can be very inaccurate and overconfident if the true distribution doesn't look like a bell curve.'
- mitthrowaway2 2y agoYou can also think of this as using the first two terms of a Taylor series approximation in log domain and throwing away the rest!
- dcreater 2y agoAs a layman in this field, I have no idea the contact it significance of this work. Can someone better informed inform us?
- cubefox 2y agoApparently they don't discuss language models at all.
- cubefox 2y agoWhich is a major omission, as transformer-based language models are the most powerful available form of "probabilistic artificial intelligence". They predict a probability distribution over a token given a sequence of previous tokens. My guess is that most of the content in the book is several years old (it's apparently based on an ETH Zurich class), despite the PDF being compiled this year, which would explain why it doesn't cover the state of the art.
- antonkar 2y agoI think we’ll need a GUI for the models to democratize interpretability and let even gamers explore them. Basically to train another model, that will take the LLM and convert it into 3D shapes and put them in some 3D world that is understandable for humans. Simpler example: represent an LLM as a green field with objects, where humans are the only agents: You stand near a monkey, see chewing mouth nearby, go there (your prompt now is “monkey chews”), close by you see an arrow pointing at a banana, father away an arrow points at an apple, very far away at the horizon an arrow points at a tire (monkeys rarely chew tires). So things close by are more likely tokens, things far away are less likely, you see all of them at once (maybe you’re on top of a hill to see farther). This way we can make a form of static place AI, where humans are the only agents
- ikeashark 2y agoWhat
- antonkar 2y agoWhat what?)
- bongodongobob 2y agoWhat
- antonkar 2y agoAnother related shocking idea: https://news.ycombinator.com/item?id=43319726 https://news.ycombinator.com/item?id=43319726
- soulofmischief 2y agoI had a mind-bending Salvia trip at eighteen that went sort of like that. My mind turned into an infinitely large department store where each aisle was a concurrent branch of thought, and the common ingredient lists above each aisle were populated with words, feelings and concepts related to each branch. The PA system replaced my internal monologue, which I no longer had, but instead I was hearing my thoughts externally as if they were another person's. I was able to walk through these aisles and marvel at the immense, fractal, interdependent web of concurrent thought my brain was producing in realtime.
- fud101 2y agoBooks suck (imho). We need a new format to teach and learn this deep technical stuff. Not youtube, something interactive with exercises and engagement.
- jgord 2y agoyeah, I mean 3Blue1Brown has done a great job .. and maybe those would be even better if you could app-ify them into something you can interact with. Current gen of LLM programming AIs might make it less leg-work to make these
- whimsicalism 2y ago3b1b is great but if you want to do deep technical work, you’re eventually going to have to get comfortable with text as a medium
- thomasahle 2y ago> something interactive with exercises and engagement Books have exercises. It's your job to engage. This book, in particular, has 3 pages of Problems per chapter. The only way to learn the math is to do all of them.
- nh23423fefe 2y agothanks. i was worried about job security for a nanosecond.
- jcgrillo 2y agoIt's a wild world where "reading the documentation" or "researching a topic" has become a career superpower. I'm glad my education largely predated social media and cell phones, and that I learned to read and work problems independently. OTOH it often makes work a very lonely, taxing experience. Being a human index into documentation is a hell of a lot less fulfilling than working with people who also can read.
- vessenes 2y agoI urge you to rethink this perspective. All research shows that paper increases comprehension significantly over screens and even over eink. Additionally hand note taking again has a positive impact.
- nbeleski 2y agoSeems similar, or at least partially overlap, with what I would say is the best reference on the subject, an Introduction to Statistical Learning from Gareth James et al [1]. I wonder it this one might be a bit more accessible, although I guess the R/Python examples are helpful on the latter. [1] https://www.statlearning.com/ https://www.statlearning.com/
- whimsicalism 2y agonot really, islr is a pretty basic book - this is about more advanced techniques to propagate probability estimates rather than point-wise and frankly i would not recommend islr anymore today, too dated
- keviniam 2y agoWhat would you (or other informed parties) recommend?
- whimsicalism 2y agoit’s been a while since I’ve been a beginner so I might not have the best resources, but I would recommend Harvard’s Stat 110 with Joe Blitzstein (lectures online) and then Machine Learning by Kevin Murphy. might be a scarier book to someone not confident in their math, but overall a better one imo for something more directly comparable to the niche ISLR filled, Bishop’s books are generally better - although I can’t recall their title
- esafak 2y agohttps://www.bishopbook.com/ https://www.bishopbook.com/ is the new one
- jgord 2y agoThe text has some great explanatory diagrams and looks to be a very high quality overview of ML thru the lens of probability, with lots of math. I was also recently impressed by Zhaos "Mathematical Foundation of Reinforcement Learning", free textbook and video lectures on YT : https://github.com/MathFoundationRL/Book-Mathematical-Foundation-of-Reinforcement-Learning https://github.com/MathFoundationRL/Book-Mathematical-Founda... If you dont have a lot of time, at least glance at Zhaos overview contents diagram, its a good conceptual map of the whole field, imo .. here : https://github.com/MathFoundationRL/Book-Mathematical-Foundation-of-Reinforcement-Learning#contents https://github.com/MathFoundationRL/Book-Mathematical-Founda... and maybe watch the intro video.
- vimgrinder 2y agoThe first lecture is so good. Not only from perspective of content, but how Zhao explain things about how to think about learning as a student. ty for recommendation.
- sdhar45 2y agoI took this class at ETH Zurich and it is one of my favorite classes. Especially how do you quantify uncertainty and how they build the starting blocks of reinforcement learning. I think it’s an excellent read for data scientists and ML engineers. This document is the lecture notes.
- chasely 2y agoKevin Murphy racing to rename his Probabilistic Machine Learning series.
- sunami-ai 2y agoI found Gaussian Processes with the right kernel to be very powerful with even just a few data points and a very small set of parameters. I don't know if I was using it correctly tbh, but it worked out great in predicting values that I could not predict so accurately. I used it as a predictable yet non-linear process to tweak the input in a computer vision task. The proof was literally in the pudding.
- wbakst 2y ago> be me > open article > "holy shit it's 400 pages" > realize i already have a grasp on most of the material from school > "phew" > oh this stuff is cool, just like i remember... > proceed to read all 400 pages well done! :clap:
- clarkedev 2y agoProbabilistic Robotics book by Thrun and co. offers a great overview of most of these concepts.
- svilen_dobrev 2y agostupid question: can a LLM (i.e neural network) tell me the probability of the answer it just spew? i.e. turn into fuzzy logic? Aaand, can it tell me how much it does believe itself? i.e. what's the probability that above probability is correct? i.e. confidence i.e. intuitionisticaly fuzzy logic? Long time ago at uni we studied these things for a while.. and even made a Prolog interpreter having both F+IF (probability + confidence) coefficients for each and every term..
- vlovich123 2y agoNot out of the box I think; I wouldn’t trust any self-assesment like that. With enough compute, you could probably come up with a metric by doing a beam search and using an LLM to evaluate how many of the resultant answers were effectively the same as a proxy for “confidence”.
- energy123 2y agoSimilar to bootstrapping a random variable in statistics. Your N estimates (each estimate is derived from a subset of the sample data) give you an estimate of the distribution of the random variable. If the variance of that distribution is small (relative to the magnitude of the point estimate) then you have high confidence that your point estimate is close to the true value. Likewise in your metric, if all answers are the same despite perturbations then it's more likely to be ... true? I'd really like to see a plot of your metric versus the SimpleQA hallucation benchmark that OpenAI uses.
- 2y ago
- overu589 2y agoExistential Reality is potential distribution not arrangement of states. Potential exists, probability is a mathematical description of its distribution. Every attribute is a dimension (vector). State is merely a passing measurement of resolve. Potential interacts through constructive and destructive interference. Constructive and destructive interference resolve to state in a momentary measure of “now” (an inevitability decaying proposition.) Existential Reality is potential distributing, not arrangements of state.