montebicyclelo

10 ms·

61.
by montebicyclelo 1y ago
The diagonal-looking attention matrix shown in the post is mathematically equivalent to 3 by 3 convolution . The model learns how to do that via its attention mechanism - it's not obvious that it would be able to do that via attention
62.
Transformer neural net learns to run Conway's Game of Life just from examples(sidsite.com)
69 points by montebicyclelo 1y ago|35 comments
63.
by montebicyclelo 1y ago
> LLMs can't think. They are generating tokens one at a time Huh? They are generating tokens one at a time - sure that's true. But who's shown that predicting tokens one at a time precludes thinking? It's been shown t
64.
by montebicyclelo 1y ago
My thoughts are that it's key that humans know they will still get credit for their contributions. E.g. imagine it was the case that you could write a blog post, with some insight, in some niche field – but you know that traffic isn&
65.
by montebicyclelo 1y ago
It's permissive; but that doesn't imply not crediting people for their work
66.
by montebicyclelo 1y ago
> king-man+woman=queen Is the famous example everyone uses when talking about word vectors, but is it actually just very cherry picked? I.e. are there a great number of other "meaningful" examples like this, or actually the maj
67.
by montebicyclelo 1y ago
> Many have touted smoking marijuana as a safer alternative to cigarettes Wierd take, given that they are completely different drugs, with completely different effects on the brain, with the only similarity being that they are both inges
68.
by montebicyclelo 1y ago
Hmm, for my use case just allowing the lines to fail would have been better, (which I told the model)
69.
by montebicyclelo 1y ago
Does the code consist of many large try except blocks that catch "Exception", which Gemini seems to like doing, (I thought it was a bad practice to catch the generic Exception in Python)
70.
by montebicyclelo 1y ago
There's also two DBs I know of that have an updated Hacker News table for running analytics on without needing to download it first. - BigQuery, (requires Google Cloud account, querying will be free tier I'd guess) — `bigquery-pub
71.
by montebicyclelo 1y ago
So could do better than chance by excluding the option it's picked?
72.
by montebicyclelo 1y ago
Awesome work / demo / blog Link to the demo in case people miss it [1] > using a customized camera app which also recorded my phone’s motion Using phone's gyro as a proxy for "controls" is very clever [1] https:
73.
by montebicyclelo 2y ago
> Always zero out your graphs Great way to end up with useless graphs where you have a tiny line at the top that's been compressed to the point where you can't see any changes... The "rule" about ensuring axes are zer
74.
by montebicyclelo 2y ago
> How close would this be to GPT 2 Here's a post from 2015 doing something a bit like this [1] [1] https://nbviewer.org/gist/yoavg/d76121dfde2618422139
75.
by montebicyclelo 2y ago
It brings way more to the table than just being fast, like people are commenting. E.g. it manages Python for your projects, so if you say you want Python 3.12 in your project, and then you do 'uv run python my script.py', it will
76.
by montebicyclelo 2y ago
Great point. I've found it's very often possible to understand and fix problems "one shot" from stack traces alone — and we're talking production builds here... So I wouldn't turn them off, (an idea mentioned i
77.
The hierarchical highlight journalling system(sidsite.com)
1 points by montebicyclelo 2y ago|0 comments
78.
by montebicyclelo 2y ago
Super cool to see this idea working. I had a go at getting an LLM to play Pokémon in 2023, with openai vision. With only 100 expensive api calls a day, I shelved the project after putting together a quick POC and finding that the model stru
79.
by montebicyclelo 2y ago
A gameplay video at the top of the post would go a long way; for those who don't have time to click through and play on itch.io. (I see a few gifs on the itch.io page, but a longer gameplay video would be nice!)
80.
by montebicyclelo 2y ago
Top runners can run fast and far without it hurting. But when going for personal bests / very fast times, my understanding is that it is painful, for most of the run. E.g. watch some parkrun vlogs, and you'll hear people say thing
81.
by montebicyclelo 2y ago
The author asks: > Has the model ever seen cosine similarity? Yes - most of the time, at least for deep learning based semantic search. E.g. for semantic search of text, the majority are using, SentenceTransformers [1], models which have
82.
by montebicyclelo 2y ago
This 100%. Vector DBs have been heavily pushed by vector DB companies and cloud providers, and despite companies often having mere MBs of documents to search through for their use cases, amounts that trivially fit into RAM / can be sea
83.
by montebicyclelo 2y ago
One approach to this is the SingleFile browser plugin [1], configured to save pages to a GitHub repository - it saves the whole web page as a single HTML file in the repo. (Ok it's probably closer to archiving than bookmarking... but i
84.
by montebicyclelo 2y ago
Conda has a faster solver, as of 2022 https://www.anaconda.com/blog/a-faster-conda-for-a-growing-c...
85.
by montebicyclelo 2y ago
QR-to-QR: https://github.com/sradc/QR-to-QR The app lets phones send messages to eachother via QR codes. The two phones have to be put face to face. Video here: https://www.youtube.com/watch?v=4zUBj8X5E
86.
by montebicyclelo 2y ago
Nice, I guess a more established framework for writing webapps in Python is Streamlit, which is pretty good - minimal syntax can create quite an advanced, "reactive", GUI. It's very quick to throw together functional / r
87.
by montebicyclelo 2y ago
Agreed. Uncomfortable in many ways: - Heavy on the head, making the neck sore after a while. - Press into the face, which can also get sore. - Seals around the face, which is claustrophobic. - Insulates head/face, making hot and sweaty
88.
by montebicyclelo 2y ago
Very cool! As a further variation on this idea, I'm imagining training a reinforcement learning agent on atari games / super mario, but with an additional music-based reward/input, to try to get a "musical" looking
89.
by montebicyclelo 2y ago
It passes the test you've proposed. For the below results, the prompt is the same for A and B, except for in B the radius of the last circle is twice as big to make the rendered output unintelligible. A. Valid Mickey is detected by the
90.
by montebicyclelo 2y ago
Chat GPT: > Given the arrangement of three overlapping circles, it resembles the classic depiction of a *Mickey Mouse* head silhouette: > The two smaller circles represent Mickey's ears. > The larger circle represents his head

More ›