jengels_

4 ms·

1.
The engineering challenges of scaling interpretability(anthropic.com)
4 points by jengels_ 2y ago|0 comments
2.
by jengels_ 2y ago
Sure, but completeness is a much higher bar than being able to find at least some things we weren’t looking for. And I’m reasonably optimistic that we’re going to make SAEs much better in the future, I agree they’re definitely imperfect rig
3.
by jengels_ 2y ago
It's a super interesting direction! That's one of the long term goals of interp research: deconstruct model behavior into circuits of features, and then turn those circuits into code (that we can maybe even formally verify!).
4.
by jengels_ 2y ago
I'm one of the first authors on this paper, happy to answer any questions :)
5.
by jengels_ 2y ago
I feel like un-supurvised methods like Anthropic's SAEs can be argued to find things we're not looking for (their most recent work is from a couple days ago: https://transformer-circuits.pub/2024/scaling-monos
6.
by jengels_ 3y ago
Pretty efficient! E.g. a recent paper describes a system to do fully private search over the common crawl (360 million web pages) with an end to end latency of 2.7 seconds: https://dl.acm.org/doi/10.1145/3600006.36
7.
GPT vs. Domain Specialized LLMs: Jack of All Trades vs. Master of Few(medium.com)
8 points by jengels_ 4y ago|1 comments
8.
by jengels_ 4y ago
Try it out! We have simple scripts you can play with and comparisons with GPT-3.
9.
by jengels_ 5y ago
ThirdAI (thirdai.com) | Houston, Bangalore, or remote | Full time | Senior Software Engineers We are a startup building high performance machine learning systems on CPUs. Our core technologies are hash-based algorithms that accelerate neura
10.
So, Gutenberg Didn’t Invent Printing as We Know It(lithub.com)
3 points by jengels_ 5y ago|0 comments
11.
by jengels_ 5y ago
ThirdAI (thirdai.com) | Houston or remote | Full time | Senior Software Engineer, VP Product, VP Engineering We are a startup building hash-based algorithms that accelerate neural network training and high dimensional near neighbor search.