whakim

19 ms·

91.
by whakim 2y ago
We might as well call any theoretical discipline like math or computer science “not science” then, but somehow I doubt that’ll persuade you since you appear to have a priori decided on the relative validity of a bunch of disciplines anyways
92.
by whakim 2y ago
I'm not really sure I understand your point - isn't it self-evident that there is new data available to different fields at different rates at different times? And that when there are fewer available data points, each newly availa
93.
by whakim 2y ago
Archaeology (as well as new technology that allows us to interpret archaeological finds in different/more interesting ways) is constantly providing new data that changes our understanding of the ancient world. To return to the original
94.
by whakim 2y ago
There is plenty of evidence for the "decline and fall" camp but notably that conclusion varies a lot based on geography and perspective. Relying on mostly archaeological evidence, we do indeed see substantial drops in urbanizati
95.
by whakim 2y ago
AI can provably write code without mistakes?
96.
by whakim 2y ago
The mutable-default-arguments issue is easy for a third-party to linter to catch because it doesn't require any specific knowledge about primary keys and databases. Are there static typing plugins for other common packages that would c
97.
by whakim 2y ago
I totally disagree with this. You might as well argue that we shouldn't use code-completion of any kind because you might accidentally pick the wrong dependency or import. Or perhaps we shouldn't use any third-party libraries at a
98.
by whakim 2y ago
Yeah, I agree with this here. I think it's totally reasonable that something as specific as multiple Stripe subscriptions wouldn't be exercised by normal unit testing; as mentioned in the post, this wouldn't have been an easy
99.
by whakim 2y ago
There's no particular requirement for a RAG application to use vector search or embeddings, and there's no requirement that semantic similarity is in play (i.e. "retriev[ing] documents...that do not contain the terms in the s
100.
by whakim 3y ago
Just wanted to say thanks for the interesting discussion - I learned a few things and I hope you did too!
101.
by whakim 3y ago
When I was looking at AL query strategies (including random sampling, which I agree should always be included as a baseline), hyperparameter tuning (and model selection more generally) was definitely performed and optimized on a per-strateg
102.
by whakim 3y ago
Unfortunately I don't have anything to share, as much of the data now rests with my former employer. That being said, a few comments: * Understood that dataset selection exists within the context of what others are doing and trying to
103.
by whakim 3y ago
I've tried PCA (and perhaps more usefully, UMAP) and while can be great for stuff like unsupervised learning, I think the trick is to choose a classifier (as per my comment above) that can do a good job with high-dimensional data.
104.
by whakim 3y ago
Extremely interesting paper that runs counter to my experience. In my previous role I ran tens of thousands of classification pipelines similar to those described in the paper, using bert-based models and then running SVMs on top as classif
105.
by whakim 3y ago
I’ve had a lot of success training a classifier on top of these embeddings. An MLP works well, as does an SVM. If you don’t have labeled data, I’ve also found that various active learning techniques can produce extremely strong results whi
106.
by whakim 3y ago
I'm still not sold on recall at such large context window sizes. It's easy for an LLM to find a needle in a haystack, but in most RAG use-cases it's like finding a needle in a stack of needles, and the benchmarks don't r
107.
by whakim 3y ago
I see what you're saying, but you're assuming that consumer products are always chatbots (and that a small language model can buy time interacting with the user while possibly providing additional context). That being said, I woul
108.
by whakim 3y ago
Could you elaborate a bit more about how that would work in practice?
109.
by whakim 3y ago
From a consumer perspective, this is a super interesting paper because it touches on one of the fundamental issues with most RAG beyond the toy case - that you need to do different stuff depending on what the user is asking for. You also (u
110.
by whakim 3y ago
That may be your experience, but certainly not a universal experience (and apparently not the author's, either). In my experience, it's pretty easy to have CSVs (or Parquet files, or whatever) that are tens or hundreds of GBs in
111.
by whakim 3y ago
To me this criticism feels excessive. It feels like the author is describing their frustrations with internal usage of CSVs - there's no mention of customers and non-technical stakeholders at all. I think it goes without saying that
112.
by whakim 3y ago
I’ve used pgvector at scale (hundreds of millions of rows) and while it does work quite well, you’ll hit some rough edges: * Reliance on vacuum means that you really have to watch/tune autovacuum or else recall will take a big hit, a
113.
by whakim 3y ago
I used to work at a big car dealership making indirect auto loans, so I don't think I am underestimating the power of in-house financing. And I definitely know Ally's history! ;)
114.
by whakim 3y ago
Many car dealers work directly with banks to provide financing for customers. In these cases, the car dealers take a cut, so a customer who walks into the same bank might be pre-approved for a lower interest rate. (This doesn't always
115.
by whakim 3y ago
I really don’t think the issue with RAG is the size of the context window. In your example, the issue is selecting which papers to use, because most RAG implementations rely on naive semantic search. If the answer isn’t to be found in tex
116.
by whakim 3y ago
100%. This is why RAG and "classical search" will converge for non-trivial use cases. The folks who are doing RAG well still rely on many tried-and-true tricks of the trade: combining semantic search with keyword-based search, usi
117.
by whakim 3y ago
That's not actually moving the goalposts ;) Feel free to run your own experiment - I can almost guarantee that people will disagree with statements that seem like common sense to you, e.g. "a space station orbiting above a park is
118.
by whakim 3y ago
Sure. But you're just moving the goalposts from "what counts as a vehicle in the park" to "what counts as a vehicle in the park using even the most tenuous logic you could imagine." And I'm sure people could di
119.
by whakim 3y ago
If more than 1 in 10 people think that a commercial airliner flying an airplane over the park constitutes a "vehicle in the park" (which I think is ridiculous but apparently plenty of other folks don't!) I'm not sure how
120.
by whakim 3y ago
I'm pretty sure we're just going round in circles now. The point is that people disagree about everything , so anything you leave up to "common sense" will cause disagreement. Any you can't possibly define rules fo

More ›