ankit219

16 ms·

181.
by ankit219 2y ago
This is done at a reranking step. It's again custom. You have two variables - 1/ relevance (which most algos focus on) 2/ Date. Create a new score based on some combination of weights for relevance and date. Eg; Could be 50%
182.
by ankit219 2y ago
I don't have links unfortunately. What I meant was that at the time of indexing, you can add more information to any chunk. This[1] is a simple example by Anthropic where they add more relevant context. In our case, say you have two mo
183.
by ankit219 2y ago
Great to see this here. We used this dataset from Tiny Stories to train small models (as small as 20M params) and test out knowledge addition. Published a paper based on this dataset. We could get coherent outputs at sizes as low as 20M-25M
184.
by ankit219 2y ago
Short answer: Yes, there are ways it can be done. Multiple. Needs to be custom built though, given no one has explored it deeply yet. One simple way is what Omar Khattab (ColBert) mentioned about scoring function instead of a simple vector.
185.
by ankit219 2y ago
The other comment posted YT videos where Open AI researchers are talking about TTC. So, I am wrong. That $200 subscription is just because the number of tokens generated are huge when CoT is involved. Usually inference output is capped at 2
186.
by ankit219 2y ago
Thank you, this is helpful. The post on X was seemingly wrong.
187.
by ankit219 2y ago
They are highly effective pattern matchers. You change the pattern, it won't work. I don't remember who, but most likely @tszzl (roon), commented on x that they still trained the traditional way, and there is no test time compute
188.
by ankit219 2y ago
I meant not a jump in text generation ability, but more like adding a completely new modality and the likes. With 4o, you can have a multimodal embedding space and provide more relevant context to a model for fewer tokens (and higher accura
189.
by ankit219 2y ago
> How can we develop anything that will still stand in 12 months time? The pace at which things are moving, likely none. You will have to keep making changes as and when you see newer things. One thing in your favor (arguably) is that ev
190.
by ankit219 2y ago
If you have to deal with domain specific data, then this would not work as well. I mean it will get you an incremental shift (based on what I see, it's just creating explicit relationships at the index time instead of letting the model
191.
by ankit219 2y ago
Deploying in production, the current agentic systems do not really work well. Workflow automation does. The reason is very native to LLMs, but also incredibly basic. Every agentic system starts with planning and reasoning module, where an L
192.
by ankit219 2y ago
Not surprising. This is an early alpha phase to be fair, where there isn't enough integration to be wowed. The features are basic, and if you use other Apple apps, you get some help, but it's not as deeply integrated as you would
193.
by ankit219 2y ago
The small model you trained, how did you annotate the dataset? Because most of the times output from big models will be subjective, and with not a drastic difference in quality. If there is a drastic difference, you dont need a model/c
194.
Adding new domain knowledge to a small language model(medium.com)
2 points by ankit219 2y ago|0 comments
195.
by ankit219 2y ago
In Indian society terms, I meant there is a behavior a society will approve of and expect from successful people. If they do not follow this norm (so to speak), they will be criticized for whatever they are doing. Ambani's wedding was
196.
by ankit219 2y ago
Many Indians (esp entrepreneurs) don't realise it, Ratan Tata's conduct is the default expectation how the successful and hyped people in India are to behave (not taking a moral stand, rather a factual one) in the society. He was
197.
by ankit219 2y ago
If you are still curious about the architecture, from the blog: > To add image input support, we trained a set of adapter weights that integrate the pre-trained image encoder into the pre-trained language model. The adapter consists of a
198.
by ankit219 2y ago
https://technode.com/2024/03/27/tsmc-sees-strong-demand-for-... Right now there are three. Intel, Apple, and AMD. In 2022, there was one. Samsung manufactured for Qualcomm who were in testing phase.
199.
by ankit219 2y ago
The market changed recently. With ARM (and not sure how many others), it democratized chip design to an extent that companies could design their own requirements, simulate, add standard functionalities, and then find a third party fab which
200.
by ankit219 2y ago
A weird article. Apple was not the first or the only company to look to East Asia for cheap manufacturing, elctronic equipments, chips, or cheap labor. The trend was started and accelerated by motor companies and hardware just followed suit
201.
by ankit219 2y ago
Been on neon for a while. Using it for simple queries and RAG (pgvector). The thing I like about it most is the speed where I can get query output in 200ms-500ms. Incredibly fast. Given the serverless nature, it's my go to for any POCs
202.
by ankit219 2y ago
Dont think Apple would make a deal with any other engine. They basically used Microsoft to get a good deal with Google, but never had any intention to use Microsoft. The thing is the deals are a revenue sharing deal - Google gives 1/3
203.
by ankit219 2y ago
Oh yeah, i thought the previous user used them interchangeably. I messed it up. I think it's exclusive in a way that no one else could make the same deal. Not explicitly locking others out, but in a significant way, it implicitly did.
204.
by ankit219 2y ago
This would have been fine if Google was not a monopoly. The key part of the ruling is that judge agrees Google is a monopoly. And it's behavior is maintaining the monopoly illegally. Because it's a monopoly, and paying over the od
205.
by ankit219 2y ago
I fail to see your reasoning. The court's argument is Google is maintaining this dominance via these default deals, not that it got there through them. Given the last 25 years of history, Google's product has gotten worse after th
206.
Apple's motion to dismiss their antitrust lawsuit [pdf](storage.courtlistener.com)
3 points by ankit219 2y ago|0 comments
207.
by ankit219 2y ago
This is true at early stage when there aren't many clear signals. When this is happening at the middle/late stage, where there is a clear market (created or existing), a mature product, and years of numbers/traction/trac
208.
by ankit219 2y ago
There is an often less discussed point here about investing in developing markets. [1] Tweet here goes into more details. The gist is that investor distribution is barbell shaped. Hence, any company which crosses a particular chasm of raisi
209.
by ankit219 2y ago
As is typical with any RAG strategy/algorithm, the implicit thing is it works on a specific dataset. Then, it solves a very specific use case. The thing is, if you have a dataset and a use case, you can have a very custom algorithm whi
210.
by ankit219 2y ago
Somewhat disagree with the point being made here. The fundamental assumption for humans is that when summarizing we will pay more attention to important bits, and sort of only give a passing mention to others if needed. For any model, the c

More ›