After the initial obsession with Jupiter-scale models, we quickly discovered that very small models can perform even better if you reorganize your problem. Las

After the initial obsession with Jupiter-scale models, we quickly discovered that very small models can perform even better if you reorganize your problem. Last week, we were trying to fine tune a 175b parameter model to take a natural language prompt with hopes of directly-outputting correct, domain-specific SQL. Now that reality has passed, we are looking at different paths. As of this week, we are trying to hit everything with the binary classification hammer. Turns out you can train a model to output 1 of 2 possible tokens with exponentially fewer parameters, training items, machine hours, etc. The statistics available in binary classification are also incredibly powerful and the results are trivial to reason with. Even if you need thousands of binary classifiers, their scale and granularity makes this a non-event or potentially an advantage. The real integration magic with AI/ML is starting to look like a weird form of set theory. At this level of complexity, detecting (and potentially confirming) the user's intention is way more important than trying to draw a direct map from input to destination.