canyon289

5 ms·

31.
by canyon289 10mo ago
So I didn't even know this was going to be made until recently, and when I saw it, it also blew my mind. I didn't realize how far along web ml community had pushed things, and was impressed by the creativity of the HF folks with v
32.
by canyon289 10mo ago
Trust me as a daily at home Gemma user myself, I'm just excited for what's upcoming as you are, maybe even more because I have some hints for what's to come. >My "dream" scenario is to have the "agentic&quot
33.
by canyon289 10mo ago
Hi all, I'm a research lead on this model. Same as every model release post, I enjoy working at Google for a multitude of reasons, and opinions here are my own. Happy to answer whatever technical questions I can!
34.
by canyon289 1y ago
I wrote a post here with zero abstractions. Its all self contained and runs locally. https://ravinkumar.com/GenAiGuidebook/language_models/Agents... https://github.com/canyon289/ai_agent_basic
35.
by canyon289 1y ago
Hi! I work directly on these teams as a model builder and have talked to my colleagues are the other labs well. All our orgs have openings and if you also could consider working for organizations such as the UK AISI team and other independe
36.
by canyon289 1y ago
BDA is THE book to learn Bayesian Modeling in depth rigorously. For different approaches there are a number shared here like Statistical Rethinking from Richard McElreath or Regression and other stories which Gelman and Aki wrote as well. I
37.
by canyon289 1y ago
> What does shaping mean in this case? What tools are used, what requirements are there, both in terms of hardware and knowledge? I'll try making an analogy to another task I like which is cooking. In cooking the chef has to make de
38.
by canyon289 1y ago
It can possibly perform basic prompted FC but I wouldn't get your hopes up. It should be to be a solild FC model if trained on specific tools and format. I would not expect great MCP performance because the context window is 32k and mo
39.
by canyon289 1y ago
Its a crucial question. I wrote up a long answer here. Let me know it helps https://news.ycombinator.com/item?id=44913558
40.
by canyon289 1y ago
The world reasonable is vague but assuming you mean something that could be run in a residential unit it would long a very long time if training from pure scratch. This is part of the rationale for releasing this model. Now you don't h
41.
by canyon289 1y ago
Thank you! The Gemma3 technical report contains many details on training setup https://arxiv.org/pdf/2503.19786 This was released with the initial batch of Gemma3 so it doesn't contain the 270m details, nonetheles
42.
by canyon289 1y ago
The Unsloth folks have finetuning numbers. Linking their post here https://www.reddit.com/r/unsloth/comments/1mq5hbb/google_gem...
43.
by canyon289 1y ago
The evaluations provide this indication. You'll see MMLU, GPQA, Big Bench etc in reports for many models. Those numbers provide the indication you're looking for. To answer a question you didn't ask. With small models especia
44.
by canyon289 1y ago
At this small scale the embeddings indeed were a big focus. Consider this thought process. The tokens themselves are a form of compression. Lets say we have the word "WaffleHouse", character level this would be 11 tokens, but with
45.
by canyon289 1y ago
Yes! To me the primary value is not just as a teaching or toy model. I see a lot o value in repeatable tasks if we think about enterprise and a local fast developer model for individual usage. Here's some examples that are inspired by
46.
by canyon289 1y ago
Hey all, I created this model with a top notch team. I answered many questions last week when this hit the front page, and happy to answer more here as well. https://news.ycombinator.com/item?id=44902148 Personally I'm
47.
by canyon289 1y ago
I'm seeing the same question come up about general performance versus specialized performance, so let me offer a longer explanation here. This might be worth a blog post at some point. We now live in a world of both readily available s
48.
by canyon289 1y ago
I'm quite glad to hear its working for you! Thank you for adding the comment here as well, we definitely try our best to make useful models, but its fantastic to hear from actual users that we hit the mark. Really appreciate it
49.
by canyon289 1y ago
Without seeing the full experiment and data its hard to tell, sort of like guessing why a soup tastes bad without trying it but here's my guesses! 1. Good instinct with LORA and PEFT. As others suggested below perhaps try changing the
50.
by canyon289 1y ago
We constantly are evaluating architectures trying to assess what will work well in the open ecosystem. It's quite a vibrant space and glad you have one option that works. For this model in particular we evaluated a couple of options be
51.
by canyon289 1y ago
Its not an MOE, its what's referred to as a dense architecture, same as the Gemma3 models (But not 3n as noted)
52.
by canyon289 1y ago
Ah yes thank you. Even I still instinctively type B
53.
by canyon289 1y ago
The blog post contains links to several ways to try this model, locally, on colab, and in the cloud. Pick what works best for you!
54.
by canyon289 1y ago
I don't think we dumped the model and say you do the rest? Myself and my colleagues spent many days transforming the weights into various open compatible formats. And its not just us there's many orgs and partners dedicating their
55.
by canyon289 1y ago
Not rude at all and I'll again share what I can. We ran a bunch of experimental architectures at this size to get a sense of performance at this size, in particular how well it was able to adapt to datasets across some loss measures. F
56.
by canyon289 1y ago
A free colab. Here's a link, you can finetune the model in ~5 minutes in this example, and I encourage you to try your own https://ai.google.dev/gemma/docs/core/huggingface_text_full_...
57.
by canyon289 1y ago
If you know you want JSON for sure constrained decoding in an inference framework will help. The model is just one part of an overall inference system. I hope this model paired with other tools help you get done whatever it is you're l
58.
by canyon289 1y ago
This is a key question we faced when building this model. It comes down to basically to "how good" to you need to be at "how many things". We had to make some choices with this model and do our best to maximize performan
59.
by canyon289 1y ago
Daniel, thanks for being here providing technical support as well. Cannot express enough how much we appreciate your all work and partnership.
60.
by canyon289 1y ago
Its hard to tell over the web whether things are sarcastic or not so excuse me if I misread the intent. At Google I've found my colleagues to be knowledgeable, kind, and collaborative and I enjoy interacting with them. This is not just

More ›