33 ms·
Gemma: New Open Models
- smcn 3y agoThere are some pretty impressive benchmarks on https://ai.google.dev/gemma https://ai.google.dev/gemma. Even the 2b model looks fairly not awful? I guess my weekend is going to be spent exploring this.
- alekandreev 3y agoHello on behalf of the Gemma team! We are really excited to answer any questions you may have about our models. Opinions are our own and not of Google DeepMind.
- brucethemoose2 3y agoWill there be "extended context" releases like 01.ai did for Yi? Also, is the model GQA?
- hustwindmaple1 3y agoIt's MQA, documented in the tech report
- nuclearjam 3y agoMay I ask what is the ram requirement for running the 2B model on CPU on an average consumer windows laptop? I have 16 gb RAM but I am seeing CPU/memory traceback. I’m using the transformer implementation.
- zitterbewegung 3y agoDo you have a plan of releasing higher parameter models?
- alekandreev 3y agoWe have many great things in research and development phases, so stay tuned. I’m hopeful we can share more in the coming weeks and month!
- brucethemoose2 3y agoThat is awesome! I hope y'all consider longer context models as well. Also, are ya'll looking alternative architectures like Mamba? Being "first" with a large Mamba model would cement your architectural choices/framework support like llama did for Meta.
- efilife 3y agoThis doesn't answer the question at all
- deleted 3y ago[deleted]
- declaredapple 3y agoCongrats on the launch and thanks for the contribution! This looks like it's on-par or better compared to mistral 7B 0.1 or is that 0.2? Are there plans for MoE or 70B models?
- kathleenfromgdm 3y agoGreat question - we compare to the Mistral 7B 0.1 pretrained models (since there were no pretrained checkpoint updates in 0.2) and the Mistral 7B 0.2 instruction-tuned models in the technical report here: https://goo.gle/GemmaReport https://goo.gle/GemmaReport
- neximo64 3y agoHow are these performing so well compared to Llama 2, are there any documents on the architecture and differences, is it MoE? Also note some of the links on the blog post don't work, e.g debugging tool.
- kathleenfromgdm 3y agoWe've documented the architecture (including key differences) in our technical report here (https://goo.gle/GemmaReport https://goo.gle/GemmaReport), and you can see the architecture implementation in our Git Repo (https://github.com/google-deepmind/gemma https://github.com/google-deepmind/gemma).
- h1t35h 3y agoIt seems you have exposed the internal debugging tool link in the blog post. You may want to do something about it.
- trisfromgoogle 3y agoAh, I see -- the link is wrong, thank you for flagging! Fixing now.
- neximo64 3y agoThe link to the debugging tool is an internal one, no one outside Google can access it
- h1t35h 3y agoThe blog post shares the link for debugging tool as https://*.*.corp.google.com/codelabs/responsible-ai/lit-gemma https://*.*.corp.google.com/codelabs/responsible-ai/lit-gemm... .corp and the login redirect makes me believe it was supposed to be an internal link
- littlestymaar 3y agoSame for the “safety classifier”
- barrkel 3y agohttps://codelabs.developers.google.com/codelabs/responsible-ai/lit-gemma https://codelabs.developers.google.com/codelabs/responsible-...
- deleted 3y ago[deleted]
- deleted 3y ago[deleted]
- deleted 3y ago[deleted]
- 3y ago
- pama 3y agoWill these soon be available on lmsys for human comparison against other models? Can they run with llama.cpp?
- ErneX 3y agoYes to llama.cpp https://twitter.com/ggerganov/status/1760293079313973408 https://twitter.com/ggerganov/status/1760293079313973408
- sbarre 3y agoI came here wondering if these models are "open" in the sense that they'll show up on sites like Ollama where you can download and run them locally. Am I correct to conclude that this means they eventually will? It's unclear to me from Google's docs exactly what "open" means for Gemma
- benpacker 3y agoYes - they are open weights and open inference code, which means they can be integrated into Ollama. They are not “open training” (either in the training code or training data sense), so they are not reproducible, which some have suggested ought to be a component of the definition of open models.
- OJFord 3y agoIt really should shouldn't it? I'm quite ML-naïve, but surely providing the model without 'training code or training data' is just like providing a self-hostable binary without the source code? Nobody calls that open source, it's not even source available.
- sunnybeetroot 3y agoThat’s why they’re called open as in free to use how you wish, not open source where the source of the training is also provided.
- artninja1988 3y agoI find the snyde remarks around open source in the paper and announcement rather off putting. As the ecosystem evolves, we urge the corporate AI community to move beyond demanding to be taken seriously as a player in open source for models that are not actually open, and avoid preaching with a PR statement that can be interpreted as uniformed at best or malicious at worst.
- silentsanctuary 3y agoWhich remarks are you referring to?
- artninja1988 3y agoThe synde remarks at metas llama license that doesn't allow companies with 700 million monthly active users to use it, while this model also doesn't have a really 'open' license itself and also this paragraph: >As the ecosystem evolves, we urge the wider AI community to move beyond simplistic ’open vs. closed’ debates, and avoid either exaggerating or minimising potential harms, as we believe a nuanced, collaborative approach to risks and benefits is essential. At Google DeepMind we’re committed to developing high-quality evaluations and invite the community to join us in this effort for a deeper understanding of AI systems.
- tomComb 3y agoWell, given that that restriction added to the meta-llama license is aimed at Google, is petty, and goes against open source norms, I think it’s reasonable that they should feel this way about it.
- lordswork 3y agoHow is this a snide remark? It's factual and prevented their team from benchmarking against Llama 2.
- trisfromgoogle 3y agoQuick question -- can you tell me where you got that quote? It's not in the main blog or any of the launch communications that I can see.
- tosh 3y agoAre there any plans for releasing the datasets used?
- alekandreev 3y agoThis would be really interesting in my opinion, but we are not releasing datasets at this time. See the C4 dataset for an earlier open dataset from Google.
- sbarre 3y agoCan the Gemma models be downloaded to run locally, like open-source models Llama2, Mistral, etc ? Or is your definition of "open" different?
- kathleenfromgdm 3y agoYes, you can get started downloading the model and running inference on Kaggle: https://www.kaggle.com/models/google/gemma https://www.kaggle.com/models/google/gemma ; for a full list of ways to interact with the model, you can check out https://ai.google.dev/gemma https://ai.google.dev/gemma.
- dartharva 3y agoCan we have llamafile releases as well? https://github.com/Mozilla-Ocho/llamafile https://github.com/Mozilla-Ocho/llamafile
- syntaxing 3y agoA small typo in your model link that breaks it. There’s an extra ; on the end.
- kathleenfromgdm 3y agoCorrected - thanks :)
- Kostic 3y agoIt should be possible to run it via llama.cpp[0] now. [0] https://github.com/ggerganov/llama.cpp/pull/5631 https://github.com/ggerganov/llama.cpp/pull/5631
- nerdix 3y agoAmazing how quickly this happened.
- tomp 3y agoTheir definition of "open" is "not open", i.e. you're only allowed to use Gemma in "non-harmful" way. We all know that Google thinks that saying that 1800s English kings were white is "harmful".
- vorticalbox 3y agoare there plans to release an official GGUF version to use with llama.ccp?
- espadrine 3y agoIt is already part of the release on Huggingface: https://huggingface.co/google/gemma-7b/blob/main/gemma-7b.gguf https://huggingface.co/google/gemma-7b/blob/main/gemma-7b.gg... It is a pretty clean release! I had some 500 issues with Kaggle validating my license approval, so you might too, but after a few attempts I could access the model.
- vorticalbox 3y agoI didn't see this when searching thanks
- sqreept 3y agoWhat are the supported languages of these models?
- alekandreev 3y agoThis v1 model is focused on English support, but you may find some multilingual capabilities.
- lnyan 3y agoWill there be Gemma-vision models or multimodal Gemma models?
- Jayakumark 3y agoHave the same question.
- alekandreev 3y agoWe have many exciting things planned that we can't reveal just yet :)
- CuriouslyC 3y agoIt's cool that you guys are able to release open stuff, that must be a nice change from the modus operandi at goog. I'll have to double check but it looks like phi-2 beats your performance in some cases while being smaller, I'm guessing the value proposition of these models is being small and good while also having more knowledge baked in?
- alekandreev 3y agoWe deeply respect the Phi team and all other teams in the open model space. You’ll find that different models have different strengths and not all can be quantified with existing public evals. Take them for a spin and see what works for you.
- turnsout 3y agoWhat is the license? I couldn’t find it on the 1P site or Kaggle.
- trisfromgoogle 3y agoYou can find the terms on our website, ai.google.dev/gemma: https://ai.google.dev/gemma/terms https://ai.google.dev/gemma/terms
- deleted 3y ago[deleted]
- spiantino 3y agoout of curiosity, why is this a "terms" and not a license? I'm used to reading and understanding the software as coming with a license to use it. Do the terms give us license to use this explicitly?
- turnsout 3y agoThey do, but unlike a known license, these terms are custom and non-standard. Which means I would guide my commercial clients away from this particular model.
- deleted 3y ago[deleted]
- audessuscest 3y agoDoes this model also thinks german were black 200 years ago ? Or is afraid to answer basic stuff ? because if this is the case no one will care about that model.
- freedomben 3y agoI don't know anything about these twitter accounts so I don't know how credible they are, but here are some examples for your downvoters that I'm guessing just think you're just trolling or grossly exaggerating: https://twitter.com/aginnt/status/1760159436323123632 https://twitter.com/aginnt/status/1760159436323123632 https://twitter.com/Black_Pilled/status/1760198299443966382 https://twitter.com/Black_Pilled/status/1760198299443966382
- robswc 3y agoYea. Just ask it anything about historical people/cultures and it will seemingly lobotomize itself. I asked it about early Japan and it talked about how European women used Katanas and how Native Americans rode across the grassy plains carrying traditional Japanese weapons. Pure made up nonsense that not even primitive models would get wrong. Not sure what they did to it. I asked it why it assumed Native Americans were in Japan in the 1100s and it said: > I assumed [...] various ethnicities, including Indigenous American, due to the diversity present in Japan throughout history. However, this overlooked [...] I focused on providing diverse representations without adequately considering the specific historical context. How am I supposed to take this seriously? Especially on topics I'm unfamiliar with?
- trackflak 3y agoFrom one of the Twitter threads linked above: > they insert random keyword in the prompts randomly to counter bias, that got revealed with something else I think. Had T shirts written with "diverse" on it as artifact This was exposed as being the case with OpenAI's DALL-E as well - someone had typed a prompt of "Homer Simpson wearing a namebadge" and it generated an image of Homer with brown skin wearing a namebadge that said 'ethnically ambiguous'. This is ludicrous - if they are fiddling with your prompt in this way, it will only stoke more frustration and resentment - achieving the opposite of why this has been implemented. Surely if we want diversity we will ask for it, but sometimes you don't, and that should be at the user's discretion.\ Another thread for context: https://twitter.com/napoleon21st/status/1760116228746805272 https://twitter.com/napoleon21st/status/1760116228746805272
- lordswork 3y agoIs there any truth behind this claim that folks who worked on Gemma have left Google? https://x.com/yar_vol/status/1760314018575634842 https://x.com/yar_vol/status/1760314018575634842
- CaffeinatedDev 3y agoThem: here to answer questions Question Them: :O
- lordswork 3y agoTo be fair, I think they are in London, so I assume they have winded down for the day. Will probably have to wait ~12-18 hours for a response.
- elcomet 3y agoIt seems very easy to check no? Look at the names in the paper and check where they are working now
- lordswork 3y agoGood idea. I've confirmed all the leadership / tech leads listed on page 12 are still at Google. Can someone with a Twitter account call out the tweet linked above and ask them specifically who they are referring to? Seems there is no evidence of their claim.
- elcomet 3y agoIt's also possible Google removed names of people who left. It's not really a research paper, more a marketing piece, so it might be possible (I don't think they would do that with a conf paper)
- lordswork 3y agoWe'll see if the person making this claim responds with specific Gemma developers that have left. Otherwise, I think it's safe to assume they are just lying.
- memossy 3y agoTraining on 4096 v5es how did you handle crazy batch size :o
- quickgist 3y agoWill this be available as a Vertex AI foundational model like Gemini 1.0, without deploying a custom endpoint? Any info on pricing? (Also, when will Gemini 1.5 be available on Vertex?)
- moffkalast 3y agoI'm not sure if this was mentioned in the paper somewhere, but how much does the super large 265k tokenizer vocabulary influence inference speed and how much higher is the average text compression compared to llama's usual 30k? In short, is it really worth going beyond GPT 4's 100k?
- dmnsl 3y agoHi, what is the cutoff date ?
- legohead 3y agoAll it will tell me is mid-2018.
- alekandreev 3y agoSeptember 2023.
- waych 3y ago[flagged]
- cypress66 3y agoCan you share the training loss curve?
- fosterfriends 3y agoNot a question, but thank you for your hard work! Also, brave of you to join the HN comments, I appreciate your openness. Hope y'all get to celebrate the launch :)
- voxgen 3y agoThank you very much for releasing these models! It's great to see Google enter the battle with a strong hand. I'm wondering if you're able to provide any insight into the below hyperparameter decisions in Gemma's architecture, as they differ significantly from what we've seen with other recent models? * On the 7B model, the `d_model` (3072) is smaller than `num_heads * d_head` (16*256=4096). I don't know of any other model where these numbers don't match. * The FFN expansion factor of 16x is MUCH higher than the Llama-2-7B's 5.4x, which itself was chosen to be equi-FLOPS with PaLM's 4x. * The vocab is much larger - 256k, where most small models use 32k-64k. * GQA is only used on the 2B model, where we've seen other models prefer to save it for larger models. These observations are in no way meant to be criticism - I understand that Llama's hyperparameters are also somewhat arbitrarily inherited from its predecessors like PaLM and GPT-2, and that it's non-trivial to run hyperopt on such large models. I'm just really curious about what findings motivated these choices.
- owl_brawl 3y agoI would love answers to these questions too, particularly on the vocab size
- LorenDB 3y agoEDIT: it seems this is likely an Ollama bug, please keep that in mind for the rest of this comment :) I ran Gemma in Ollama and noticed two things. First, it is slow. Gemma got less than 40 tok/s while Llama 2 7B got over 80 tok/s. Second, it is very bad at output generation. I said "hi", and it responded this: ``` Hi, . What is up? melizing with you today! What would you like to talk about or hear from me on this fine day?? ``` With longer and more complex prompts it goes completely off the rails. Here's a snippet from its response to "Explain how to use Qt to get the current IP from https://icanhazip.com https://icanhazip.com": ``` python print( "Error consonming IP arrangration at [local machine's hostname]. Please try fufing this function later!") ## guanomment messages are typically displayed using QtWidgets.MessageBox ``` Do you see similar results on your end or is this just a bug in Ollama? I have a terrible suspicion that this might be a completely flawed model, but I'm holding out hope that Ollama just has a bug somewhere.
- mark_l_watson 3y agoI was going to try these models with Ollama. Did you use a small number of bits/quantization?
- LorenDB 3y agoThe problem exists with the default 7B model. I don't know if different quantizations would fix the problem. The 2B model is fine, though.
- jmorgan 3y agoHi! This is such an exciting release. Congratulations! I work on Ollama and used the provided GGUF files to quantize the model. As mentioned by a few people here, the 4-bit integer quantized models (which Ollama defaults to) seem to have strange output with non-existent words and funny use of whitespace. Do you have a link /reference as to how the models were converted to GGUF format? And is it expected that quantizing the models might cause this issue? Thanks so much!
- espadrine 3y agoAs a data point, using the Huggingface Transformers 4-bit quantization yields reasonable results: https://twitter.com/espadrine/status/1760355758309298421 https://twitter.com/espadrine/status/1760355758309298421
- kleiba 3y ago> We are really excited to answer any questions you may have about our models. I cannot count how many times I've seen similar posts on HN, followed by tens of questions from other users, three of which actually get answered by the OP. This one seems to be no exception so far.
- spankalee 3y agoWhat are you talking about? The team is in this thread answering questions.
- AlexeyBelov 3y agoOnly simple and convenient ones.
- alekandreev 3y agoSorry, doing our best here :)
- kleiba 3y agoThank you!
- owl_brawl 3y agoHi alekandreev, Any reason you decided to go with a token vocabulary size of 256k? Smaller vocab/vector sizes like most models in this size seem to be using (~16-32k) are much easier to work with. Would love to understand the technical reasoning here that isn't detailed in the report unfortunately :(.
- Havoc 3y agoTaking a page out of metas book with open models. I wonder what the game plan here is. Nice that it allows commercial use!
- gaogao 3y agoMostly to boost research and commercial usage around JAX/Gemini is my read. Any internal research using Gemma is now more easily externally reproducible, external research and frameworks are easier to translate over, goodwill especially from researchers.
- gaogao 3y agoThere's also less of a special sauce for text models itself these days with the propietary being more on the pre-training data and training stack (e.g. how to get 10k GPUs/TPUs running together smoothly). Multi-modal models (or adjacent like Sora) are less likely to be open sourced in the immediate term.
- smarterclayton 3y agoThere is a lot of work to make the actual infrastructure and lower level management of lots and lots of GPUs/TPUs open as well - my team focuses on making the infrastructure bit at least a bit more approachable on GKE and Kubernetes. https://github.com/GoogleCloudPlatform/ai-on-gke/tree/main https://github.com/GoogleCloudPlatform/ai-on-gke/tree/main and https://github.com/google/xpk https://github.com/google/xpk (a bit more focused on HPC, but includes AI) and https://github.com/stas00/ml-engineering https://github.com/stas00/ml-engineering (not associated with GKE, but describes training with SLURM) The actual training is still a bit of a small pool of very experienced people, but it's getting better. And every day serving models gets that much faster - you can often simply draft on Triton and TensorRT-LLM or vLLM and see significant wins month to month.
- sidcool 3y agoAvailable on Ollama?
- blooalien 3y agohttps://ollama.com/library?q=gemma https://ollama.com/library?q=gemma Library search says "Nope". At least not yet.
- tomd 3y agoIt's there now
- kevsim 3y agoAnd now it says "Yup". That was pretty quick!
- blooalien 3y agoDang, that was really quick! According to the listed time of your reply vs. mine, less than an hour from the time I checked? Quick turnaround indeed. Already been pulled from there over 3,700 times since then, too (as of the time of this reply mere hours later). Seems like quite a bit more'n a few Ollama users were "waitin' with bated breath" for that one to drop. :grin:
- SushiHippie 3y agoSupport for gemma in llama.cpp just got merged, so it may take some time (could be hours or days) until this lands in ollama https://github.com/ggerganov/llama.cpp/pull/5631 https://github.com/ggerganov/llama.cpp/pull/5631
- dcchambers 3y agoIt's now in the 0.1.26 pre-release: https://github.com/ollama/ollama/releases/tag/v0.1.26 https://github.com/ollama/ollama/releases/tag/v0.1.26
- chown 3y agoAvailable in pre-release now which means you’d have to update manually in future.
- mustafabisic1 3y agoThe fact Gemma team is in the comments section answering questions is praiseworthy to me :)
- p1esk 3y agohttps://twitter.com/yar_vol/status/1760314018575634842 https://twitter.com/yar_vol/status/1760314018575634842
- pphysch 3y agoWhy is this anonymous tweet with no evidence or engagement being posted by multiple users in this thread? Why not just make the same claim directly?
- gregorygoc 3y agoProgramming popularized -> more people -> more cases of knee jerk reaction encountered. Most programmers are really not that smart nowadays. I’ve seen too many cases of people throwing around claims without a second of deep and critical thought.
- callalex 3y agoThe link is broken. On HN (or any forum really) it is expected for a brief description of the content to be provided when posting a link. Links die all the time, but forum posts don’t have to die with them.
- carom 3y agoI've worked at Google. It is the organization with highest concentration of engineering talent I've ever been at. Almost to the point that it is ridiculous because you have extremely good engineers working on internal reporting systems for middle managers.
- ilc 3y agoIf everyone is great. Someone has to draw the short straw. At MIT they said: You know the kid who sat at the front of the room. Now you are with ALL of the kids who sat in the front of the room. Guess what? There's still going to be a kid who sits at the front of the room. I'd imagine Google or anyplace with a stiff engineering filter will have the same issues.
- deleted 3y ago[deleted]
- Kelteseth 3y agoCan this run on my AMD Vega VII on Windows 11? As always, AMD is missing: > Optimization across multiple AI hardware platforms ensures industry-leading performance, including NVIDIA GPUs and Google Cloud TPUs.
- lordswork 3y agoAMD Vega VII meets the memory requirements. Once tools like LM Studio, ollama, etc. add support for the model, you should be able to run locally like you would any other open weights model.
- GaggiX 3y agoThey have implemented the model also on their own C++ inference engine: https://github.com/google/gemma.cpp https://github.com/google/gemma.cpp
- 0xbadc0de5 3y agoThank you for releasing this.
- vanderboyd 3y agoThe 2B model seems underwhelming. For instance, compared to the recent StableLM2 1.6B model that is slightly smaller and probably wastes some "English metric points" by being multilingual. The latter (and other similar open models) seem to do similarly well in benchmarks (much better in Math?) with way less fancy stuff. For instance, public data and no secretive filtering with pre trained models or synthetic data. My take is that using the vanilla approaches take you really far. And many of the latest tricks and hours-of-work buy you little... Will be interesting to see how this plays out, especially for the open source community.
- impulser_ 3y agoGo back 5 years and ask anyone on this site what companies do you think will be the most open about AI in the future OpenAI, Meta, or Google. I bet 10/10 people would pick OpenAI. Now today Meta and Google, both trillion dollars companies, are releasing very powerful open models with the ability to be used commercially. Ironic.
- phillipcarter 3y agoI would have picked Google five years ago, since nobody was releasing commercially viable LLMs at the time, and Google was the center of all the research that I knew of.
- infecto 3y agoIronic but I wonder how true this would be if Google was first to market.
- deleted 3y ago[deleted]
- vmfunction 3y agoNot surprising, just like when MS went to shit, and then they start to embrace 'open source'. Seems like PR stunt. And when it comes to LLM there is millions of dollar barrier to entry to train the model, so it is ok to open up their embedding etc. Today big corp A will open up a little to court the developers, and tomorrow when it gains dominance it will close up, and corp B open up a little.
- nalzok 3y agoCongratulations on the release! How can we download the model and run inference locally?
- kathleenfromgdm 3y agoThank you! You can get started downloading the model and running inference on Kaggle: https://www.kaggle.com/models/google/gemma https://www.kaggle.com/models/google/gemma ; for a full list of ways to interact with the model, you can check out https://ai.google.dev/gemma https://ai.google.dev/gemma.
- aphit 3y agoFYI the ; broke the link, but I found it easily anyway.
- kathleenfromgdm 3y agoGood catch - just corrected. Thanks!
- austinvhuang 3y agoYou can download the model checkpoints from kaggle https://www.kaggle.com/models/google/gemma https://www.kaggle.com/models/google/gemma and huggingface https://huggingface.co/blog/gemma https://huggingface.co/blog/gemma Besides the python implementations, we also implemented a standalone C++ implementation that runs locally with just CPU simd https://github.com/google/gemma.cpp https://github.com/google/gemma.cpp
- tveita 3y agoAre there any cool highlights you can give us about gemma.cpp? Does it have any technical advantages over llama.cpp? It looks like it introduces its own quantization format, is there a speed or accuracy gain over llama.cpp's 8-bit quantization?
- 3y ago
- espadrine 3y agoI notice a few divergences to common models: - The feedforward hidden size is 16x the d_model, unlike most models which are typically 4x; - The vocabulary size is 10x (256K vs. Mistral’s 32K); - The training token count is tripled (6T vs. Llama2's 2T) Apart from that, it uses the classic transformer variations: MQA, RoPE, RMSNorm. How big was the batch size that it could be trained so fast? https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.2/blob/main/config.json https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.2/bl...
- GaggiX 3y agoLooking at the config.json of Gemma 7B the feedfoarward hidden size is 8x, not 16x
- espadrine 3y agoHuh, indeed, that's what the config.json[0] says; the report[1] indicates “Feedforward hidden dims: 49152”. [0]:https://huggingface.co/google/gemma-7b-it/blob/main/config.json https://huggingface.co/google/gemma-7b-it/blob/main/config.j... [1]: https://storage.googleapis.com/deepmind-media/gemma/gemma-report.pdf https://storage.googleapis.com/deepmind-media/gemma/gemma-re...
- GaggiX 3y agoI don't see the number 49152 reported in the config.json, what line are you referring to? I just see the intermediate_size of 24576 (so 8x). EDIT: I didn't read the comment correctly, you have noticed the same thing.
- SahAssar 3y agoRead the parent comment again. It says the paper says 49152, not the config.json.
- deleted 3y ago[deleted]
- 3y ago
- margorczynski 3y agoIs there a chance we'll get a model without the "aligment" (lobotomization)? There are many examples where answers from Gemini are garbage because of the ideological fine tuning.
- yakorevivan 3y agoThey have released finetuning code too. You can finetune it to remove the alignment finetuning. I believe it would take just a few hours at max and a couple of dollars.
- FergusArgyll 3y agoYou can (and someone will) fine tune it away. There are datasets which are foss you can use on hugging face. Or you can just wait, it'll be done soon...
- declaredapple 3y agoYou can but it'll never be the same as the base model. That said it appears they also released the base checkpoints that aren't fine-tuned for alignment
- joshelgar 3y agoCould you give an example of these datasets?
- FergusArgyll 3y agoI think they should be easy to find (I never actually used one, but I keep on seeing references...) here's one https://huggingface.co/datasets/cognitivecomputations/WizardLM_alpaca_evol_instruct_70k_unfiltered https://huggingface.co/datasets/cognitivecomputations/Wizard...
- FergusArgyll 3y agohttps://huggingface.co/datasets/Fredithefish/openassistant-guanaco-unfiltered https://huggingface.co/datasets/Fredithefish/openassistant-g...
- tosh 3y agoBenchmarks for Gemma 7B seem to be in the ballpark of Mistral 7B +-------------+----------+-------------+-------------+ | Benchmark | Gemma 7B | Mistral 7B | Llama-2 7B | +-------------+----------+-------------+-------------+ | MMLU | 64.3 | 60.1 | 45.3 | | HellaSwag | 81.2 | 81.3 | 77.2 | | HumanEval | 32.3 | 30.5 | 12.8 | +-------------+----------+-------------+-------------+ via https://mistral.ai/news/announcing-mistral-7b/ https://mistral.ai/news/announcing-mistral-7b/
- brucethemoose2 3y agoOnly 8K context as well, like Mistral. Also, as always, take these benchmarks with a huge grain of salt. Even base model releases are frequently (seemingly) contaminated these days.
- tosh 3y agoAgree: will be interesting how Gemma does on ChatBot Arena
- DreamGen 3y agoMistral Instruct v0.2 is 32K.
- Kydlaw 3y agoThey state in their report that they filter evaluation data off their training data, see p.3 - Filtering: "Further, we filter all evaluation sets from our pre-training data mixture, run targeted contamination analyses to check against evaluation set leakage, and reduce the risk of recitation by minimizing proliferation of sensitive outputs."
- deleted 3y ago[deleted]
- zdimension 3y agoNice to see more open models. Props to the team for coming to the HN comment section to answer questions
- rvz 3y agoGreat! Google is now participating in the AI race to zero with Meta, as predicted that $0 free AI models would eventually catch up against cloud-based ones. You would not want to be in the middle of this as there is no moat around this at all. Not even OpenAI.
- rvnx 3y agoAbout 5 months until we see widespread local LLMs, thanks to Apple.
- dingclancy 3y agoApple needs to be known as an AI leader first.
- thejohnconway 3y agoWhy?
- rvz 3y agoAbsolutely this.
- dingclancy 3y agoLLM is the dumb pipe but so far ChatGPT is the most successful generative AI product. It remains to be seen. OpenAI’s models are barely leading Gemini Ultra now, but as chat product it is still miles ahead of the Gemini interface.
- rvnx 3y agoThe main problem of Gemini 1.5 is that you cannot access it at all as a user :|
- staticman2 3y agoIf meta keeps spending tens of millions of dollars each year to release free AI models it might seem like there is no moat, but under normal circumstances wouldn't the cost to develop a free model be considered a moat?
- ijustlovemath 3y agoHope to see support for this in ollama soon!
- ericskiff 3y agoHas anyone found the context length for these models yet? So far I haven't seen it mentioned in their write-up or the model card
- kathleenfromgdm 3y agoThe context length for these models is 8192 tokens.
- minimaxir 3y agoFor posterity, an easy way to find the context length of a LLM hosted on Hugging Face is to look at the max_position_embeddings in the config.json, which shows the 8192 mentioned in another comment. (although in this case you need to sign the agreement first)
- brucethemoose2 3y agoThere are some exceptions, like Mistral 0.1 (which is technically 32K according to the config but practically 8K because the sliding window is awful) and InternLM (which (at least initially) used auto rope scaling to extend the context as part of the model's architecture).
- minimaxir 3y agoYes, RoPE has thrown a wrench into things a bit.
- xena 3y agoWhat is the context window?
- kathleenfromgdm 3y agoThe context length for these models is 8192 tokens.
- jerrygenser 3y agoLooking forward to Gemma 7bx8 moe
- AlbatrossAA 3y ago[dead]
- neximo64 3y agoIs this the Deepmind guy in Google more now? what a change the past year has made
- deleted 3y ago[deleted]
- DebtDeflation 3y agoHopefully not totally gimped like Gemini. Are they releasing an uncensored version?
- dougmwne 3y agoThese are downloadable open models that can be fined tuned. They are the opposite of censored. If you have the motivation, you can bias them however you please.
- willy_k 3y agoIs “the opposite of censored” accurate for something that’s default and considerably easier to access mode of operation won’t say many things for sociopolitical reasons? Able to be un censored, sure, but the extent of that is debatable as well.
- dougmwne 3y agoThere is no default and easy access mode. These are raw model weights and only enthusiasts and researchers will download the necessary packages to run it locally. Much more likely is that some popular fine tunes show up on hugging face for more general access.
- willy_k 3y agoI agree that there probably will be “uncensored” fine tuned models that become available, my point was just that it’s not accurate to call Gemma “the opposite of censored” because there is a somewhat involved step that needs to be taken before it even appears uncensored. It’s also likely missing a lot of useful context that was removed from the training set and not meaningfully replaced during fine-tuning, and besides that any fine tuned “uncensored” model will be based on Gemma, not Google’s Gemma itself. IMO “Opposite of uncensored” suggests a model whose original form eagerly gives out controversial / typically censored information, not a model that is censored but able to be fine tuned away from censorship.
- 3y ago
- Workaccount2 3y agoIs it pronounced jem-a or ghem-a?
- davidmurdoch 3y agoProbably "Jemma" (the superior spelling of the name). It's a play on their "Gemini" product.
- pfooti 3y agoIt's pronounced like "gif".
- milliams 3y agoThey're really trying hard to avoid saying what kind of "models" these are. I think they're language models, but it's hard to say for sure.
- lordswork 3y agoYou're right that they don't call them language models. The technical report says: Gemma models demonstrate strong performance across academic benchmarks for language understanding, reasoning, and safety. Maybe they are reserving the right to expand Gemma model family to multi-modal models.
- AlbatrossAA 3y ago[dead]
- hawk01 3y agoCan't wait to try it out with ollama locally
- FergusArgyll 3y agoSomeone should try to make a MOE of 2b models
- w4 3y agoParameter counts notwithstanding, it’s an objectively funny outcome that Meta, Microsoft, and Google are all releasing cutting edge open models, while OpenAI keeps theirs closed source.
- spacebanana7 3y agoIt's ironic but actually follows their business interests. Microsoft & google have large cloud divisions that benefit from open models. The lower the cost of AI models, the more they get run and the greater the cloud spend. Meta is a consumer of AI. They themselves want cheap and effective AI for targeting adverts and building metaverses. A loose analogy is that both oil producers and car companies want refining to be cheap.
- deleted 3y ago[deleted]
- deleted 3y ago[deleted]
- anshumankmr 3y agoAre these any good? I have been trying the non pro version of Gemini, and that seems awful at code generation. I am more keen on getting access to the best model and I would pay for it if I wasn't already paying for ChatGPT 4.
- brucethemoose2 3y agoYou should be looking at Deepseek's coding models, and finetunes of those. I run 33B on my desktop, and find it to be sufficient for many tasks.
- anshumankmr 3y agoWhat card do you have? Don't have a desktop myself.
- robswc 3y agoI often talk with GPT4 on road trips about topics I'm interested in. Its great for passing the time. I tried the same thing with Gemini and its full of nonsense. I was talking with it about the "Heian period" of Japan and it made up all sorts of stuff but you really only could tell because it was so ridiculous. Talked about European women and Native Americans roaming around the famous grassy plains of japan wielding katana and traditional weaponry... in the 1100s. No such issue with GPT4. I haven't tried it with code though, since I already have co-pilot. Really hard to trust anything it says after it started making stuff up about such a simple time period.
- wantsanagent 3y agoThe utter bullshit of these licenses has got to stop. Do not, under any circumstances, consider using these commercially. "Google reserves the right to restrict (remotely or otherwise) usage of any of the Gemma Services that Google reasonably believes are in violation of this Agreement." This is a kill switch that Google maintains in perpetuity over any system you build relying on these models. Our legal review of the Llama license came to the same conclusion, we cannot rely on the goodwill of Meta for any core service, and we shouldn't rely on the same from Google. Now, perhaps less materially important, but just as infuriating is the "Prohibited Use[s]". These cover just enough to placate the most sensitive, but omit any real harms (waging war, developing weapons) that coincidentally have massive commercial value. Use the model to build a biological weapon (as an authorized govt official)? Cool. Use it to play a prank that deceives someone? Policy violation. And of course, as the coup de grâce, they throw in a DMCA style provision to make sure you can't modify the models in any way that could cause them to violate their kid-glove precepts.
- candiddevmike 3y agoCould you share what models you consider to be OK for commercialization?
- wantsanagent 3y agoMistral series in particular but those with OSI approved licenses such as Apache 2.0, MIT, etc.
- stale2002 3y agoWait, you actually care about the license and read it? It's seems like you aren't up to date. Most of the startup space is entirely ignoring all these licenses. If the weights are available, it is being used commerically without regards to any licensing. And everyone is getting away with it and nobody is being sued. Good luck trying to keep up if you aren't doing the same! Feels free to hamstring yourself though if you like.
- hansonpeter 3y ago[dead]
- zemo 3y ago> Open models feature free access to the model weights, but terms of use, redistribution, and variant ownership vary according to a model’s specific terms of use, which may not be based on an open-source license. does a model being "open" say anything about how it was trained?
- dcchambers 3y agoAlready available in Ollama v0.1.26 preview release, if you'd like to start playing with it locally: - https://github.com/ollama/ollama/releases/tag/v0.1.26 https://github.com/ollama/ollama/releases/tag/v0.1.26
- jmu1234567890 3y agoI wonder if people will get confused with the naming Gemma, Gemini pro, Gemini advanced, Gemini ultra To a layperson it is not obvious which one is better than the other
- marban 3y agoUnbefkglievable — Another week, another new name?
- sqreept 3y agoTried inference with the 7B model and without flash attention this is soooooo slow. With flash attention the fine-tunning requires A100 or H100. Also the inference doesn't always stop generating resulting in garbage being added to the response.
- brucethemoose2 3y ago> Also the inference doesn't always stop generating resulting in garbage being added to the response. That sounds like a chat format misconfiguration. This could partially be Google's fault, as they used yet another novel prompting format. Also, for sane inference speed on H100s, you'll have to wait for architecture support from the optimized frameworks. Vanilla transformers is beyond awful even with FA2.
- alekandreev 3y agoWe have implementations in different ML frameworks, so I am not quite sure which one you are referring to. Would you like to file a bug at the relevant GitHub repo?
- sqreept 3y agoFirst of all, I'm using 2 x 4090 for testing. 4090 has 16384 CUDA cores which will become relevant a bit later. I dug a bit deeper and it seems that with transformers==4.37.0 everything works fine with other HF hosted models (like Llama) but you'll rightfully get this when trying to use Gemma: ImportError: cannot import name 'GemmaForCausalLM' from 'transformers' After installing transformers==4.38.0 the fine-tunning speed of Llama drops to 25% (?!?) of what used to be for a reason that I think HF should fix. Testing Gemma it seems I'm hitting a hardware limit as Gemma has a hidden size which is bigger than the available CUDA cores. This seems to make both inference & fine-tunning about 25 times slower than similarly sized Llama 7B. I guess some operations have to be broken down in multiple round trips to the GPU due to my low CUDA core count. All in all, even if HF fixes the recently introduced slowdown, Gemma seems to be fine-tuneable in reasonable amount of time only by the lucky ones with access to A100/H100. EDIT: I managed to hack my env to be able to run inference on Gemma with transformers==4.37.0 by keeping the necessary classes in loaded in RAM. It works about 4x faster but still very slow. And both the 7B and the 2B versions behave the same way. EDIT2: I tried latest transformers from main branch (4.39.0.dev) and behaves the same as 4.38.0.
- smpanaro 3y agoHas perplexity fallen out of favor? I didn't see it mentioned anywhere. I tried using lm-eval for the 2B model but the results seem wrong (46.1288).
- 7moritz7 3y agoThr landing page on ai.google.com seems to be machine translated, for Huggingface it uses the literal German translation (Umarmungen Gesicht)
- chown 3y agoIf you are looking for a nice chat UI to try out Gemma (and other offline + online models) locally, I'm working on an app [1] that is offline and privacy focused. I've just added support for Gemma 7B. [1]: https://msty.app https://msty.app
- Alifatisk 3y agoI wish I could install it through chocolatey
- chown 3y agoSure. I would love to add support for that. I had someone else asking for it too. Will be supporting it very soon.
- dhbradshaw 3y agoHandy app for model testing! One usage question: after you've downloaded a model and are finished trying it out, how do you remove it?
- chown 3y agoThanks! If you go to where you installed the model from and click on the download button, you can install additional models or remove installed models. Now that I think of it, it could be a bit confusing. Thanks for asking, I feel like I need to improve this a bit.
- dizhn 3y agoWhat's the license of the software?
- modelx 3y agoThey also implemented it in PyTorch. Cool! https://github.com/google/gemma_pytorch https://github.com/google/gemma_pytorch
- modelx 3y agoThey also implemented in PyTorch. Cool. https://github.com/google/gemma_pytorch https://github.com/google/gemma_pytorch
- brrrrrm 3y agoIt looks like it's pretty resistant to quantization. ollama 4bit 7B doesn't work very well, but the 16bit 2B does
- petercooper 3y agoThat's useful to know. My experiments with the 4bit 7B currently tagged for use on ollama are not going well at all. Lots of refusals and junk. Downloading 7b-instruct-fp16 now! :-) (Update: Yes, much better, though much slower too, of course.)
- janwas 3y agoFYI we have an experimental 4.5-bit format (NUQ) in gemma.cpp, our C++ inference engine, which from my testing works reasonably well.
- petercooper 3y agoThanks, I'm going to give that a go too. The good news is the very latest Ollama update (0.1.27) has resolved things (though I also had to redownload the quant model for 7B).
- simonw 3y agoThe terms of use: https://ai.google.dev/gemma/terms https://ai.google.dev/gemma/terms and https://ai.google.dev/gemma/prohibited_use_policy https://ai.google.dev/gemma/prohibited_use_policy Something that caught my eye in the terms: > Google may update Gemma from time to time, and you must make reasonable efforts to use the latest version of Gemma. One of the biggest benefits of running your own model is that it can protect you from model updates that break your carefully tested prompts, so I’m not thrilled by that particular clause.
- tgtweak 3y agoI don't think there's a way they can enforce that reasonably. There's no connection to the mothership to report back what version is being used or license keys at runtime... Seems more like a "if we discover something unsafe you should update your model and we aren't liable if you don't" than something that would make your model stop working.
- legohead 3y agoSounds like it's "reasonable" for you not to update then.
- wahnfrieden 3y agoIt says you must make efforts (to a reasonable extent), not that you must give a reason for not making efforts
- reissbaker 3y agoThis is a TOS, meaning their enforcement option is a lawsuit. In court, if you convincingly argue why it would take an unreasonable amount of effort to update, you win. They can't compel you to unreasonable effort as per their own TOS.
- generalizations 3y agoThis assumes they even know that the model hasn't been updated. Who is this actually intended for? I'd bet it's for companies hosting the model. In those cases, the definition of reasonable effort is a little closer to "it'll break our stuff if we touch it" rather than "oh silly me, I forgot how to spell r-s-y-n-c".
- Alifatisk 3y agoThis is such a powerful move!
- circusfly 3y agoGemma, Mistral, I feel like Rip van Winkle, asleep for 20 years only to wake up and find the whole tech world changed.
- spiantino 3y agoMaybe a dumb question, but why is there a Terms instead of a license? That feels a little flimsier as an open source offering
- robswc 3y agoI personally can't take any models from google seriously. I was asking it about the Japanese Heian period and it told me such nonsensical information you would have thought it was a joke or parody. Some highlights were "Native American women warriors rode across the grassy plains of Japan, carrying Yumi" and "A diverse group of warriors, including a woman of European descent wielding a katana, stand together in camaraderie, showcasing the early integration of various ethnicities in Japanese society" Stuff like that is so obviously incorrect. How am I supposed to trust it on topics where such ridiculous inaccuracies aren't so obvious to me? I understand there will always be an amount of incorrect information... but I've never seen something this bad. Llama performed so much better.
- cooper_ganglia 3y agoI wonder if they have a system prompt to promote diversity in outputs that touch on race at all? I’ve seen several instances of people requesting a photo of a specific people, and it adds in more people to diversify. Not inherently bad, but it is if it forces it to provide incorrect answers like in your example.
- robswc 3y agoThat's what I don't understand. I asked it why it assumed Native Americans were in Japan and it said: > I assumed [...] various ethnicities, including Indigenous American, due to the diversity present in Japan throughout history. However, this overlooked [...] I focused on providing diverse representations without adequately considering the specific historical context. I see no reason why this sort of thing won't extend to _all_ questions/prompts, so right now I have 0 reason to use Gemini over current models. From my testing and use, it isn't even better at anything to make fighting with it worth it.
- sorokod 3y agoPretty funny as Japan is known to be one of the least ethnically diverse countries in the world.
- 3y ago
- vonwoodson 3y agoThe scariest difference between OpenAI and Google right now is: Ask Gemini who owns the code it writes, and it'll confidently say that Google does. Ask OpenAI, and it'll say that you do. It's that easy to choose which one is the better decision.
- pseudosavant 3y agoConsidering the nuanced nature of copyrighting AI outputs, it isn't clear that either answer is correct.
- wouldbecouldbe 3y agoI really don't get why there is this obsession with safe "Responsible Generative AI". I mean it writes some bad words, or bad pics, a human can do that without help as well. The good thing about dangerous knowledge and generative AI is that you're never sure haha, you'd be a fool to ask GPT to make a bomb. I mean it would probably be safe, since it will make up half of the steps.
- refulgentis 3y agoI guess what I'd tell you is, there's a lot of fools in this world.
- deleted 3y ago[deleted]
- myaccountonhn 3y agoBecause otherwise stuff like this happens, and you get (rightfully) upset customers: https://www.theguardian.com/technology/2018/jan/12/google-racism-ban-gorilla-black-people https://www.theguardian.com/technology/2018/jan/12/google-ra... https://www.bbc.com/news/technology-58462511 https://www.bbc.com/news/technology-58462511 Also people are using LLMs to learn (horrifying but reality), it would be unresponsible for them to let it to propagate negative stereotypes and biases.
- wouldbecouldbe 3y agoBut that's exactly because it's trying to be righteous.
- pradn 3y agoBias is a real problem, but more than that - an adversarial press and public won't forgive massive brands like Google for making AIs that spit out racist answers.
- IceHegel 3y agoGoogle, at the moment, is a tech company whose products are actively engaged in the falsification of history for political purposes. I honestly have no idea where they are going with this but I don't want to be part of it.
- BryanLegend 3y agoAndrej Karpathy's take from twitter. (https://twitter.com/karpathy/status/1760350892317098371 https://twitter.com/karpathy/status/1760350892317098371) Seeing as I published my Tokenizer video yesterday, I thought it could be fun to take a deepdive into the Gemma tokenizer. First, the Gemma technical report [pdf]: https://storage.googleapis.com/deepmind-media/gemma/gemma-report.pdf https://storage.googleapis.com/deepmind-media/gemma/gemma-re... says: "We use a subset of the SentencePiece tokenizer (Kudo and Richardson, 2018) of Gemini for com- patibility. It splits digits, does not remove extra whitespace, and relies on byte-level encodings for unknown tokens, following the techniques used for both (Chowdhery et al., 2022) and (Gemini Team, 2023). The vocabulary size is 256k tokens." The tokenizer.model file is with this code release: https://github.com/google/gemma_pytorch/blob/main/tokenizer/tokenizer.model https://github.com/google/gemma_pytorch/blob/main/tokenizer/... I decoded this model protobuf in Python and here is the diff with the Llama 2 tokenizer: https://diffchecker.com/TRnbKRMH/ https://diffchecker.com/TRnbKRMH/ Notes: - vocab size is quite large: 32K -> 256K - add_dummy_prefix is False. Different from Llama but consistent with GPT. This is a bit more consistent w.r.t. "leave the data alone", as there is no preprocessing step that adds a space to the encoding text. - the model_prefix is the path of the training dataset, which is amusing to look at: "/cns/mf-d/home/gemini-data-access/tokenizers/final_v1_51GB_run1/bpe_coverage_0_999995_v5/255969". Seems to indicate the tokenizer training corpus was ~51GB (?). - a lot of user_defined symbols (i.e. special tokens) are present, e.g. "hardcoding" a sequence of up to 31 newlines as tokens, and a large number of other unclear tokens. I tried decoding the octal representations but it's not clear what's happening here. Also a lot of more special tokens for what look like html elements, e.g. <table>, <tr>, <td>, <i>, <b>, etc. Not 100% sure what the unused tokens are for, maybe this is pre-allocated space to make easier future finetunes that try to add more special tokens, as there is no need to resize vocabularies and perform model surgeries (?). TLDR this is basically the Llama 2 tokenizer, except bigger (32K -> 256K), with a lot more special tokens, and the only functional departure is that add_dummy_prefix is turned off to False. So e.g. tokenizing: "hello world" becomes: [17534, 2134] ['hello', 'world'] which otherwise would have been preprocessed to " hello world" (note leading space) and tokenized as: [25612, 2134] ['hello', 'world'] cool
- exacube 3y agommm.. > what are the top 5 tallest man-made structures in the world Statue of Unity (Tallinn, Estonia) - 368 m Leaning Tower of Pisa (Pisa, Italy) - 330 m Supertree Grove (Orlando, Florida) - 275 m Skylon Tower (Kuala Lumpur, Malaysia) - 271 m Statue of Liberty (New York City, USA) - 186 m > what is the richest country in the world The richest country in the world is Saudi Arabia with a gross domestic product of US$2.3 trillion. > how many cm are in a ft A foot is equal to 0.0254 metres (ft).
- rany_ 3y agoI don't have this issue, what are you using to run the model? These are the answers I got out of it (first output, no cherry-picking): 1. The tallest man-made structure in the world is the Burj Khalifa, standing at 828 meters tall. The Burj Khalifa is a skyscraper in Dubai, United Arab Emirates. Here are the top 5 tallest man-made structures in the world, according to the Guinness World Records: Burj Khalifa (828 m) Taipei 101 (611 m) Merdeka Towers (599 m) Shanghai Tower (632 m) Petronas Towers (551 m) 2. According to the World Bank, the richest country in the world is Saudi Arabia with a gross domestic product (GDP) of $3944 billion.The richest country in the world is Saudi Arabia with a gross domestic product (GDP) of $3944 billion. 3. A foot (ft) is equal to 12 inches (in). There are 2.54 cm in an inch, so there are 2.54 cm x 12 = 30.48 cm in a foot.
- stochastimus 3y ago> Go to Google announcement > Find “license” in page: no matches > Go to HN thread > Find “license” in page: 28 matches > Read a few sigh could have been exciting
- mark_l_watson 3y agoNice, more choices are good. I just saw that the Ollama project already has these models available (date stamp is 58 minutes ago), so I will use that rather than Colab (I live Colab, but I like to run stuff locally).
- th0ma5 3y ago"Carefully tested prompts" sounds a lot like "these are the lotto numbers we know are right" kind of thing? How in the world are these things used for anything programmatically deterministic?
- nojvek 3y agoI applaud the Google team openly engaging on HN here. Q: how sure are you that the newer models trained from trillions of tokens - a huge chunk of open web, hasn't been accidentally polluted by slurping test data?
- ofermend 3y agoGemma-7B (instruction tuned version) is now on the Vectara HHEM leaderboard, with 100% answer rate and 7.5% hallucination rate. Pretty good for a model with 7B params. https://huggingface.co/spaces/vectara/leaderboard https://huggingface.co/spaces/vectara/leaderboard
- smusamashah 3y agoIs there any research on using smaller, lower capability models to act comparable to high quality models? Even if it's just prompt engineering or doing lots of attempts to accomplish the task? If somehow that is possible it means we only need a capable enough model and can use it reliably for lots of practical things.
- Aiedail 3y agoThis is commendable, but there's room for improvement. Up until now, SOTA-level "open-source" LLM models (LLaMA, Mistral, etc.) have usually only made their inference code and model architecture public. While these elements are not insignificant, they are somewhat trivial when compared to the training code and training datasets, as these two factors largely determine the performance of the model. This is not open at all. It goes without saying that sharing the training datasets and process with other AI researchers is crucial. This transparency would not only help to improve the model(for others could contribute to it) but also contribute to the whole community, as they usually advertised. Otherwise, it will be difficult for these efforts to truly promote the development of LLM.
- Jonanin 3y agoIt is surprising how willing Google is to stretch the truth in the marketing for its AI initiatives. Although it's being compared against 7B models, Gemma "7B" is actually much more than 8B parameters in total.
- yudhanjaya 3y agoI’m surprised TheBloke hasn’t quantized this yet.
- EarWig100 3y ago[dead]
- greatNespresso 3y agoWhat caught my attention was the debugging tool, it looks new to me to provide such a utility, is it?
- dmezzetti 3y agoHopefully, they re-release this under an open license. Making everyone go through the exercise to authenticate and agree to terms hasn't worked for any model to-date. It just limits it's reach. We saw the same thing with Phi-2.
- alexwhb 3y agoI find it a bit disheartening that the new wave of “open source” with regards to AI is open weights. That’s like giving someone compiled and obfuscated binaries and saying that’s open source.