17 ms·
RedPajama: Reproduction of LLaMA with friendly license
- miohtama 4y agoThey missed the chance to call it OpenPajama
- almost_usual 4y agoName is obviously inspired from the Anna Dewdney children’s books.
- michael_j_ward 4y agoMy kids love that book, and my oldest had me read it to his preschool class earlier this year. Here is a much more creative reading by Ludacris [0] [0] https://www.youtube.com/watch?v=PFtHeo7oMSU https://www.youtube.com/watch?v=PFtHeo7oMSU
- return_to_monke 4y agowith both this and https://Open-Assistant.io https://Open-Assistant.io, I believe we have entered the Stable Diffusion era of large language models
- idle_zealot 4y agoDidn't Open Assistant just announce that they weren't releasing their model weights due to safety concerns? Seems like another "Open" AI initiative.
- deleted 4y ago[deleted]
- circuit10 4y agoUnless something changed, I thought it was that they literally cannot legally release the weights that are based on LLaMA (except maybe with an xor thing) so they’re going to train it based on something else
- mindcrime 4y agoIs any of the Open Assistant stuff based on LLaMA? I thought they release (at least some version) before LLaMA even dropped?
- circuit10 4y agoYes, there’s also something based on Pythia but it’s a smaller model
- selfhoster11 4y agoIIRC, the video said they will train it on a properly open-source model as well.
- akiselev 4y agoThat was a joke in the release video. The Pythia model is already released at [1] and the deltas for the LLaMa model should be up here [2] in the next few days. [1] https://huggingface.co/OpenAssistant/oasst-sft-4-pythia-12b-epoch-3.5 https://huggingface.co/OpenAssistant/oasst-sft-4-pythia-12b-... [2] https://huggingface.co/OpenAssistant/oasst-llama-based-models https://huggingface.co/OpenAssistant/oasst-llama-based-model...
- bugglebeetle 4y agoOnly if they actually start performing at the level of OpenAI’s models. I’m not a fan of StableDiffusion, but at least their models work at general parity with private offerings. All the LLama derivatives and OpenAssistant stuff performs far below GPT-3.5 for everything I’ve tested.
- GaggiX 4y ago>I’m not a fan of StableDiffusion For some technical reason?
- bugglebeetle 4y agoNo, technically it’s all very impressive. My displeasure with them was there doing a Napster-style maneuver to force artists into accepting AI art generation
- pluijzer 4y agoDo you mean the use of uncredited use of artists artwork without paying royalties for the training set or AI art generation in general?
- bugglebeetle 4y agoWhat I mean is releasing a free service out into the world that allows anyone to effectively pirate an artist’s work. Their intention was obviously to be rewarded by established players for doing this bit of dirty work, forcing artists to accept terms they wouldn’t have otherwise.
- CuriouslyC 4y agoThe training was legal, and artists don't have a say under the current law, so your analogy doesn't hold.
- bugglebeetle 4y ago
- HopenHeyHi 4y ago[flagged]
- thrtythreeforty 4y agoI'm very glad people are starting to push back against claims of various LLMs being open source. I was beginning to be worried that the term would be forcefully redefined in the ML space to mean "weights available." With the kickoff of projects like this and Databricks' Dolly, I'm heartened to see the community saying "no, we are willing to spend the compute to make actually open models." (While it's true that the actual model code of Llama is properly open source, it's also useless for inference by itself. Claiming these models are open source seems like having your cake and eating it too - you get accolades for "open sourcing" but still get to control what happens with it.)
- ninjin 4y agoI can only agree. The number of times we have seen corporations abuse “open source” and “open science” in the context of large language models have been baffling: OPT/LLaMA disallowing commercial usage, BLOOM having an ethical non-open license, GLM having a clause not to “undermine [the People’s Republic of China’s] national security and national unity”, etc. Every single one of these models have been happy to ride on the coattails of the hard work of the open movements by calling themselves open, while only paying lip service to the ideals and definitions underpinning them. While RedPajama has yet to commit to a license (from what I can see, it is late at night…), they are making all the right noises and I am hopeful that my prediction that we are about to see the floodgates of truly open models blow open and that OpenAI’s “moat” will be proving to be a lot shallower than what they and many others have made us believe over the last six months will come true.
- yieldcrv 4y ago> not to undermine the national security and national unity this is a required statement to conform with China’s constitution, or the superseding authoritative social contract there. think of it like if the Patriot Act was an article of the constitution instead of a random law subservient to the constitution, it would negate other parts of the constitution that we hold near and dear. this is a useful similarity as both constitutions have assurances of free speech just one has a fatal heavily leveraged clause that undermines all other parts of that constitution and dictates all facets of life
- worldsayshi 4y ago> GitHub: GitHub data, filtered by licenses and quality Does anyone know which licenses are filtered into the dataset?
- mananaysiempre 4y agoThe description on the linked HuggingFace page[1] says MIT, BSD and Apache. [1] https://huggingface.co/datasets/togethercomputer/RedPajama-Data-1T#github https://huggingface.co/datasets/togethercomputer/RedPajama-D...
- asddubs 4y agoit's better than laundering gpl code, but it still breaks the licensing terms of those licenses as well, namely attribution
- worldsayshi 4y agoI guess that could potentially be fixed if citation ejection can somehow be implemented into it, which seem to be at least feasible?
- Tepix 4y agoGreat initiative. Next, we need a lot of compute! Perhaps Tenstorrent wants to make a good impression?
- rnosov 4y ago> we are training a full suite of models, with the first becoming available in the coming weeks. Sounds like they already have the compute and began training.
- t00 4y agoObligatory Dall-E version https://labs.openai.com/s/Httd7N2ZF5kynUnzp0vCVOjN https://labs.openai.com/s/Httd7N2ZF5kynUnzp0vCVOjN
- smrtinsert 4y agoSo is the next step is for someone to come in a fine tune on top of it in order to make it a Vicuna? Or can current vicuna deltas be applied?
- rafaelero 4y agoYeah, it's pretty trivial to change the base model from LLaMa to this next one. You just have to finetune it with the same data used previously to train Vicuna.
- wesleychen 4y agoThere's no model yet, only a dataset.
- omneity 4y agoMy understanding is that LLaMa's architecture is open, so the most difficult part is: 1. Getting data of equal or better quality 2. Securing the funding/hardware required for training 3. Learning/figuring out the training challenges needed to tune the process (the PhD part) It seems #1 is the relatively lowest hanging fruit and a prerequisite for the other two, and that's what the project is (rightfully) tackling at this stage. #2 could be solved by many ways, and doesn't require much innovation if the project and the team are solid. Which takes me to #3, which on the other hand seems to be the make or break part of the project. I'm not one to doubt the technical prowesses of the RedPajama's team and their contributors, I rather see it economically. How can an AI open-source project compete with big tech in attracting the brilliant minds of our generation? It's enough to look at levels.xyz to see the battle is not ... level. There's a serious economical challenge in here to have any sort of sustainable open source initiative in AI.
- jamiedg 4y agoHi! I lead Product at Together. We will be releasing a full suite of models trained on this data starting with the first models in the coming weeks. We will release RedPajama base models and RedPajama instruction-tuned models. All of the models will be released under the Apache 2.0 license, allowing commercial use. Therefore, anyone will be able to fine-tune the RedPajama models using Vicuna or other datasets, given they will be fully open-source. The RedPajama instruction-tuned models will be fine-tuned only with instruction labels from human labelers and OpenChatKit feedback (). We feel this will keep these models fully "clean" for use in commercial applications without using the output of other commercial models like were used in Alpaca or Vicuna. However, we'll be excited to see all the great fine-tunes created by the open community and are eager to see how close open-source models can get to the quality of leading commercial models over time!! () OpenChatKit: https://huggingface.co/spaces/togethercomputer/OpenChatKit https://huggingface.co/spaces/togethercomputer/OpenChatKit
- rafaelero 4y agoThat's awesome! Are people thinking about training it for more than just 1 epoch? I believe Gallactica showed that training for even 4 epochs is ok. Also, how amazing would be if the next gen of open-source LLM's increased context window, like adding 8k more tokens? That's probably expensive, but totally doable.
- deleted 4y ago[deleted]
- sp332 4y agoIt's including Common Crawl data 4 or 5 times, does that count?
- avereveard 4y agoThe issue with tokens is that they shoot up inference memory usage
- muttled 4y agoOnce this barrier is broken down we'll see a lot of cool things. 32k on GPT-4 is already pretty cool but once we get into hundreds of thousands/millions of tokens of context we'll be able to easily do things only currently achievable with fine tuning and "memory" tricks. Assistants that remember everything you've ever told them, asking detailed questions about large datasets, even complex systems that are bootstrapped from the context.
- deleted 4y ago[deleted]
- bmc7505 4y ago@dang The title should be changed from MILA to Mila/IQIA.
- MangezBien 4y agoDefinitely thought this was about the kid's book.
- Havoc 4y agoLove this - I'll happily accept a bit of a quality trade-off for a pure open model. Its a bit like I'm willing to accept trade-offs to ensure my IoT gear is local only even if that means loss of cloud convenience
- omneity 4y agoAn actually open source LLM would be a game changer. We might need a new license that englobes model usage and training, something GPL-like whereby distributing a retrained model requires contributing data back or making it public, but not if you use it privately. This will definitely accelerate progress in LLM research, productization and safety. Alpaca, vicuna, gpt4all and others are sporadic repesentations of this that could become a continuous improvement process were the LLM and its license truely open source. An interesting possible side effect of a GPL-like license is that AIs become unlikely to be trained on private data, the usual moat that big tech wouldn't want/just can't make public if it were to use those GPL-like licensed models.
- jupp0r 4y agoAs with original GPL, this would be almost useless in a commercial context.
- e12e 4y agoThere are commercial devices that ship with a Linux kernel?
- ijustlovemath 4y agoI think they mean in terms of enforcement when there's a violation
- Bjartr 4y agoBasically every Android device for starters.
- sp332 4y agoUsing a Linux kernel doesn't mean you have to make your whole project GPL, unless your project is specifically kernel code.
- wongarsu 4y agoNeither would the proposed model license. Just like the kernel's GPL stops at the userspace boundary, the proposed license would only cover the model definition and weights.
- Jayakumark 4y agoThis is huge, was just checking today on what would take someone to get a model similar to Llama, since Meta did not share Training code or Dataset.. Looks like they have figured out how to make the dataset ,Main Problem here is pre-processing them. Second step is to make the code to train model and final one do it cheaply.
- brucethemoose2 4y agoMaybe they should use whatever Cerebras used. The whole point of their own LLM release was as a maximum compute/$ demonstration on their platform. Surely there is a better alternative than a bunch of A100s on AWS...
- inciampati 4y agoYeah. They will use Frontier at Oak Ridge, also known as the most powerful supercomputing system in the world. Maybe it can run some expensive LLM training for once rather than it's typical diet of physics simulations and quadratic gene-gene interaction models :)
- DogTweezers 4y ago[flagged]
- hsuduebc2 4y agoI'm somehow scared and somehow amazed by speed of this progress.
- deleted 4y ago[deleted]
- simonw 4y agoThe training data - all 1.2 trillion tokens - can be downloaded by grabbing each of the 2,084 URLs listed here: https://data.together.xyz/redpajama-data-1T/v1.0.0/urls.txt https://data.together.xyz/redpajama-data-1T/v1.0.0/urls.txt I ran a HEAD request against them all to sum up the total file size, and it's 2.67TB total. Here's a Datasette Lite URL that lets you explore the size metadata about those files: https://lite.datasette.io/?json=https://gist.github.com/simonw/73d15c0dd1025d1196829740bacf4464#/data/raw?_facet=top_folder&_facet=top_folders https://lite.datasette.io/?json=https://gist.github.com/simo... And a SQL query that shows the breakdown across the different sources: https://lite.datasette.io/?json=https://gist.github.com/simonw/73d15c0dd1025d1196829740bacf4464#/data?sql=select+top_folder%2C+sum%28size_gb%29+from+raw+group+by+top_folder+order+by+sum%28size_gb%29+desc https://lite.datasette.io/?json=https://gist.github.com/simo... Sizes here are in GB: common_crawl 1341.6166818914935 c4 806.7667234372348 github 212.1786002581939 wikipedia 111.89125544670969 book 100.43162744678557 arxiv 87.35323827341199 stackexchange 74.54870238155127 Common Crawl is in there a few times - they have the following folders: common_crawl/2020-05 198 files common_crawl/2021-04 176 files common_crawl/2023-06 175 files common_crawl/2022-05 157 files common_crawl/2019-30 153 files And then C4 as well, which is "a colossal, cleaned version of Common Crawl's web crawl corpus. It was based on Common Crawl dataset": https://paperswithcode.com/dataset/c4 https://paperswithcode.com/dataset/c4
- fnands 4y agoNice. Thanks for the summary. So ~4x the size of the Pile, any idea how it stacks up in terms of quality to other big datasets?
- doctoboggan 4y agoI am a little concerned that they have only about 60% of the code tokens (GitHub and stackexchange). Given that so far the only concrete use case I have for LLMs is coding assistance I wouldn't want this open source model to be and less quality in that area. In your opinion do you think this will hamper the model at all? Or is it still more than enough to get good coding assistance?
- 4y ago
- deleted 4y ago[deleted]
- local_crmdgeon 4y agoSo how do I use this? As someone new to the domain.
- skybrian 4y agoYou don't, since they're not done yet. Someone will come up with a way to use it when they're done.
- tinco 4y agoYou download the 2.76TB of data. Then you run it through Llama's training script for a couple months on 40 NVidia A100's, and you should have yourself a pretty fine large language model you could use to host your own ChatGPT service. It'll be significantly worse than ChatGPT for reason's that aren't yet fully clear because OpenAI switched its mission from protecting the earth from nefarious AI developments, to being itself being the origin of possibly nefarious AI developments.
- DigitalDopamine 4y agoRenting 40 nvidia a100s is around 70k dolar per month (on vultr i see). So this would only cost 420k for 6 months. Seems doable. Is 40 a100s enough though? I am interested in what this would cost.
- mlboss 4y agoIt would be great if this can be done on 3090s. Used 3090 usually costs $500-1000 to buy.
- nl 4y agoLLama 65B used 2048 80G A100s for 21 days: > When training a 65B-parameter model, our code processes around 380 tokens/sec/GPU on 2048 A100 GPU with 80GB of RAM.[1] Note that you probably need to budget for double to triple that because things go wrong and it usually takes multiple starts to get a good training run. Smaller models are cheaper though. [1] https://arxiv.org/pdf/2302.13971.pdf https://arxiv.org/pdf/2302.13971.pdf
- dwheeler 4y agoHas anyone investigated to see if OpenCyc can be converted to natural language (presumably English) and then injested into this? Cyc made an attempt years ago to "encode common sense" and a subset called OpenCyc was released. That might be a great way to kickstart information representation of the real world. The latest version of Cyc is proprietary but I think there OpenCyc is an open subset (though I'm having trouble confirming that, so the licensing may not be good). Some links: https://github.com/bovlb/opencyc https://github.com/bovlb/opencyc https://github.com/asanchez75/opencyc https://github.com/asanchez75/opencyc
- sp332 4y agoI've been wondering this for a while now. Cyc has tons of knowledge in a white-box, formal system. If it just had a front-end that could convert from natural language to Cyc knowledge queries and back, we wouldn't have to worry so much about hallucinations, catastrophic forgetting, or trying to fit the entire database in VRAM.
- speed_spread 4y agoMy understanding is that LLM and Cyc are fundamentally different forms of AI. Even if you could turn OpenCyc into text rules, once ingested it would just dissolve into the ocean of training text data and would not significantly gain more apparent "common sense" than it already had. Maybe a more interesting combination could be to have both Cyc and LLM working side by side and comparing notes before agreeing on a result.
- btown 4y agoGPT-4 seems capable of creating CycL output from a text prompt. It might be an interesting guard against hallucination - much like a student being asked to show their work, you're forcing the LLM to go through the steps of framing the problem logically, in a way that's interpretable by the teacher independent from the student's thought processes. That said, it certainly seems like there hasn't been recent work on hosting the OpenCyc knowledge graph in a reasonably modern way, much less the more recent closed-source work by Cycorp (https://cyc.com/ https://cyc.com/). And it's likely GPT-4 doesn't know the full capabilities beyond whatever tutorials were on the web at the time of its training. If I were Cycorp I'd be seriously looking at developing this kind of hybrid model, with an agent model having access to recall their closed-source examples, as a paid cloud offering; there would likely be many who would desire this best-of-both-worlds.
- mgaunard 4y agoPyjama singular actually works, but I'm not sure Pajamas can be singular.
- FranchuFranchu 4y agoI think that at this point, LLM etymology is way more interesting than LLMs themselves.
- martythemaniak 4y agoThis is cool, now we just need to locate 1,000,000 A100-80GB equivalent GPU-hours. If we had a SETI@Home type project setup for this, it would be straightforward - only $50K worth of electricity for the 65B model. Given the immense momentum behind LLaMA, I'm pretty disappointed that Meta won't just open-source it, but I guess reproducing it is better long-term.
- DesiLurker 4y agoI think the time for a folding@home or Berkeley bionic@home style project is now. it will also serve as a backend server farm for all the university based research activities, thereby ensuring the research outcome are not beholden to any one company or benefactor. I remember setting up my PS3 & home desktop for folding project. Its fair game especially if I can use the box to heat the room instead of the furnance.
- sytelus 4y agoGreat to see this but dataset is the trickiest part. There is no way to confirm if this is good dataset unless model is actually trained on it. To reproduce LLaMA, you need $2M of compute.
- Robotbeat 4y agoDo you have a calculation that shows where that $2M number comes from, EXACTLY?
- eiz 4y agohttps://arxiv.org/pdf/2302.13971.pdf https://arxiv.org/pdf/2302.13971.pdf table 15. 1770394 A100-80GB hours to train the entire model suite at the going rate for cloud 8xA100-80GBs (~$12/hr if you could actually get capacity) is ~$2.6M, under extremely optimistic assumptions. YMMV on bulk pricing ;) "the more you buy the more you save"
- Robotbeat 4y agoHmmm… the values in the 7B model seem feasible. An order of magnitude lower GPU hours, plus presumably the lower parameter count means it probably could fit on a 24GB Radeon RX 7900 XTX, which has higher single precision flops than the A100 and costs $1000 instead of $15,000. An order of magnitude lower GPU-hour time, plus if you train it for 210 days instead of 21 days, means you could do a 7B model with 20 consumer GPUs which are $1000 apiece. $20k, not counting mainboard, etc. Really not bad. Might even be doable as a volunteer project.
- nl 4y agoI'm not aware of any efficient transformer training code for AMD cards. Also most training is done using bfloat, not single precision (which is usually only used for accumulators)
- titaniumtown 4y ago
- deleted 4y ago[deleted]
- franzypants 4y agoIt might be a little late, but I hope datasets start incorporating patent texts as well: 1. It's a large corpus of technical knowledge; 2. The language is written by experts in a field and reviewed many times, and 3. They have technical drawings with labels and references in the text The only downside I suppose is that sometimes patents are written with "just enough knowledge" to get it granted but not too much to give away the secret sauce. That's not really that different from many scholarly papers though. To give a size of scale, the granted patent texts of 2020 (without images) is about 160 GB of data, and we have digitized grants going back to at least 1970.
- seunosewa 4y agoYou wouldn't want chatbots to answer you with the kind of language used in patent texts.
- Nitrolo 4y agoI don't know how complete the digitization of old texts is, but if you go to worldwide.espacenet.com, search for "airship" and reverse sort by date you get documents from the 1880s. In fact I'm downloading a whole batch of patent texts right now because I wanted to experiment with semantic search on patent texts. Anyone here have any pointers on what the state of the art method for semantic search through a large corpus would be? I've just started researching and BERT and friends seems like it was popular about 2 years ago but things move so fast I wouldn't know what I should do now. What about a medium sized corpus of text, say 100.000 pages of text?
- piannucci 4y agoIf the name is a reference to Ogden Nash's poem then I am very tickled: https://www.madisonpubliclibrary.org/engagement/poetry/poem-a-day/lama https://www.madisonpubliclibrary.org/engagement/poetry/poem-...
- ricketycricket 4y agoI'd guess it's the book Llama Llama Red Pajama: https://openlibrary.org/books/OL24377652M/Llama_Llama_Red_Pajama https://openlibrary.org/books/OL24377652M/Llama_Llama_Red_Pa...
- nailer 4y agoSomeone on HN made a point that weights can’t even have copyright- they lack two of the requirements for being copyrightable: https://news.ycombinator.com/item?id=35508651 https://news.ycombinator.com/item?id=35508651
- macinjosh 4y agoThis guy has kids, so we all know he.. nevermind. I love the name being a parent myself.
- deleted 4y ago[deleted]
- FloatArtifact 4y agoCode generation, I wonder the difference in output given order of operations with training and fine tuning. What if the model was trained on the documentation and the code base for Python as an example. Then fine tuning came from training on actual python code on GitHub. At the model understands the python documentation and the implementation standard library/interpreter. Then is there a reduction of data needed for code generation therefore reducing the size of the data set used for code generation?
- lhl 4y agoI saw this in my feed recently which was an interesting analysis on how code training was added as a fine tune (Codex) on a foundational model (GPT-3): https://yaofu.notion.site/How-does-GPT-Obtain-its-Ability-Tracing-Emergent-Abilities-of-Language-Models-to-their-Sources-b9a57ac0fcf74f30a1ab9e3e36fa1dc1 https://yaofu.notion.site/How-does-GPT-Obtain-its-Ability-Tr... I do wonder if anyone is considering mixing in larger and larger percentages of The Stack https://huggingface.co/datasets/bigcode/the-stack https://huggingface.co/datasets/bigcode/the-stack with this or the Pile to get more code and see what happens. (Likely beyond mere mortals' budgets though.)
- quickthrower2 4y agoAs I understand it they have the input data, but next up they are creating the model. I could make a joke about drawing an owl ... but that would be a bit mean. I am really glad people are working on this. I wonder... who is paying? Will there be restrictions like ethics clauses and suchlike. Not necessarily a bad thing if they do. Will there be restrictions on commercial use.
- jamiedg 4y agoHi! I lead Product at Together. We will be releasing a full suite of models trained on this data starting with the first models in the coming weeks. We will release base models and instruction-tuned models. All of the models will be released under the Apache 2.0 license, allowing commercial use.
- quickthrower2 4y agoThanks, that is exciting! Given these LLMs seem to cost $millions (or hundreds of thousands) to train, how is this funded. Is it a government / research funded thing, or VC, or philanthropy for example?
- bobwernstein1 4y agowhen will the first code writing specific model arrive?
- t3estabc 4y ago[dead]
- m3kw9 4y agoProb should be compressed as 2k odd separate files and then unzip as needed likely get you to 650gb
- m3kw9 4y agoHow does one just use token and train a model? I thought you need to label each one on what it means?
- abrichr 4y agoYou are thinking of supervised learning, in which a model is trained to generate labels for inputs. In self-supervised learning, the training target is a modified version of the input. Transformers are trained in a self supervised manner. The problem they solve is "given a sequence of N tokens, what is the next most likely token?" No labels required :)
- m3kw9 4y agoWhat about the human reinforcement part of it?
- TOMDM 4y agoThere's no traditional human reinforcement. Models like gpt3 get turned into models like ChatGPT through RLHF (reinforcement learning from human feedback), by fine-tuning the model further on prompts in the style we'd like them to respond in, typically User: question Bot: Response This is done by handcrafting or modifying data from places like stack exchange.
- Zuiii 4y agoI hope they also build a variant dataset that includes high-quality translations and enough material in other languages. Even with the limited non-english content that slipped through LLaMA's trainig dataset, LLaMA still shows strong (albeit useless) evidence that it would have excelled in expressing knowledge coherently in other languages in my testing. Obviously this would increase training costs substantially, so I understand why more languages are not included in the base dataset.
- crakenzak 4y agohmm, could increasing the amount of data in other languages potentially improve performance within the english portion of the model? from it building deeper understandings of concepts that are found within other languages.
- robwwilliams 4y agoLove to see the inclusion of PubMed Central papers, all Pubmed ID abstracts, bioRxiv and medRxiv papers, and NCBI summaries of gene function in the base training set.
- deleted 4y ago[deleted]
- deleted 4y ago[deleted]
- moelf 4y agowhy people don't compress plain text: 2.5G filtered_08cdfa755e6d4d89b673d5bd1acee5f6.sampled.jsonl 834M filtered_08cdfa755e6d4d89b673d5bd1acee5f6.sampled.jsonl.lz4
- vrglvrglvrgl 4y ago[dead]
- AbhiAmbad 4y ago1.8T tokens is very high...
- antman 4y agoIf we were to dream it could also include libgen-text, which I think is as close as it gets to a detailed repository of world knowledge. It's only 1.2TB more. Torrent/Magnet from an older HN post: https://www.offlineos.com/ https://www.offlineos.com/
- betch8975 4y ago[dead]
- velvet6755 4y ago[dead]
- canyon754 4y ago[dead]
- collective765 4y ago[dead]
- MagicMoonlight 4y agoToo many of these models are just using stuff like wikipedia. If you want true language comprehension you need something like reddit which contains actual conversation
- today123 4y agoIs there a train/test/eval split available somewhere on this dataset too?
- today123 4y agoIs there a train/test/eval split available for this dataset somewhere?