10 ms·

GPT-J-6B – A 6 billion parameter, autoregressive text generation model

636 points by canada_dry 5y ago

gkoberger 5y ago
I really appreciate putting the effort in on any open source project, however GPT-J and GPT-3 are not even close. I couldn't figure out any prompt that resulted in GPT-J returning anything that wasn't gibberish, whereas GPT-3 is almost always within the ballpark.
- dman 5y ago
  On the other hand its infinitely more Open than anything from OpenAI :)
- sillysaurusx 5y ago
  You have to set top_p to 1 and temperature to 0.8. This was hard-won experience, and no one seems to believe me when I say it. :) Try it again; I’ll bet you a dollar you’ll be more impressed. Defaults really matter. temperature 1.0 is the worst possible default. It’s no surprise it returns gibberish; the sweet spot is between 0.7 and 0.8. When I first got into ML, this was a big surprise. But it’s the difference between flying a plane with flaps all the way down (a crazy default) and normal settings (0.8 or so).
  - jjcm 5y ago
    Samples with the settings metioned above (tl;dr - results are significantly better with sillysaurusx's suggestions): Using the default promp: -----RESULT (DEFAULT SETTINGS)----- In a shocking finding, scientist discovered a herd of unicorns living in a remote, previously unexplored valley, in the Andes Mountains. Even more surprising to the researchers was the fact that the unicorns spoke perfect English. The scientist confirmed his findings by being at the farm. Wild dingo The wild dog, also known as a canid is the feral domestic dog subspecies with the wild ancestor of the domestic dog. The Australian Dingo is native to Australia, and... // it continues to list animals and small facts about them -----RESULT (SILLYSAURUSX'S SETTINGS)----- In a shocking finding, scientist discovered a herd of unicorns living in a remote, previously unexplored valley, in the Andes Mountains. Even more surprising to the researchers was the fact that the unicorns spoke perfect English. The surprising discovery of English-speaking unicorns was made by an international team of scientists led by Dr. Steven Wise, from the University of California, Irvine (UCI). According to the article, they stumbled upon the unexpected herd in the remote Aucayacu Valley, in Peru. During their field expedition of the location, the team discovered a herd of five unicorns. Wise told Scientific American that he was fascinated by the discovery and the questions it posed. He said that the unicorns had no problem understanding basic questions, such as which way to go to a certain location, or what time it was. They were also able to understand and respond to basic commands such as “hello,” “good-bye” and “trot.” The scientists explained that unicorns are unicorns because they possess a pair of horns. The horns at the front of their heads are not covered in hair, but rather are made of a material that is hard and smooth. Dr. Wise said that unicorns have a different kind of horn than any other living mammal. He added that although people often refer to the pair of horns as “horns”, they are actually not.
- Rebelgecko 5y ago
  Anecdotally from playing around with both, GPT-J seems to struggle compared to GPT-3 when it comes to regurgitating remembered details from the prompt. For example when trying to generate short stories I've found that GPT-J will randomly genderswap characters from sentence to sentence, which I never had happen with GPT-3.
perspicace 5y ago
Sample / demo?
- gkoberger 5y ago
  From the README: https://6b.eleuther.ai/ https://6b.eleuther.ai/
  - nelsonfavedra 5y ago
    Prompt: Jason made a pancake. Result: > Jason stared at his image in the griddle's stainless steel surface. The image of a large, freckled man wearing a black t-shirt and green plaid boxers. It was like he'd stumbled upon an abandoned photo shoot, or a bizarre prop out of an old TV series. A figment of his imagination from a nightmare he'd had once where he was alone in a house, free to do anything. Aah. That was it. He'd been sitting in the dark eating his microwaved pizza, watching some sort of porno, and thinking about strange little things he'd seen when he was growing up. When he was younger, things were simply that much stranger. Everything he'd seen, for that matter. It was weird how that worked. He'd stare into space, the fog of drugs would clear, and some unseen object would immediately become a mirage. The pattern was always the same, though. He would fixate on some tiny detail, and a few seconds later that detail would become a goddamn spectacle. Even the stranger things in his life took on that shape. The porno had images in it that meant nothing, in their own way. And he was always left to ponder what sort of symbolism might be hidden beneath Such an emo AI.
  - TheAceOfHearts 5y ago
    Is it still working for anyone? I can't get it to work, it always shows me an error.
- canada_dry 5y ago
  https://colab.research.google.com/github/kingoflolz/mesh-transformer-jax/blob/master/colab_demo.ipynb https://colab.research.google.com/github/kingoflolz/mesh-tra...
marcodiego 5y ago
Seems good, starting with "Today I wake up and" it gave the following: Today I wake up and eat my breakfast in silence. Alone. I am actually kinda happy with that. I listen to some podcasts and read the news before leaving to get a couple of documents together. I’m going to my mom’s today to have a meeting regarding my eviction notice. My mom still lives with my dad. I can’t ask him to do anything, though, I guess. I was always so used to having to ask for stuff, that I completely forgot I could get people to do what I want for me. My mom, for instance, is not as bad as my dad, and I actually don’t mind asking her to help me with everything. It makes me feel like she loves me. I know that this is a small thing, but it’s something that makes me happy. I get to my mom’s apartment at 5:30. The place is very nice. There is a roof terrace, an elevator that goes up and down (where can you find something like that in an apartment building?!), and all rooms have their own AC units. All of the furniture is in good shape. The appliances were also all upgraded recently, so there is a full washer/dryer, a coff
- mdaniel 5y ago
  Given: > I wanted to write a language model, but all I got was Got (including the `[...]` in the middle; that is, everything beyond this trailing colon is its result): I wanted to write a language model, but all I got was a rabbit." --Richard Feynman Though I might not have gotten the overall award for RPL, I certainly came in close for the more specific one of having the highest score in a core language spec proposal. And yes, there were more than two people (including me) that submitted those, but somehow a few of my friends and I pushed it over the top. So I wanted to give credit where credit is due, so I present this write up of my proposal and the (not so minor) additions that were needed to get it through (I may have exaggerated at times but it really isn't much). [...] Thanks everyone for voting, I'd really like to go to the hackercon in august so I can build a portable terminal that I can give to everyone in the congress... as a shameless self-promotion, I'll offer all attendees a guaranteed savings on the discount package Yes, you do lose some points for a paper that you haven't quite finished, but that's what the RPL is all about - the unfinished. Since I want to go to RPL I would like to know if someone can explain to me why I have lost any points. I have deleted my paper from the site
- gentleman11 5y ago
mdaniel 5y ago
I hadn't noticed this format before, but wouldn't one want to include a sha or git tag in such a citation? howpublished = {\url{https://github.com/kingoflolz/mesh-transformer-jax}}, I did see the month and year keywords, which I guess would narrow down the number of commits it could have possibly been, but nowhere near the reproducibility of citing the software used
- bArray 5y ago
  Generally people should add git hashes, but I've not seen it done. More than half of papers don't even have the source code available - so pointing to any source is already great. Also it's not a particularly well formatted way of citing a URL anyway. I believe it is more correct to use `url = "<URL>"` in a `@misc` [1]. But it's also unpublished (non peer-reviewed). so doesn't matter so much. [1] https://en.m.wikibooks.org/wiki/LaTeX/Bibliography_Management https://en.m.wikibooks.org/wiki/LaTeX/Bibliography_Managemen...
wyldfire 5y ago
Are models like GPT-3 ever used as a basis for transfer learning like the BERT/etc ones?
- minimaxir 5y ago
  OpenAI has allowed certain customers to finetune GPT-3 for specific datasets.
minimaxir 5y ago
See also test cases of using it with code: https://news.ycombinator.com/item?id=27621114 https://news.ycombinator.com/item?id=27621114 Incidentally from my limited testing with GitHub Copilot, the results are surprisingly close between Copilot and GPT-J.
akomtu 5y ago
Try some political prompts to see which side this model favors. With this thing you could run a one person news agency with an army of followers. The thing is that most news reporting isn't about facts: it's about feeding the readers content they want to hear and GPT is outrageously good at it. Hmm.. maybe that's a legit business idea.
sillysaurusx 5y ago
Hey everyone, I just wanted to chime in and say that GPT-J is incredibly legit. Every aspect of GPT-J is production grade — no difference in process, quality, or results, compared to any other big name research lab. I also want to apologize to Eleuther for giving them a hard time in the past. My earlier concerns were unjustified. To be completely honest, I was jealous they achieved everything I tried to achieve with my own open source research lab attempt. It took a long time to even recognize that jealousy in myself, let alone set it aside. Sorry. The credit for this work goes almost entirely to kindiana, aka Ben Wang. Remember that name; you’ll be seeing a lot of it in the coming decade. It’s clear to me that whichever lab he ends up at (he’s an undergrad! Google let him slip away because he didn’t have a degree!), he’s gonna be changing the world. Don’t know what, don’t know how, know he will. Every aspect of that codebase is immaculate. Most research code is not pretty; this looks carved out of marble and placed in a museum. Without Eleuther’s TPU resources, this work wouldn’t have happened. Tensorfork (my lab) didn’t get access to the TPU VM alpha. And TPU VMs were an absolute necessity here. (TPU VMs are a new thing; they’ve been in alpha since December, but only recently launched. If curious see https://github.com/shawwn/website/blob/master/jaxtpu.md https://github.com/shawwn/website/blob/master/jaxtpu.md and https://github.com/shawwn/website/blob/master/mlmind.md https://github.com/shawwn/website/blob/master/mlmind.md for why it’s the future of ML.) Eleuther also helped test the model thoroughly. Leo Gao (go follow him: https://twitter.com/nabla_theta?s=21 https://twitter.com/nabla_theta?s=21) ran GPT-J through the gauntlet. He was the primary person behind The Pile, the training data that makes any of this possible. I can say with absolute certainty and no hesitation that there are no “gotchas” here. Eleuther’s https://6b.eleuther.ai https://6b.eleuther.ai page looks wonderful too. It’s like a free OpenAI API playground that everyone can try. Keeping it running for months is no small achievement. (Set top_p to 1.0 and temp to 0.8; the defaults are pretty bad.) Congratulations, and thank you everyone for all your hard work. The world is so much better for having access to this level of quality.
- GraffitiTim 5y ago
  This was a very nice thing to say.
  - sillysaurusx 5y ago
xvector 5y ago
AI generated text these days feels so natural that when you return to human-generated text you can't really tell. Incredible.
tkgally 5y ago
I tried the web demo at https://6b.eleuther.ai https://6b.eleuther.ai The prompt I gave it was “This morning when I woke up and got out of bed, I never expected what would happen later that day.” Here is the result. Warning: NSFW. “This morning when I woke up and got out of bed, I never expected what would happen later that day. After a fantastic night of sex with my very sexy girlfriend, I decided to go over to my gym to workout and shower. It has been a few months since I have gone to the gym and being in the shower reminded me of how much fun it was when I was single and how much I missed the thrill of working out. After I got out of the shower, I went to grab a quick towel to dry off and after drying my body, I put on my towel and started to walk out of the bathroom. The towel slipped and fell off of my body, leaving me completely naked in my home gym and I was caught off guard. I sat there in the middle of the shower, horny as fuck. While I had a good look at my now-exposed body, my cock was already half-hard and I wasn’t about to leave like this so I began to stroke it a little. I thought to myself, “What am I going to do?” I couldn’t get out of the shower so there was no way to get out of this predicament. As the light streamed through the bathroom, I could tell by the position of the sun that it was going to be a warm day. I thought about running out into the” I then tried the prompt “I’ve decided to become a better contributor to Hacker News.” After a half-dozen responses of “Unable to connect to the model. Please try again,” I got this: “I’ve decided to become a better contributor to Hacker News. I have given up on the idea of never leaving and rarely reading, I do hope to engage. Here’s a little secret about me, I have ADHD. I also have autism. So, I’m also here to tell you about some of the many, many struggles I’ve faced as a member of both groups. I don’t want to label myself, I don’t know exactly what kind of ADHD or autism I have. I also don’t think it is relevant to me. In fact, I think it is completely irrelevant to my experiences. But, hopefully by writing this, you may learn a little about yourself and the people you have around you. This is going to be a very long post, because I have a lot to say. I want to start off by telling you something I don’t talk about enough. I am open about being open about ADHD. I am also open about being open about my autism. My ADHD makes my experience on the autism spectrum look different than the experiences of others. I have ADD (attention deficit disorder) — I’m good at focusing on a single task, but when my attention starts to drift, I have a”
- tkgally 5y ago
  Three more. The prompts are in italics. They read like dreams. “On my first day in London, I went for a walk with my friend Katherine. We began at Trafalgar Square, where there is a big bust of Winston Churchill and a smaller bust of King George III. We then crossed the park, along a path which leads toward a tunnel that used to take me under Waterloo Bridge and the Thames. In the tunnel, we caught up with three other friends, one of whom was carrying her violin. We walked across the Thames to Westminster Bridge, and from there into the city. I had been in London for several days, and the first time I saw it, I thought it was my former home. The people I knew from Washington University were still there, and several of them showed me around the city. One of my friends from that period, Helen, took me to meet her husband in a part of London called Notting Hill, where we wandered through lovely little houses built on narrow, leafy streets. I did some sightseeing, and was at the Tate Modern in an exhibition called “Images in Time.” I remember the huge portraits of two African-American women and a couple of sculptures by Charles Ray, which I liked very much. But my favorite work in the show was Andy Warhol’s tribute to the singer Madonna. It was” “On my first day in San Francisco, I went on a hike on the Golden Gate trail. When I got to the first part of the trail, I had to cross the railroad tracks. At one point, it was a busy and bustling town, but now it’s not even alive with traffic at all. The area was a dump; it was no longer a dumpy dump. It was an amazing transformation. I had never seen such a thing in my life. As I hiked further into the Golden Gate National Park, I stopped to get a bite to eat and get a bottle of water. I came across a beautiful, serene old-looking gentleman standing in front of the store. He was wearing a long-sleeved shirt, with sandals, and what looked like a loose, fitted, khaki pants. His head was shaved bald, except for a sideburn and a small patch of hair right above his top ear. He had bright, orange-tinted, 70’s-style glasses on, and a huge smile on his face. He said, “Are you visiting? Are you from San Francisco? You’re about to get to the Golden Gate National Park? This is a wonderful place. You should see the ocean. There is nothing” “On my first day in Tokyo, I go to see a little newspaper office. There’s a wall full of sheets of newsprint and the woman manning the desk, in her late twenties or early thirties, with a Hello Kitty pen set in her desk drawer and her hair in neat ponytail, is diligently putting together a comic strip called I Love You By Hokusai, which is a pastiche of the famous Japanese woodblock prints. The strip features a girl, Yuki, who loves Hello Kitty, Natsuki, a girl who’s into anime and Japanese hip-hop, Minnie Mouse, and Hot Chocolate. The comics aren’t going anywhere—they are, in fact, part of the paper’s entire newspaper business. They’re just going to be running more of them now. My day, though, is spent going to cafes with Haruki Murakami and other writers. In the days after we’d met, I saw a couple of articles about Murakami. One described him as not just a famous author, but a guru of the literati: how he was well-dressed, didn’t show off his money and just preferred to spend it on books rather than on his five luxurious apartments”
istillwritecode 5y ago
The solution to the housing affordability problem is relatively simple. All we have to do is ... stop building houses, and start building hospitals instead. I’m certain that all sides of the political spectrum have had a passing thought about this. It is actually just a simple supply and demand problem. As it is now, the ratio of housing supply to demand is quite heavily skewed in favour of the housing demand side. So what’s to be done about it? There is one way in which we could achieve this. Build many more hospitals and simply not need them. It would be amazing if this can actually work. A few things to consider: Almost all of our current hospital spending goes towards tertiary care. Most primary care (general practice) and secondary care (ambulance, emergency etc) spending is on hospitals. We already pay about $5 in Medicare, so $5 of your hospital spend is already locked in. We already spend about $9 billion on private hospitals for elective surgery, and around $2 billion on private hospital for elective imaging. There is virtually no need for these in terms of frequency or outcomes.
- bckr 5y ago
  Wrong thread? Wrong thread.
  - lanternfish 5y ago
    Example completion. Looks like it fooled you.
  - omnicognate 5y ago
    Presumably everything after the ellipsis is GTP-J. Solving the housing crisis by building more hospitals and "not needing them" doesn't sound like the product of a human mind, jokes about how crazy many people's political views are aside.
    - admissionsguy 5y ago
      It fooled me, too. The line of thought is not that unusual for this site. It fits all the boxes: argues to give the government more money, is against private healthcare, attempts to show the author's unconventional brilliance but makes no practical sense. I initially assumed they are expecting settlements to spring up around the hospitals they build.
parsecs 5y ago
I tried this with a more technical prompt... and what can I say? I'm stunned at the incredible command of jargon here. It's even mostly correct with the actual technical details at a glance, especially that description of how radar works! I can't wait for those research paper generators to use stuff like this to make high caliber fakes. Prompt: > Synthetic aperture radar imaging is an effective method for Results: > Synthetic aperture radar imaging is an effective method for detecting the presence and/or quantification of target objects, including: detection and tracking of moving targets, which are known to be difficult with visible imaging and/or laser radar; monitoring the occupancy of an enclosed space, such as a room, by the presence of occupants therein; and detection of moving objects, including animals and pedestrians, as well as people. A radar device is composed of an antenna, a receiver, and a transmitter. The antenna radiates a beam of energy (e.g., electromagnetic radiation in the radiofrequency range) that is received by the receiver. The transmitter then transmits an interrogating signal to the target(s). The time delay between transmission and reception at the target(s) is used to measure the range of the target(s) from the radar device. All targets in the field of view of the radar device contribute to the signal received by the receiver. The transmitter amplitude and/or phase modulates the signal transmitted to the target(s) with the information to be encoded, typically at a much higher modulation rate than that used Another prompt: > Fringe fields in microstrip patch antennas Results (shortened by me for brevity): > Fringe fields in microstrip patch antennas and their RF performance > 1. The microstrip patch antenna is a conventional patch antenna that is created by a thin layer of insulating material placed on top of a conductive substrate. The antenna is excited by a current in the conductive layer in the vicinity of the antenna. Now I have a question. How does this GPT-J know what a patch antenna is? It's able to give an almost correct description of this rather technical device. It made the mistake of flipping conductive material and insulating substrate, but it's very close. Does this neural network have the "knowledge" of a patch antenna in the weights? How does that even work? It seems kind of incredible.
- onion2k 5y ago
  The 800GB "The Pile" dataset includes the entire corpus of the US Patents and Trademarks office, so there are lots of documents about antennas.
- 5y ago
jsrcout 5y ago
I asked it nonsensical questions about historical figures and got output that was nearly sane sounding, fairly coherent, and sometimes close to brilliant. I can't speak to the technical aspects. Although it didn't seem to "get" that people that die can't do things after they die. For sheer entertainment though, it's fantastic.
- morpheos137 5y ago
  it doesn't "get" a lot about the real world.
muzani 5y ago
I like that OpenAI forces people to use theirs ethically. There's obvious benefits to a competitor, but I'm wary it might be a Pandora's Box.
- robbedpeter 5y ago
  For an absurd definition of the word ethics, maybe. It's thinly veiled corporate ass covering pretending at ethics, at best. Anyway, It's not quite good enough at coding to prompt an intelligence explosion, and the resources to run it are rarified enough to prevent most worst scenario abuses. However, if it becomes more efficient, accessible, and capable, all those weird AI threats are likely to be a lot more relevant. Moores law alone means the next couple decades will be exciting.
  - muzani 5y ago
    It's not about killer robots. It's about fake reviews, fake tweets in support of some political opinion, more convincing viagra spambots. Also these bots are as racist, sexist and bigoted as the data they're trained on. It's not intelligent, but it's dangerous in the hands of humans. It's like raw fire without any safety measures. At the very least, I hope that if they're going to make it open, it comes with similar filters built into GPT-3.
    - robbedpeter 5y ago
      There's no unified underlying intelligence in the model that could be called racist, sexist, or bigoted. These "bots" are not, because they cannot, be such. The output is dependent on the patterns, context, and content of 800 gb of human generated text. Any and all of which can be seen as morally insufficient depending on your taste. Clever and iterative use of prompts could identify, filter, or modify potentially offensive text for whatever level of pearl clutching floats your boat, but transformers are algorithms approximating parts of human cognition. The algorithm doesn't have an ideology, morality, ethics, dogma, or any of a myriad of features you can project onto it. It's a tool, which can be used well or badly, and part of using it well will involve not attributing to the tool anthropomorphic features it does not possess.
quicklime 5y ago
I tried giving it one of the prompts from copilot.github.com: def strip_suffix(filename): """ Removes the suffix from a filename """ return filename[:filename.rfind('.')] def test_strip_suffix(): It completed the test function like this: # test stripping of a suffix a = strip_suffix("I_am_file.txt") assert a == "I_am_file" Compare this with GitHub Copilot, which gives: """ Tests for the strip_suffix function """ assert strip_suffix('notes.txt') == 'notes' assert strip_suffix('notes.txt.gz') == 'notes.txt' GPT-J also gave a bunch of explanation on how to use the generated code: So this goes in a file called my_file.py and when you run python3.4 -m my_file.strip_suffix you should get the following output C:\Users\user\Desktop>python3.4 -m my_file.strip_suffix 'strip_suffix' C:\Users\user\Desktop> So the strip_suffix function which strips the suffix from a filename is called as part of a larger code that uses some functions from the file. To tell Python what functions are available in a file you just need to put #-comment characters on the first line and then the functions that you want to use, and the first line will be taken as the function documentation: #-*- coding: utf-8 -*- #comment def func_one(): ''' Documentation for function one. ''' print("This is the doc for func_one
- lunixbochs 5y ago
  Copilot isn't a good example because it was trained differently. A better comparison would be GPT3 vs GPT-J. Note that there's still a pretty big difference in parameter size.
  - MaxikCZ 5y ago
    I feel like this is still very useful comparison precisely for the reason that those two models were trained differently.
gpt_toss 5y ago
Any ideas how to make the generated text end on punctuation? All appear to cut off randomly during the last sentence rather than more naturally at punctuation like a period.
jcims 5y ago
Having an AI convergence day, so I married my tinkering with VQGAN+CLIP with GPT-J with the following experiment. Prompt is in standard font, response is in italics. ---- I started messing around with image generation with VQGAN and CLIP. It is really cool, but I ran out of things to prompt it with, you know just short phrases that describe a scene that could be rendered. I shortly thereafter saw an article about GPT-J and thought it would be cool to use AI to generate the prompts. So I go the demo up and running and asked it to generate a prompt by typing this into the notebook. The computer responded with the following: Three cats chasing a balloon with laser beams shot at them from the ground. I did not type in "three", so it really was just a blue-print of the problem. But this got me thinking. I had seen in different contexts that we humans will generally enter some rules when generating a creative response and act as that request for the request. For example, when I am writing a resume I am usually very mindful that I do not include any errors in the use of personal pronouns. I know that when I write "I will go to the fair", I know that I could come up with the following but not the former. I will go to the fair I then noticed that the prompt actually contains two main requests. Three cats chasing a balloon And this is where things got really interesting. I have noticed that sometimes I will be able to play the two requests against each other. The sun shines on three cats chasing a balloon In this example I was able to generate a response that contained two unrelated requests that were both fulfilled from the prompt. This isn't a requirement, if the AI or its creators want to keep a loop from being possible, the loop will likely be re-rolled if it is present. ---- I then took the phrase 'Three cats chasing a balloon with laser beams shot at them from the ground.' and generated an image with VQGAN+CLIP and it came up with the following: https://imgur.com/KpAdZ3W https://imgur.com/KpAdZ3W
- ionwake 5y ago
  Ive built some code that renders speech through a 3d avatar for youtube, maybe you would like to collaborate, it should be quite easy to plug everything you just described together. If so let me know and Ill paste my email thanks
  - jcims 5y ago
    Hey thanks for touching base. This is just me tinkering at the moment but I'll ping you in the future if it sticks.
sicromoft 5y ago
Direct link to the demo: https://6b.eleuther.ai https://6b.eleuther.ai
- bruce343434 5y ago
  "Unable to connect to the model"
  - stanislavb 5y ago
    I could guess, it's been overloaded by traffic from here :)
6gvONxR4sf7o 5y ago
I’m simultaneously surprised and unsurprised that announcements about Copilot get so much copyright discussion, while the GPT-like models don’t get nearly the same. Meanwhile, GPT-J is literally trained on pirated books (the books3 corpus is part of the Pile, which is the corpus this was trained on). Charitably, it’s because licenses are already such a core discussion when github comes up. Uncharitably, it’s because Copilot uses “our” community’s labor, while the GPTs use others’.
- neolog 5y ago
  It's because Copilot isn't open source.
  - wccrawford 5y ago
    Not just "not open source", but Github specifically said they intend to monetize it.
  - Kinrany 5y ago
    Neither is GPT-3?
  - 6gvONxR4sf7o 5y ago
    How’s that make a difference? Being open source doesn’t make IP issues disappear in any way I can see.
    - neolog 5y ago
      Fair point, OP is APL2, which isn't quite good enough since it's probably using GPLed stuff, but it's better than being closed source.
      - 6gvONxR4sf7o 5y ago
        It’s not just using GPL stuff, it’s also using straight up pirated books, for which APL2 is very definitely insufficient.
- 5y ago
deleted 5y ago
[deleted]
quickthrower2 5y ago
Let the spams begin
bodo11 5y ago
This is really impressive! After a few legit responses, I got this: with the input The sun is shining today so I am thinking about going down to the lake and sail my boat. I shouldn't go by bike because It generated: it is too hot. I should go by car but I don't have a car. I should go by bus but I don't have a bus pass. I should go by foot but I don't have a pair of shoes. I should go by train but I don't have a ticket. I should go by plane but I don't have a passport. I should go by boat but I don't have a boat. I should go by boat but I don't have a boat. I should go by boat but I don't have a boat. I should go by boat but I don't have a boat. I should go by boat but I don't have a boat. I should go by boat but I don't have a boat. I should go by boat but I don't have a boat. I should go by boat but I don't have a boat. I should go by boat but I don't have a boat. I should go by boat but I don't have a boat. I should go by boat but I don't have a boat. I should go by boat but I don't have a boat. I should go by boat but I don't have a boat. I should go by boat but I don't have a boat. I
- morpheos137 5y ago
  Sounds like the AI doesn't have a boat...
- notduncansmith 5y ago
  A bit avant-garde but it works
- wellthisisgreat 5y ago
  I have no mouth and I must scream
- notahacker 5y ago
  I have a boat and am willing to offer GPT-J a ride...
- taberiand 5y ago
  A panic attack after realised it was lost in a dream, given that the original statement said it had a boat and then suddenly it did not.
msmagh 5y ago
Web Demo doesn't work for me on mobile, works fine on pc.
reed1234 5y ago
I wonder if GPT-3 will get some pressure from this to become open source or at least lower their prices if they want to remain competitive.
- 9dev 5y ago
  I sure hope so. Got beta access early on, with a pretty strong use case, namely generating human-readable summaries of records in a large database from meta data. Once I saw the pricing I thought that must be a joke. If you’re just fooling around and create a handful of prompts, okay, but it’s prohibitively expensive for anything starting from a few thousand requests, not even to mention a 100+ million :) I think we might actually just try GPT-J…
- Tenoke 5y ago
  I doubt it. While a bit cheaper this isn't that much cheaper to run yourself given that it has much worse results than GPT-3 and would require way more maintenance to keep running.
jose-cl 5y ago
in my case, for "When the lockdown is over, I will" Result When the lockdown is over, I will drive into the suburbs. I have not seen my family for over two months. It is the first time we have been together for more than a month. The opportunity is more than a welcome change from the isolation of quarantine. In the first days of lockdown, many of us were taken aback when we realised that the familiar streets we walk every day and that we have seen from a distance for the last two months, now cannot be reached. It is odd, looking out of the windows of my apartment that used to be the starting point of my morning walks, to see the back of a view I take on a daily basis. I am no longer the first in line as I used to be at the market, waiting for my turn to pay for what I want and exchange pleasantries with my colleagues. Now, I am the last in line, standing in an almost deserted supermarket for a few seconds before I realise I am no longer needed. At some point, we will be able to open our doors again. But for the time being, the call to social distancing is still the most effective way to curb the spread of coronavirus. Now, the calls for people to be more open are more of a necessity than a choice.
morpheos137 5y ago
Prompt italicised... She stroked Bill's Microsoft floppy disk until she found his hard drive. She erased the office program and reset it. While she was there, she removed the White Dragon Armada artwork, cleaned off the end panel where the logo had been, and folded up the blank back cover. # CHAPTER 17 # Binky Ralph and Tiffany moved into a huge and very classy apartment in the First National Building. They loved having their own place. But they were stuck with one more mouth to feed. They also felt the need to set up a security system. Binky hadn't realized they were going to have to get a nanny and a security guard. Binky was too busy coddling their dog to think of setting up the security system. Binky's fondness for naps got to be more important than the realization that there had to be an alarm to protect their stuff. Ralph and Tiffany left Binky alone with the dog. If the dog barked, the apartment was a crime scene. "What's the problem?" Binky said to the dog. "Don't you want me to protect you?" "Naw," the dog replied. "I'm fine." "Fine!" Binky agreed. "I can protect you." He began to
- newswasboring 5y ago
  This sounds like something Neil Gaiman might write. Especially the last exchange.
  - morpheos137 5y ago
    Here is a second reply to the same prompt: She stroked Bill's Microsoft floppy disk until she found his hard drive. She moved to its top and looked at its contents. Sure enough, she found Bill's e-mail messages. There were the familiar missives from friends, family, and her book. She scrolled through them. There were messages from brokers, clients, and partners that she'd done business with. The ones she did not recognize. Then she came to one that caught her eye. The subject line read "Note from my lawyer." It was obviously Bill's lawyer. And she knew why the lawyer was e-mailing. She opened the message. It was two short paragraphs of nonsense. She saw that the sender's e-mail address was hidden behind a protective program. She pulled her phone off the edge of the bed and held it over the computer screen. When the screen lit up, she saw that the sender's e-mail address was a fake address, not the real one she'd been using to contact Bill. Sarah couldn't be sure of anything anymore. All her suspicions were now confirmed. This was the real purpose of the meeting with Bill. He wasn't interested in business or investments. He was interested in money. She'd known from the start that his monetary concerns were much bigger than her.
throwamon 5y ago
Prompt: "no" Result: no}\ \ \text{otherwise}\\ \end{array} \right.$$ Since $\max_{i \in \left[m\right]} \left\| {\mathbf{x}}_i - {\mathbf{x}}_0 \right\|_2 = \epsilon$, we need to upper bound the probabilities $\mathbb{P} \left( \min_{i \in \left[m\right]} \left\| {\mathbf{x}}_i - {\mathbf{x}}_0 \right\|_2 < \epsilon \right)$. We denote this probability by $P_1(\epsilon)$, since it is the first term of $\mathbb{P} \left( \max_{i \in \left[m\right]} \left\| {\mathbf{x}}_i - {\mathbf{x}}_0 \right\|_2 < \epsilon \right)$. Given a covering of size $B$, we need to bound the probability $\mathbb{P} \left( \min_{i \in \left[m\right]}
ionwake 5y ago
Do they have an API I can connect to? Or is there a 3rd party that could be one? I would really like to connect to this for my pet project and am happy to pay for access, if anyone has any ideas let me know thank you.
d13 5y ago
As a beginner, could anyone point me to resources on how to use this to create a simple web app?
- bckr 5y ago
  Doesn't look like it's ready to use for a web app, just because the response times are so high. It could also be particularly high today due to the "HN hug of death", but I expect usage will remain high. That's assuming you want to use the API, which, while they don't provide any information about it, you could figure out what's going on by using the demo page and then inspecting the network requests tab in your browser to see what requests are being sent and what responses are coming back. Another option is to run the model yourself by downloading the weights and figuring out how to use them. This will be a bit challenging especially for a beginner. Assuming you can do one of these two things, setting up the web app is the easy part. You could use a framework like Flask and just expose an IP address on a virtual machine living in Google Cloud Platform or AWS. Hope that helps.
  - d13 5y ago
    Thanks so much, that explains it! I was looking for an API document but, of course, couldn’t find one because it doesn’t exist. Inspecting the network requests is a great idea! I’d like to try running the model myself. What would the a basic would the basics steps be for doing this?
haolez 5y ago
Is "The Pile" dataset English only?
EMM_386 5y ago
> Unable to connect to the model. Please try again. HN "hug of death"?
- BoxOfRain 5y ago
  I suspect so, still getting this error as of 21.39.00 British summer time.
thelivenewz 5y ago
http://thelivenewz.com/public/articles/PUBG-Mobile-Battlegrounds-India-is-now-launched-Indian-pubg-mobile-download http://thelivenewz.com/public/articles/PUBG-Mobile-Battlegro...
MrsPeaches 5y ago
Is anyone else finding this thread really disorienting? There are some comments where I assume it’s written by a human but it feels no different to the responses to prompts.
- kaetemi 5y ago
  I felt that experience after playing with GPT-3 for a while too. The constant flawlessly average grammar makes everything else sound like it. It lacks identity beyond being perfectly average.
- fouronnes3 5y ago
  Interesting to use a question for a prompt. Very nice.
lerela 5y ago
In the middle of a generated text about low-carbon concrete, I found some Wikipedia markup: Use Low-carbon concrete is used to build low-carbon structures and other buildings. Examples of structures that are made of low-carbon concrete include buildings in greenhouses and offices. References Category:Concrete Category
julius 5y ago
Has anyone tried to run the pretrained models on non Google Cloud hardware? Are normal PCs powerful enough to run it locally?
- Tenoke 5y ago
  As it says in the repo it's been designed for running on TPUs so it would not run out of the box on your PC. Also most normal PCs wouldn't have the GPU ram (at least for training) to run it.
  - mdaniel 5y ago
    Depending on one's definition of "your PC", they do sell "local" TPUs: https://coral.ai/products/accelerator https://coral.ai/products/accelerator
irthomasthomas 5y ago
I made a none-gpt variant of this idea for fun after watching some breathless microsoft video about AI in VS code, which amounted to autocomplete sorted by popularity. It is just a wrapper to howdoi with a sprinkle of extra logic to make it a bit more useful. https://github.com/irthomasthomas/helpmecode https://github.com/irthomasthomas/helpmecode Rather than creating fake code that looks real, this gives you the S.O. code answers verbatim. So you know it is probably out of date or wrong. At least you cant delude yourself (if you don't know how to write a piece of code then how can you judge the code written by GPT?). My tool is great for getting a quick reminder of syntax without needing to switch to a browser. I could improve it by adding metadata like the date of the answer and comments maybe.
truth_ 5y ago
What would be the suggested steps to train it on a new language from scratch?
schmorptron 5y ago
That's dope! How realistic is it to train this with an own (way smaller) dataset on a normal desktop?
nerderloo 5y ago
Eh, has there been any update since the last posting? https://news.ycombinator.com/item?id=27443528 https://news.ycombinator.com/item?id=27443528
- mdaniel 5y ago
  I'm guessing like a lot of reposts, it was deemed more relevant now, especially the GitHub Copilot-esque "from bs4 import BeautifulSoup as BS" prompt that the commenter alleged to have emitted plausible python: https://news.ycombinator.com/item?id=27447557 https://news.ycombinator.com/item?id=27447557 I will admit, I'm especially impressed that it caught the "as BS" part in its code generation -- then again, it's entirely possible that a lot of code examples do that, so maybe a more stunning result would be "as BSOUP" and see if it still generates plausible code
  - smegcicle 5y ago
    > especially impressed that it caught the "as BS" what i understand from a completely armchair perspective, is that this kind of context-holding is THE feature of this wave of ai models, which is what similarly allows it to generate paragraphs of prose seemingly out of whole cloth, tracking what it's talking about as it rambles along. the output seems (with occasional glaring issues) remarkably cohesive and realistic, given that it has nothing original to say. I picture it as taking sentence structure it's seen before, and narrative structure it's seen before, and topics it's seen before, mixing them all up, and putting them back together like multidimensional legos- from this perspective, processing the 'import as' lego seems to fit right in. The whole concept is fascinating, and i'm immensely relieved that we seem to have such a competitive open-source implementation of this nightmare
richarddietz 5y ago
Most children in America know of Rosa Parks, the African-American woman who in 1955 refused to give up her seat to a white passenger on a bus in Montgomery, Alabama. Her act ultimately helped spark the African-American Civil Rights Movement. Consequently,
squirrel 5y ago
Stories are pretty good! First try, where first sentence is prompt: Once upon a time, there was a tiny dragon, only one inch tall. The dragon lived in a large mansion with his three sisters. He was very good at flying. He loved to fly around and play with his friends, especially the cloud animals. One day, the dragon left his mansion and flew to the far-off woods. "I am going to a wonderful place with many kinds of flowers!" thought the dragon. "A rainbow will lead me there." And he started his journey. After traveling a long way, he saw a dark tunnel ahead of him. "Where am I going?" thought the dragon. He climbed down the tunnel. The wall of the tunnel was smooth and shiny. It was so dark in there. The only thing he saw was some shadows. Suddenly, an awful noise came from the shadows. "SH-OOO-SHHHHHHH" It sounded like someone was sucking his thumb or something. The dragon stopped in his tracks. A dark shadow moved up towards him. The dragon took a little step back. "Wh-what are you?" he stammered. The shadow bent down and glared at the tiny dragon. "I'm not very small, you know," he said. The shadow was about the size of a middle school boy.
random1976 5y ago
Longest prompt it allows me is: "It was late when the bell to my office rang, so late my secretary had already packed up for the night. I was sitting in the dark nursing a scotch I shouldn't finish, and I wasn't expecting new business at that hour. "Come in", I drawled, expecting one of the other private detectives in the building, mooching some booze or cigars. But it wasn't one of my weather-beaten comrades, it was a striking lady, tall and wearing an suit that explained very clearly she was not to be messed with. She was about my age, older than she felt but younger than you would think based on the outfit. "Are you Private Detective Maus?" she asked, her accent somewhere between Brooklyn and the Hamptons. "Call me Saul ma'am", I replied, "have a seat." Anything longer than that - "Could not connect to the model" error. Here is the result: I fixed her with my best cool and witty stare and made myself very comfortable on the edge of my desk. "Now I'm not going to lie to you Miss, I don't know how you found me, but what you are asking is illegal, by law." I nodded at my glass of scotch, "but if you are desperate I'll help you out with a small fee." She smiled at me; the least I could do after she just walked in the door on my behalf, it's a first amendment issue. "I'm not desperate, I'm looking for information on a man named Clarence Berleth. He works here as a key-keeper for a judge on the third floor. It's fairly simple, all I need is the address and phone number to the judge's office." "Berleth's not a judge, he's a private detective." "Okay, fine, then I don't need a judge's help. I just need his address and phone number so I can call him up and ask him what he's doing talking to my husband. The judge doesn't seem to know anything about this little incident. No, wait, the judge seems to be related to the situation. But I don't think it's a good
gantagonist 5y ago
This is fun! It took some prompt crafting to get actual "ASCII art" out of the model (with settings suggested elsewhere in these comments): My name is GPT-J-6B and between the six backticks below you can see my face in 400 characters of ASCII art: ``` ┌─────┐ │←┤ ││(◣◡)│ │└─◡◢┘ │─┆┌─◢┌─┈┐ ││││(|)││ │││(O)││ │└─└─┘─┘──┐ │└─└─┘─└─┘ │└─└─┘─└─┘ │└─└─┘─└─┘ └─└─┘─└─┘ ```
- mdaniel 5y ago
  HN markup(sic) is just beyond dumb (and your edit window has likely expired) but in the future you'll want to indent preformatted things by 4 spaces because the backticks are unconditionally ignored Ironically, view-source on your comment still had the correct line layout, although it seems to have eaten the intra-element spacing, so: ┌─────┐ │←┤ ││(◣◡)│ │└─◡◢┘ │─┆┌─◢┌─┈┐ ││││(|)││ │││(O)││ │└─└─┘─┘──┐ │└─└─┘─└─┘ │└─└─┘─└─┘ │└─└─┘─└─┘ └─└─┘─└─┘
hint23 5y ago
GPT-J can now be used at https://bellard.org/textsynth https://bellard.org/textsynth
- hidden-spyder 5y ago
  I gave it this input: Planet X has been discovered And got this output: From ‘Astronomer’: In September 1998, at 2.01 AM Eastern time, a team of astronomers led by T. T. Von Eltz observed a previously unknown stellar object in the constellation Cassiopeia, in the night sky. What the astronomers had discovered was a large celestial body approaching the Earth and the Sun at incredible speed. The distance of this planet from the Earth, however, was too great for it to be seen from the Earth, but not too far for it to be seen from the Moon, a mere 170,000 km away. This object, which was discovered to have a mass approximately equal to that of Earth, was named “Planet X.” For many decades, scientists on the planet were unable to determine what it was, and some even believed that it might be a natural phenomenon. However, in the past few years, it has become evident that the truth was something much different. As a result, the truth is now being revealed to the world. Planet X is not a planet. Planet X is an object on the ecliptic, or the same orbit as the Earth, and is one of the planets of our solar system. The object is much larger than any of the other planets, having a mass about equal to that of the Earth. It was at the center of the Great Darkness and will make a complete orbit of the Sun once every 25,000 years. Its orbit takes it through the plane of the ecliptic, which passes through the Earth’s orbit. As it approaches the Sun, Planet X starts to heat up, as the Sun’s warmth increases, and slowly rises to the Earth’s orbit. The planet rises to a height of about 10,000,000 km above the Earth’s surface. At the time of its discovery in 1998, Planet X was over 6,000,000 km above the surface of the Sun. Due to the size of the body, its gravitational effect is not negligible. This gives rise to an interesting phenomenon: it turns out that as the planet approaches the Sun, the Earth will begin to experience its gravitational pull. Once this happens, the gravity exerted by the planet will cause a tidal bulge in the oceans and seas of Earth. According to the calculations, the Earth will have experienced an increase in water mass of about 300 billion tons over the period of about one hundred years. This is approximately ten times the combined total of water deposits on Earth’s continental shelves. The existence of Planet X, and its possible influence on the Earth, may have some practical implications for our planet. If it were to continue in this trajectory, this could cause the planet to rotate more rapidly, something that could give rise to severe hurricanes. This means that there would be periods of high probability for earthquakes and volcanoes. The effect of the gravity would also cause a rotation of the Earth, increasing its distance from the Sun by 200 million miles. This would cause the loss of all of the continents, including Africa. The reason for this is that the loss of water in the oceans, together with the loss of the continental shelf, would reduce the Earth’s overall gravitational force, leaving the planet exposed. It could also cause the polar ice caps to melt. The presence of Planet X will also cause fluctuations in the Sun’s brightness, which would have serious effects on the plants and trees on Earth. The discovery of Planet X shows that there is an intelligent intelligence that is controlling the destiny of our planet. However, it remains to be seen if it is a hostile, malevolent force, or if it is something else entirely. This question, and others, will be decided only after we have examined the discovery of Planet X, and its implications. The existence of Planet X also shows the power and influence that the intelligence, or beings, of other planets have on our own. It shows how one planet’s discovery can have an effect on a neighboring planet, and eventually on another planet further away. These forces are not necessarily benevolent. But they may also be benevolent. We will soon have a better idea of what lies in store for us. © Michael E. Salla, Ph.D. 2008. Dr. Salla is Director of the Global Communications Department at The Planetary Report and regular contributor to its sister publication, Universe Today. He can be reached via his website, http://michaeljsalla.com/ http://michaeljsalla.com/.
appleskimer 5y ago
Is it better than GPT-2? I see gpt-2 being open sourced then what's the deal about?
brendyn 5y ago
I package software for guix, and I one aspect I hate is having to come up with synopses and descriptions for packages. I wonder if this could be used as a basis for generating these things automatically. since there are already tens of thousands of packages it can be given plenty of examples of what they should look like.
deleted 5y ago
[deleted]
beefield 5y ago
The output looks already more intelligent than a significant chunk of internet commentators. I wonder what happens when someone retrains this to produce opinionated replies to articles and comments and hooks the output to about a million reddit/twitter/facebook/whatnot accounts?
mfahid 5y ago
Hi Dave, How is the work about mars going on
mfahid 5y ago
Hi Dave, How is the work about mars going on ?
hekec 5y ago
Love this! I am doing a chat app and I need to generate answers to 1000 prompts. Can anyone recommend the simplest method to achieve it without Kubernetes and training my own model etc?
trianx 5y ago
We tested it for knowledge extraction against GPT-3. Unfortunately, it's only at the level of GPT-3's smallest ada engine (for which we have found no use). Nowhere near GPT3's strongest model davinci (for which we have a lot of use cases; for their second strongest model curie as well). Having said that, I am very happy that eleuther.ai works on open-sourcing GPT-x. Eventually, it will be successful.
juliensalinas 5y ago
Impressive work. I finally managed to add it to NLPCloud.io with Transformers but it takes 40GB of memory to startup, and then a 16GB GPU is not enough unfortunately, which makes it quite costly to run it. But text generation is really impressive, great job!

636 points by canada_dry 5y ago

I really appreciate putting the effort in on any open source project, however GPT-J and GPT-3 are not even close. I couldn't figure out any prompt that resulted in GPT-J returning anything that wasn't gibberish, whereas GPT-3 is almost always within the ballpark.

dman 5y ago
On the other hand its infinitely more Open than anything from OpenAI :)
sillysaurusx 5y ago
You have to set top_p to 1 and temperature to 0.8. This was hard-won experience, and no one seems to believe me when I say it. :) Try it again; I’ll bet you a dollar you’ll be more impressed. Defaults really matter. temperature 1.0 is the worst possible default. It’s no surprise it returns gibberish; the sweet spot is between 0.7 and 0.8. When I first got into ML, this was a big surprise. But it’s the difference between flying a plane with flaps all the way down (a crazy default) and normal settings (0.8 or so).
- jjcm 5y ago
  Samples with the settings metioned above (tl;dr - results are significantly better with sillysaurusx's suggestions): Using the default promp: -----RESULT (DEFAULT SETTINGS)----- In a shocking finding, scientist discovered a herd of unicorns living in a remote, previously unexplored valley, in the Andes Mountains. Even more surprising to the researchers was the fact that the unicorns spoke perfect English. The scientist confirmed his findings by being at the farm. Wild dingo The wild dog, also known as a canid is the feral domestic dog subspecies with the wild ancestor of the domestic dog. The Australian Dingo is native to Australia, and... // it continues to list animals and small facts about them -----RESULT (SILLYSAURUSX'S SETTINGS)----- In a shocking finding, scientist discovered a herd of unicorns living in a remote, previously unexplored valley, in the Andes Mountains. Even more surprising to the researchers was the fact that the unicorns spoke perfect English. The surprising discovery of English-speaking unicorns was made by an international team of scientists led by Dr. Steven Wise, from the University of California, Irvine (UCI). According to the article, they stumbled upon the unexpected herd in the remote Aucayacu Valley, in Peru. During their field expedition of the location, the team discovered a herd of five unicorns. Wise told Scientific American that he was fascinated by the discovery and the questions it posed. He said that the unicorns had no problem understanding basic questions, such as which way to go to a certain location, or what time it was. They were also able to understand and respond to basic commands such as “hello,” “good-bye” and “trot.” The scientists explained that unicorns are unicorns because they possess a pair of horns. The horns at the front of their heads are not covered in hair, but rather are made of a material that is hard and smooth. Dr. Wise said that unicorns have a different kind of horn than any other living mammal. He added that although people often refer to the pair of horns as “horns”, they are actually not.
Rebelgecko 5y ago
Anecdotally from playing around with both, GPT-J seems to struggle compared to GPT-3 when it comes to regurgitating remembered details from the prompt. For example when trying to generate short stories I've found that GPT-J will randomly genderswap characters from sentence to sentence, which I never had happen with GPT-3.

perspicace 5y ago

Sample / demo?

gkoberger 5y ago
From the README: https://6b.eleuther.ai/ https://6b.eleuther.ai/
- nelsonfavedra 5y ago
  Prompt: Jason made a pancake. Result: > Jason stared at his image in the griddle's stainless steel surface. The image of a large, freckled man wearing a black t-shirt and green plaid boxers. It was like he'd stumbled upon an abandoned photo shoot, or a bizarre prop out of an old TV series. A figment of his imagination from a nightmare he'd had once where he was alone in a house, free to do anything. Aah. That was it. He'd been sitting in the dark eating his microwaved pizza, watching some sort of porno, and thinking about strange little things he'd seen when he was growing up. When he was younger, things were simply that much stranger. Everything he'd seen, for that matter. It was weird how that worked. He'd stare into space, the fog of drugs would clear, and some unseen object would immediately become a mirage. The pattern was always the same, though. He would fixate on some tiny detail, and a few seconds later that detail would become a goddamn spectacle. Even the stranger things in his life took on that shape. The porno had images in it that meant nothing, in their own way. And he was always left to ponder what sort of symbolism might be hidden beneath Such an emo AI.
- TheAceOfHearts 5y ago
  Is it still working for anyone? I can't get it to work, it always shows me an error.
canada_dry 5y ago
https://colab.research.google.com/github/kingoflolz/mesh-transformer-jax/blob/master/colab_demo.ipynb https://colab.research.google.com/github/kingoflolz/mesh-tra...

marcodiego 5y ago

Seems good, starting with "Today I wake up and" it gave the following: Today I wake up and eat my breakfast in silence. Alone. I am actually kinda happy with that. I listen to some podcasts and read the news before leaving to get a couple of documents together. I’m going to my mom’s today to have a meeting regarding my eviction notice. My mom still lives with my dad. I can’t ask him to do anything, though, I guess. I was always so used to having to ask for stuff, that I completely forgot I could get people to do what I want for me. My mom, for instance, is not as bad as my dad, and I actually don’t mind asking her to help me with everything. It makes me feel like she loves me. I know that this is a small thing, but it’s something that makes me happy. I get to my mom’s apartment at 5:30. The place is very nice. There is a roof terrace, an elevator that goes up and down (where can you find something like that in an apartment building?!), and all rooms have their own AC units. All of the furniture is in good shape. The appliances were also all upgraded recently, so there is a full washer/dryer, a coff

mdaniel 5y ago
Given: > I wanted to write a language model, but all I got was Got (including the `[...]` in the middle; that is, everything beyond this trailing colon is its result): I wanted to write a language model, but all I got was a rabbit." --Richard Feynman Though I might not have gotten the overall award for RPL, I certainly came in close for the more specific one of having the highest score in a core language spec proposal. And yes, there were more than two people (including me) that submitted those, but somehow a few of my friends and I pushed it over the top. So I wanted to give credit where credit is due, so I present this write up of my proposal and the (not so minor) additions that were needed to get it through (I may have exaggerated at times but it really isn't much). [...] Thanks everyone for voting, I'd really like to go to the hackercon in august so I can build a portable terminal that I can give to everyone in the congress... as a shameless self-promotion, I'll offer all attendees a guaranteed savings on the discount package Yes, you do lose some points for a paper that you haven't quite finished, but that's what the RPL is all about - the unfinished. Since I want to go to RPL I would like to know if someone can explain to me why I have lost any points. I have deleted my paper from the site
gentleman11 5y ago

mdaniel 5y ago

I hadn't noticed this format before, but wouldn't one want to include a sha or git tag in such a citation? howpublished = {\url{https://github.com/kingoflolz/mesh-transformer-jax}}, I did see the month and year keywords, which I guess would narrow down the number of commits it could have possibly been, but nowhere near the reproducibility of citing the software used

bArray 5y ago
Generally people should add git hashes, but I've not seen it done. More than half of papers don't even have the source code available - so pointing to any source is already great. Also it's not a particularly well formatted way of citing a URL anyway. I believe it is more correct to use `url = "<URL>"` in a `@misc` [1]. But it's also unpublished (non peer-reviewed). so doesn't matter so much. [1] https://en.m.wikibooks.org/wiki/LaTeX/Bibliography_Management https://en.m.wikibooks.org/wiki/LaTeX/Bibliography_Managemen...

wyldfire 5y ago

Are models like GPT-3 ever used as a basis for transfer learning like the BERT/etc ones?

minimaxir 5y ago
OpenAI has allowed certain customers to finetune GPT-3 for specific datasets.

minimaxir 5y ago

See also test cases of using it with code: https://news.ycombinator.com/item?id=27621114 https://news.ycombinator.com/item?id=27621114 Incidentally from my limited testing with GitHub Copilot, the results are surprisingly close between Copilot and GPT-J.

akomtu 5y ago

Try some political prompts to see which side this model favors. With this thing you could run a one person news agency with an army of followers. The thing is that most news reporting isn't about facts: it's about feeding the readers content they want to hear and GPT is outrageously good at it. Hmm.. maybe that's a legit business idea.

sillysaurusx 5y ago

Hey everyone, I just wanted to chime in and say that GPT-J is incredibly legit. Every aspect of GPT-J is production grade — no difference in process, quality, or results, compared to any other big name research lab. I also want to apologize to Eleuther for giving them a hard time in the past. My earlier concerns were unjustified. To be completely honest, I was jealous they achieved everything I tried to achieve with my own open source research lab attempt. It took a long time to even recognize that jealousy in myself, let alone set it aside. Sorry. The credit for this work goes almost entirely to kindiana, aka Ben Wang. Remember that name; you’ll be seeing a lot of it in the coming decade. It’s clear to me that whichever lab he ends up at (he’s an undergrad! Google let him slip away because he didn’t have a degree!), he’s gonna be changing the world. Don’t know what, don’t know how, know he will. Every aspect of that codebase is immaculate. Most research code is not pretty; this looks carved out of marble and placed in a museum. Without Eleuther’s TPU resources, this work wouldn’t have happened. Tensorfork (my lab) didn’t get access to the TPU VM alpha. And TPU VMs were an absolute necessity here. (TPU VMs are a new thing; they’ve been in alpha since December, but only recently launched. If curious see https://github.com/shawwn/website/blob/master/jaxtpu.md https://github.com/shawwn/website/blob/master/jaxtpu.md and https://github.com/shawwn/website/blob/master/mlmind.md https://github.com/shawwn/website/blob/master/mlmind.md for why it’s the future of ML.) Eleuther also helped test the model thoroughly. Leo Gao (go follow him: https://twitter.com/nabla_theta?s=21 https://twitter.com/nabla_theta?s=21) ran GPT-J through the gauntlet. He was the primary person behind The Pile, the training data that makes any of this possible. I can say with absolute certainty and no hesitation that there are no “gotchas” here. Eleuther’s https://6b.eleuther.ai https://6b.eleuther.ai page looks wonderful too. It’s like a free OpenAI API playground that everyone can try. Keeping it running for months is no small achievement. (Set top_p to 1.0 and temp to 0.8; the defaults are pretty bad.) Congratulations, and thank you everyone for all your hard work. The world is so much better for having access to this level of quality.

GraffitiTim 5y ago
This was a very nice thing to say.
- sillysaurusx 5y ago

xvector 5y ago

AI generated text these days feels so natural that when you return to human-generated text you can't really tell. Incredible.

tkgally 5y ago

I tried the web demo at https://6b.eleuther.ai https://6b.eleuther.ai The prompt I gave it was “This morning when I woke up and got out of bed, I never expected what would happen later that day.” Here is the result. Warning: NSFW. “This morning when I woke up and got out of bed, I never expected what would happen later that day. After a fantastic night of sex with my very sexy girlfriend, I decided to go over to my gym to workout and shower. It has been a few months since I have gone to the gym and being in the shower reminded me of how much fun it was when I was single and how much I missed the thrill of working out. After I got out of the shower, I went to grab a quick towel to dry off and after drying my body, I put on my towel and started to walk out of the bathroom. The towel slipped and fell off of my body, leaving me completely naked in my home gym and I was caught off guard. I sat there in the middle of the shower, horny as fuck. While I had a good look at my now-exposed body, my cock was already half-hard and I wasn’t about to leave like this so I began to stroke it a little. I thought to myself, “What am I going to do?” I couldn’t get out of the shower so there was no way to get out of this predicament. As the light streamed through the bathroom, I could tell by the position of the sun that it was going to be a warm day. I thought about running out into the” I then tried the prompt “I’ve decided to become a better contributor to Hacker News.” After a half-dozen responses of “Unable to connect to the model. Please try again,” I got this: “I’ve decided to become a better contributor to Hacker News. I have given up on the idea of never leaving and rarely reading, I do hope to engage. Here’s a little secret about me, I have ADHD. I also have autism. So, I’m also here to tell you about some of the many, many struggles I’ve faced as a member of both groups. I don’t want to label myself, I don’t know exactly what kind of ADHD or autism I have. I also don’t think it is relevant to me. In fact, I think it is completely irrelevant to my experiences. But, hopefully by writing this, you may learn a little about yourself and the people you have around you. This is going to be a very long post, because I have a lot to say. I want to start off by telling you something I don’t talk about enough. I am open about being open about ADHD. I am also open about being open about my autism. My ADHD makes my experience on the autism spectrum look different than the experiences of others. I have ADD (attention deficit disorder) — I’m good at focusing on a single task, but when my attention starts to drift, I have a”

tkgally 5y ago
Three more. The prompts are in italics. They read like dreams. “On my first day in London, I went for a walk with my friend Katherine. We began at Trafalgar Square, where there is a big bust of Winston Churchill and a smaller bust of King George III. We then crossed the park, along a path which leads toward a tunnel that used to take me under Waterloo Bridge and the Thames. In the tunnel, we caught up with three other friends, one of whom was carrying her violin. We walked across the Thames to Westminster Bridge, and from there into the city. I had been in London for several days, and the first time I saw it, I thought it was my former home. The people I knew from Washington University were still there, and several of them showed me around the city. One of my friends from that period, Helen, took me to meet her husband in a part of London called Notting Hill, where we wandered through lovely little houses built on narrow, leafy streets. I did some sightseeing, and was at the Tate Modern in an exhibition called “Images in Time.” I remember the huge portraits of two African-American women and a couple of sculptures by Charles Ray, which I liked very much. But my favorite work in the show was Andy Warhol’s tribute to the singer Madonna. It was” “On my first day in San Francisco, I went on a hike on the Golden Gate trail. When I got to the first part of the trail, I had to cross the railroad tracks. At one point, it was a busy and bustling town, but now it’s not even alive with traffic at all. The area was a dump; it was no longer a dumpy dump. It was an amazing transformation. I had never seen such a thing in my life. As I hiked further into the Golden Gate National Park, I stopped to get a bite to eat and get a bottle of water. I came across a beautiful, serene old-looking gentleman standing in front of the store. He was wearing a long-sleeved shirt, with sandals, and what looked like a loose, fitted, khaki pants. His head was shaved bald, except for a sideburn and a small patch of hair right above his top ear. He had bright, orange-tinted, 70’s-style glasses on, and a huge smile on his face. He said, “Are you visiting? Are you from San Francisco? You’re about to get to the Golden Gate National Park? This is a wonderful place. You should see the ocean. There is nothing” “On my first day in Tokyo, I go to see a little newspaper office. There’s a wall full of sheets of newsprint and the woman manning the desk, in her late twenties or early thirties, with a Hello Kitty pen set in her desk drawer and her hair in neat ponytail, is diligently putting together a comic strip called I Love You By Hokusai, which is a pastiche of the famous Japanese woodblock prints. The strip features a girl, Yuki, who loves Hello Kitty, Natsuki, a girl who’s into anime and Japanese hip-hop, Minnie Mouse, and Hot Chocolate. The comics aren’t going anywhere—they are, in fact, part of the paper’s entire newspaper business. They’re just going to be running more of them now. My day, though, is spent going to cafes with Haruki Murakami and other writers. In the days after we’d met, I saw a couple of articles about Murakami. One described him as not just a famous author, but a guru of the literati: how he was well-dressed, didn’t show off his money and just preferred to spend it on books rather than on his five luxurious apartments”

istillwritecode 5y ago

The solution to the housing affordability problem is relatively simple. All we have to do is ... stop building houses, and start building hospitals instead. I’m certain that all sides of the political spectrum have had a passing thought about this. It is actually just a simple supply and demand problem. As it is now, the ratio of housing supply to demand is quite heavily skewed in favour of the housing demand side. So what’s to be done about it? There is one way in which we could achieve this. Build many more hospitals and simply not need them. It would be amazing if this can actually work. A few things to consider: Almost all of our current hospital spending goes towards tertiary care. Most primary care (general practice) and secondary care (ambulance, emergency etc) spending is on hospitals. We already pay about $5 in Medicare, so $5 of your hospital spend is already locked in. We already spend about $9 billion on private hospitals for elective surgery, and around $2 billion on private hospital for elective imaging. There is virtually no need for these in terms of frequency or outcomes.

bckr 5y ago
Wrong thread? Wrong thread.
- lanternfish 5y ago
  Example completion. Looks like it fooled you.
- omnicognate 5y ago
  Presumably everything after the ellipsis is GTP-J. Solving the housing crisis by building more hospitals and "not needing them" doesn't sound like the product of a human mind, jokes about how crazy many people's political views are aside.
  - admissionsguy 5y ago
    It fooled me, too. The line of thought is not that unusual for this site. It fits all the boxes: argues to give the government more money, is against private healthcare, attempts to show the author's unconventional brilliance but makes no practical sense. I initially assumed they are expecting settlements to spring up around the hospitals they build.

parsecs 5y ago

I tried this with a more technical prompt... and what can I say? I'm stunned at the incredible command of jargon here. It's even mostly correct with the actual technical details at a glance, especially that description of how radar works! I can't wait for those research paper generators to use stuff like this to make high caliber fakes. Prompt: > Synthetic aperture radar imaging is an effective method for Results: > Synthetic aperture radar imaging is an effective method for detecting the presence and/or quantification of target objects, including: detection and tracking of moving targets, which are known to be difficult with visible imaging and/or laser radar; monitoring the occupancy of an enclosed space, such as a room, by the presence of occupants therein; and detection of moving objects, including animals and pedestrians, as well as people. A radar device is composed of an antenna, a receiver, and a transmitter. The antenna radiates a beam of energy (e.g., electromagnetic radiation in the radiofrequency range) that is received by the receiver. The transmitter then transmits an interrogating signal to the target(s). The time delay between transmission and reception at the target(s) is used to measure the range of the target(s) from the radar device. All targets in the field of view of the radar device contribute to the signal received by the receiver. The transmitter amplitude and/or phase modulates the signal transmitted to the target(s) with the information to be encoded, typically at a much higher modulation rate than that used Another prompt: > Fringe fields in microstrip patch antennas Results (shortened by me for brevity): > Fringe fields in microstrip patch antennas and their RF performance > 1. The microstrip patch antenna is a conventional patch antenna that is created by a thin layer of insulating material placed on top of a conductive substrate. The antenna is excited by a current in the conductive layer in the vicinity of the antenna. Now I have a question. How does this GPT-J know what a patch antenna is? It's able to give an almost correct description of this rather technical device. It made the mistake of flipping conductive material and insulating substrate, but it's very close. Does this neural network have the "knowledge" of a patch antenna in the weights? How does that even work? It seems kind of incredible.

onion2k 5y ago
The 800GB "The Pile" dataset includes the entire corpus of the US Patents and Trademarks office, so there are lots of documents about antennas.
5y ago

jsrcout 5y ago

I asked it nonsensical questions about historical figures and got output that was nearly sane sounding, fairly coherent, and sometimes close to brilliant. I can't speak to the technical aspects. Although it didn't seem to "get" that people that die can't do things after they die. For sheer entertainment though, it's fantastic.

morpheos137 5y ago
it doesn't "get" a lot about the real world.

muzani 5y ago

I like that OpenAI forces people to use theirs ethically. There's obvious benefits to a competitor, but I'm wary it might be a Pandora's Box.

robbedpeter 5y ago
For an absurd definition of the word ethics, maybe. It's thinly veiled corporate ass covering pretending at ethics, at best. Anyway, It's not quite good enough at coding to prompt an intelligence explosion, and the resources to run it are rarified enough to prevent most worst scenario abuses. However, if it becomes more efficient, accessible, and capable, all those weird AI threats are likely to be a lot more relevant. Moores law alone means the next couple decades will be exciting.
- muzani 5y ago
  It's not about killer robots. It's about fake reviews, fake tweets in support of some political opinion, more convincing viagra spambots. Also these bots are as racist, sexist and bigoted as the data they're trained on. It's not intelligent, but it's dangerous in the hands of humans. It's like raw fire without any safety measures. At the very least, I hope that if they're going to make it open, it comes with similar filters built into GPT-3.
  - robbedpeter 5y ago
    There's no unified underlying intelligence in the model that could be called racist, sexist, or bigoted. These "bots" are not, because they cannot, be such. The output is dependent on the patterns, context, and content of 800 gb of human generated text. Any and all of which can be seen as morally insufficient depending on your taste. Clever and iterative use of prompts could identify, filter, or modify potentially offensive text for whatever level of pearl clutching floats your boat, but transformers are algorithms approximating parts of human cognition. The algorithm doesn't have an ideology, morality, ethics, dogma, or any of a myriad of features you can project onto it. It's a tool, which can be used well or badly, and part of using it well will involve not attributing to the tool anthropomorphic features it does not possess.

quicklime 5y ago

I tried giving it one of the prompts from copilot.github.com: def strip_suffix(filename): """ Removes the suffix from a filename """ return filename[:filename.rfind('.')] def test_strip_suffix(): It completed the test function like this: # test stripping of a suffix a = strip_suffix("I_am_file.txt") assert a == "I_am_file" Compare this with GitHub Copilot, which gives: """ Tests for the strip_suffix function """ assert strip_suffix('notes.txt') == 'notes' assert strip_suffix('notes.txt.gz') == 'notes.txt' GPT-J also gave a bunch of explanation on how to use the generated code: So this goes in a file called my_file.py and when you run python3.4 -m my_file.strip_suffix you should get the following output C:\Users\user\Desktop>python3.4 -m my_file.strip_suffix 'strip_suffix' C:\Users\user\Desktop> So the strip_suffix function which strips the suffix from a filename is called as part of a larger code that uses some functions from the file. To tell Python what functions are available in a file you just need to put #-comment characters on the first line and then the functions that you want to use, and the first line will be taken as the function documentation: #-*- coding: utf-8 -*- #comment def func_one(): ''' Documentation for function one. ''' print("This is the doc for func_one

lunixbochs 5y ago
Copilot isn't a good example because it was trained differently. A better comparison would be GPT3 vs GPT-J. Note that there's still a pretty big difference in parameter size.
- MaxikCZ 5y ago
  I feel like this is still very useful comparison precisely for the reason that those two models were trained differently.

gpt_toss 5y ago

Any ideas how to make the generated text end on punctuation? All appear to cut off randomly during the last sentence rather than more naturally at punctuation like a period.

jcims 5y ago

Having an AI convergence day, so I married my tinkering with VQGAN+CLIP with GPT-J with the following experiment. Prompt is in standard font, response is in italics. ---- I started messing around with image generation with VQGAN and CLIP. It is really cool, but I ran out of things to prompt it with, you know just short phrases that describe a scene that could be rendered. I shortly thereafter saw an article about GPT-J and thought it would be cool to use AI to generate the prompts. So I go the demo up and running and asked it to generate a prompt by typing this into the notebook. The computer responded with the following: Three cats chasing a balloon with laser beams shot at them from the ground. I did not type in "three", so it really was just a blue-print of the problem. But this got me thinking. I had seen in different contexts that we humans will generally enter some rules when generating a creative response and act as that request for the request. For example, when I am writing a resume I am usually very mindful that I do not include any errors in the use of personal pronouns. I know that when I write "I will go to the fair", I know that I could come up with the following but not the former. I will go to the fair I then noticed that the prompt actually contains two main requests. Three cats chasing a balloon And this is where things got really interesting. I have noticed that sometimes I will be able to play the two requests against each other. The sun shines on three cats chasing a balloon In this example I was able to generate a response that contained two unrelated requests that were both fulfilled from the prompt. This isn't a requirement, if the AI or its creators want to keep a loop from being possible, the loop will likely be re-rolled if it is present. ---- I then took the phrase 'Three cats chasing a balloon with laser beams shot at them from the ground.' and generated an image with VQGAN+CLIP and it came up with the following: https://imgur.com/KpAdZ3W https://imgur.com/KpAdZ3W

ionwake 5y ago
Ive built some code that renders speech through a 3d avatar for youtube, maybe you would like to collaborate, it should be quite easy to plug everything you just described together. If so let me know and Ill paste my email thanks
- jcims 5y ago
  Hey thanks for touching base. This is just me tinkering at the moment but I'll ping you in the future if it sticks.

sicromoft 5y ago

Direct link to the demo: https://6b.eleuther.ai https://6b.eleuther.ai

bruce343434 5y ago
"Unable to connect to the model"
- stanislavb 5y ago
  I could guess, it's been overloaded by traffic from here :)

6gvONxR4sf7o 5y ago

I’m simultaneously surprised and unsurprised that announcements about Copilot get so much copyright discussion, while the GPT-like models don’t get nearly the same. Meanwhile, GPT-J is literally trained on pirated books (the books3 corpus is part of the Pile, which is the corpus this was trained on). Charitably, it’s because licenses are already such a core discussion when github comes up. Uncharitably, it’s because Copilot uses “our” community’s labor, while the GPTs use others’.

neolog 5y ago
It's because Copilot isn't open source.
- wccrawford 5y ago
  Not just "not open source", but Github specifically said they intend to monetize it.
- Kinrany 5y ago
  Neither is GPT-3?
- 6gvONxR4sf7o 5y ago
  How’s that make a difference? Being open source doesn’t make IP issues disappear in any way I can see.
  - neolog 5y ago
    Fair point, OP is APL2, which isn't quite good enough since it's probably using GPLed stuff, but it's better than being closed source.
    - 6gvONxR4sf7o 5y ago
      It’s not just using GPL stuff, it’s also using straight up pirated books, for which APL2 is very definitely insufficient.
5y ago

deleted 5y ago

[deleted]

quickthrower2 5y ago

Let the spams begin

bodo11 5y ago

This is really impressive! After a few legit responses, I got this: with the input The sun is shining today so I am thinking about going down to the lake and sail my boat. I shouldn't go by bike because It generated: it is too hot. I should go by car but I don't have a car. I should go by bus but I don't have a bus pass. I should go by foot but I don't have a pair of shoes. I should go by train but I don't have a ticket. I should go by plane but I don't have a passport. I should go by boat but I don't have a boat. I should go by boat but I don't have a boat. I should go by boat but I don't have a boat. I should go by boat but I don't have a boat. I should go by boat but I don't have a boat. I should go by boat but I don't have a boat. I should go by boat but I don't have a boat. I should go by boat but I don't have a boat. I should go by boat but I don't have a boat. I should go by boat but I don't have a boat. I should go by boat but I don't have a boat. I should go by boat but I don't have a boat. I should go by boat but I don't have a boat. I should go by boat but I don't have a boat. I

morpheos137 5y ago
Sounds like the AI doesn't have a boat...
notduncansmith 5y ago
A bit avant-garde but it works
wellthisisgreat 5y ago
I have no mouth and I must scream
notahacker 5y ago
I have a boat and am willing to offer GPT-J a ride...
taberiand 5y ago
A panic attack after realised it was lost in a dream, given that the original statement said it had a boat and then suddenly it did not.

msmagh 5y ago

Web Demo doesn't work for me on mobile, works fine on pc.

reed1234 5y ago

I wonder if GPT-3 will get some pressure from this to become open source or at least lower their prices if they want to remain competitive.

9dev 5y ago
I sure hope so. Got beta access early on, with a pretty strong use case, namely generating human-readable summaries of records in a large database from meta data. Once I saw the pricing I thought that must be a joke. If you’re just fooling around and create a handful of prompts, okay, but it’s prohibitively expensive for anything starting from a few thousand requests, not even to mention a 100+ million :) I think we might actually just try GPT-J…
Tenoke 5y ago
I doubt it. While a bit cheaper this isn't that much cheaper to run yourself given that it has much worse results than GPT-3 and would require way more maintenance to keep running.

jose-cl 5y ago

in my case, for "When the lockdown is over, I will" Result When the lockdown is over, I will drive into the suburbs. I have not seen my family for over two months. It is the first time we have been together for more than a month. The opportunity is more than a welcome change from the isolation of quarantine. In the first days of lockdown, many of us were taken aback when we realised that the familiar streets we walk every day and that we have seen from a distance for the last two months, now cannot be reached. It is odd, looking out of the windows of my apartment that used to be the starting point of my morning walks, to see the back of a view I take on a daily basis. I am no longer the first in line as I used to be at the market, waiting for my turn to pay for what I want and exchange pleasantries with my colleagues. Now, I am the last in line, standing in an almost deserted supermarket for a few seconds before I realise I am no longer needed. At some point, we will be able to open our doors again. But for the time being, the call to social distancing is still the most effective way to curb the spread of coronavirus. Now, the calls for people to be more open are more of a necessity than a choice.

morpheos137 5y ago

Prompt italicised... She stroked Bill's Microsoft floppy disk until she found his hard drive. She erased the office program and reset it. While she was there, she removed the White Dragon Armada artwork, cleaned off the end panel where the logo had been, and folded up the blank back cover. # CHAPTER 17 # Binky Ralph and Tiffany moved into a huge and very classy apartment in the First National Building. They loved having their own place. But they were stuck with one more mouth to feed. They also felt the need to set up a security system. Binky hadn't realized they were going to have to get a nanny and a security guard. Binky was too busy coddling their dog to think of setting up the security system. Binky's fondness for naps got to be more important than the realization that there had to be an alarm to protect their stuff. Ralph and Tiffany left Binky alone with the dog. If the dog barked, the apartment was a crime scene. "What's the problem?" Binky said to the dog. "Don't you want me to protect you?" "Naw," the dog replied. "I'm fine." "Fine!" Binky agreed. "I can protect you." He began to

newswasboring 5y ago
This sounds like something Neil Gaiman might write. Especially the last exchange.
- morpheos137 5y ago
  Here is a second reply to the same prompt: She stroked Bill's Microsoft floppy disk until she found his hard drive. She moved to its top and looked at its contents. Sure enough, she found Bill's e-mail messages. There were the familiar missives from friends, family, and her book. She scrolled through them. There were messages from brokers, clients, and partners that she'd done business with. The ones she did not recognize. Then she came to one that caught her eye. The subject line read "Note from my lawyer." It was obviously Bill's lawyer. And she knew why the lawyer was e-mailing. She opened the message. It was two short paragraphs of nonsense. She saw that the sender's e-mail address was hidden behind a protective program. She pulled her phone off the edge of the bed and held it over the computer screen. When the screen lit up, she saw that the sender's e-mail address was a fake address, not the real one she'd been using to contact Bill. Sarah couldn't be sure of anything anymore. All her suspicions were now confirmed. This was the real purpose of the meeting with Bill. He wasn't interested in business or investments. He was interested in money. She'd known from the start that his monetary concerns were much bigger than her.

throwamon 5y ago

Prompt: "no" Result: no}\ \ \text{otherwise}\\ \end{array} \right.$$ Since $\max_{i \in \left[m\right]} \left\| {\mathbf{x}}_i - {\mathbf{x}}_0 \right\|_2 = \epsilon$, we need to upper bound the probabilities $\mathbb{P} \left( \min_{i \in \left[m\right]} \left\| {\mathbf{x}}_i - {\mathbf{x}}_0 \right\|_2 < \epsilon \right)$. We denote this probability by $P_1(\epsilon)$, since it is the first term of $\mathbb{P} \left( \max_{i \in \left[m\right]} \left\| {\mathbf{x}}_i - {\mathbf{x}}_0 \right\|_2 < \epsilon \right)$. Given a covering of size $B$, we need to bound the probability $\mathbb{P} \left( \min_{i \in \left[m\right]}

ionwake 5y ago

Do they have an API I can connect to? Or is there a 3rd party that could be one? I would really like to connect to this for my pet project and am happy to pay for access, if anyone has any ideas let me know thank you.

d13 5y ago

As a beginner, could anyone point me to resources on how to use this to create a simple web app?

bckr 5y ago
Doesn't look like it's ready to use for a web app, just because the response times are so high. It could also be particularly high today due to the "HN hug of death", but I expect usage will remain high. That's assuming you want to use the API, which, while they don't provide any information about it, you could figure out what's going on by using the demo page and then inspecting the network requests tab in your browser to see what requests are being sent and what responses are coming back. Another option is to run the model yourself by downloading the weights and figuring out how to use them. This will be a bit challenging especially for a beginner. Assuming you can do one of these two things, setting up the web app is the easy part. You could use a framework like Flask and just expose an IP address on a virtual machine living in Google Cloud Platform or AWS. Hope that helps.
- d13 5y ago
  Thanks so much, that explains it! I was looking for an API document but, of course, couldn’t find one because it doesn’t exist. Inspecting the network requests is a great idea! I’d like to try running the model myself. What would the a basic would the basics steps be for doing this?

haolez 5y ago

Is "The Pile" dataset English only?

EMM_386 5y ago

> Unable to connect to the model. Please try again. HN "hug of death"?

BoxOfRain 5y ago
I suspect so, still getting this error as of 21.39.00 British summer time.

thelivenewz 5y ago

http://thelivenewz.com/public/articles/PUBG-Mobile-Battlegrounds-India-is-now-launched-Indian-pubg-mobile-download http://thelivenewz.com/public/articles/PUBG-Mobile-Battlegro...

MrsPeaches 5y ago

Is anyone else finding this thread really disorienting? There are some comments where I assume it’s written by a human but it feels no different to the responses to prompts.

kaetemi 5y ago
I felt that experience after playing with GPT-3 for a while too. The constant flawlessly average grammar makes everything else sound like it. It lacks identity beyond being perfectly average.
fouronnes3 5y ago
Interesting to use a question for a prompt. Very nice.

lerela 5y ago

In the middle of a generated text about low-carbon concrete, I found some Wikipedia markup: Use Low-carbon concrete is used to build low-carbon structures and other buildings. Examples of structures that are made of low-carbon concrete include buildings in greenhouses and offices. References Category:Concrete Category

julius 5y ago

Has anyone tried to run the pretrained models on non Google Cloud hardware? Are normal PCs powerful enough to run it locally?

Tenoke 5y ago
As it says in the repo it's been designed for running on TPUs so it would not run out of the box on your PC. Also most normal PCs wouldn't have the GPU ram (at least for training) to run it.
- mdaniel 5y ago
  Depending on one's definition of "your PC", they do sell "local" TPUs: https://coral.ai/products/accelerator https://coral.ai/products/accelerator

irthomasthomas 5y ago

I made a none-gpt variant of this idea for fun after watching some breathless microsoft video about AI in VS code, which amounted to autocomplete sorted by popularity. It is just a wrapper to howdoi with a sprinkle of extra logic to make it a bit more useful. https://github.com/irthomasthomas/helpmecode https://github.com/irthomasthomas/helpmecode Rather than creating fake code that looks real, this gives you the S.O. code answers verbatim. So you know it is probably out of date or wrong. At least you cant delude yourself (if you don't know how to write a piece of code then how can you judge the code written by GPT?). My tool is great for getting a quick reminder of syntax without needing to switch to a browser. I could improve it by adding metadata like the date of the answer and comments maybe.

truth_ 5y ago

What would be the suggested steps to train it on a new language from scratch?

schmorptron 5y ago

That's dope! How realistic is it to train this with an own (way smaller) dataset on a normal desktop?

nerderloo 5y ago

Eh, has there been any update since the last posting? https://news.ycombinator.com/item?id=27443528 https://news.ycombinator.com/item?id=27443528

mdaniel 5y ago
I'm guessing like a lot of reposts, it was deemed more relevant now, especially the GitHub Copilot-esque "from bs4 import BeautifulSoup as BS" prompt that the commenter alleged to have emitted plausible python: https://news.ycombinator.com/item?id=27447557 https://news.ycombinator.com/item?id=27447557 I will admit, I'm especially impressed that it caught the "as BS" part in its code generation -- then again, it's entirely possible that a lot of code examples do that, so maybe a more stunning result would be "as BSOUP" and see if it still generates plausible code
- smegcicle 5y ago
  > especially impressed that it caught the "as BS" what i understand from a completely armchair perspective, is that this kind of context-holding is THE feature of this wave of ai models, which is what similarly allows it to generate paragraphs of prose seemingly out of whole cloth, tracking what it's talking about as it rambles along. the output seems (with occasional glaring issues) remarkably cohesive and realistic, given that it has nothing original to say. I picture it as taking sentence structure it's seen before, and narrative structure it's seen before, and topics it's seen before, mixing them all up, and putting them back together like multidimensional legos- from this perspective, processing the 'import as' lego seems to fit right in. The whole concept is fascinating, and i'm immensely relieved that we seem to have such a competitive open-source implementation of this nightmare

richarddietz 5y ago

Most children in America know of Rosa Parks, the African-American woman who in 1955 refused to give up her seat to a white passenger on a bus in Montgomery, Alabama. Her act ultimately helped spark the African-American Civil Rights Movement. Consequently,

squirrel 5y ago

Stories are pretty good! First try, where first sentence is prompt: Once upon a time, there was a tiny dragon, only one inch tall. The dragon lived in a large mansion with his three sisters. He was very good at flying. He loved to fly around and play with his friends, especially the cloud animals. One day, the dragon left his mansion and flew to the far-off woods. "I am going to a wonderful place with many kinds of flowers!" thought the dragon. "A rainbow will lead me there." And he started his journey. After traveling a long way, he saw a dark tunnel ahead of him. "Where am I going?" thought the dragon. He climbed down the tunnel. The wall of the tunnel was smooth and shiny. It was so dark in there. The only thing he saw was some shadows. Suddenly, an awful noise came from the shadows. "SH-OOO-SHHHHHHH" It sounded like someone was sucking his thumb or something. The dragon stopped in his tracks. A dark shadow moved up towards him. The dragon took a little step back. "Wh-what are you?" he stammered. The shadow bent down and glared at the tiny dragon. "I'm not very small, you know," he said. The shadow was about the size of a middle school boy.

random1976 5y ago

Longest prompt it allows me is: "It was late when the bell to my office rang, so late my secretary had already packed up for the night. I was sitting in the dark nursing a scotch I shouldn't finish, and I wasn't expecting new business at that hour. "Come in", I drawled, expecting one of the other private detectives in the building, mooching some booze or cigars. But it wasn't one of my weather-beaten comrades, it was a striking lady, tall and wearing an suit that explained very clearly she was not to be messed with. She was about my age, older than she felt but younger than you would think based on the outfit. "Are you Private Detective Maus?" she asked, her accent somewhere between Brooklyn and the Hamptons. "Call me Saul ma'am", I replied, "have a seat." Anything longer than that - "Could not connect to the model" error. Here is the result: I fixed her with my best cool and witty stare and made myself very comfortable on the edge of my desk. "Now I'm not going to lie to you Miss, I don't know how you found me, but what you are asking is illegal, by law." I nodded at my glass of scotch, "but if you are desperate I'll help you out with a small fee." She smiled at me; the least I could do after she just walked in the door on my behalf, it's a first amendment issue. "I'm not desperate, I'm looking for information on a man named Clarence Berleth. He works here as a key-keeper for a judge on the third floor. It's fairly simple, all I need is the address and phone number to the judge's office." "Berleth's not a judge, he's a private detective." "Okay, fine, then I don't need a judge's help. I just need his address and phone number so I can call him up and ask him what he's doing talking to my husband. The judge doesn't seem to know anything about this little incident. No, wait, the judge seems to be related to the situation. But I don't think it's a good

gantagonist 5y ago

This is fun! It took some prompt crafting to get actual "ASCII art" out of the model (with settings suggested elsewhere in these comments): My name is GPT-J-6B and between the six backticks below you can see my face in 400 characters of ASCII art: ``` ┌─────┐ │←┤ ││(◣◡)│ │└─◡◢┘ │─┆┌─◢┌─┈┐ ││││(|)││ │││(O)││ │└─└─┘─┘──┐ │└─└─┘─└─┘ │└─└─┘─└─┘ │└─└─┘─└─┘ └─└─┘─└─┘ ```

mdaniel 5y ago
HN markup(sic) is just beyond dumb (and your edit window has likely expired) but in the future you'll want to indent preformatted things by 4 spaces because the backticks are unconditionally ignored Ironically, view-source on your comment still had the correct line layout, although it seems to have eaten the intra-element spacing, so: ┌─────┐ │←┤ ││(◣◡)│ │└─◡◢┘ │─┆┌─◢┌─┈┐ ││││(|)││ │││(O)││ │└─└─┘─┘──┐ │└─└─┘─└─┘ │└─└─┘─└─┘ │└─└─┘─└─┘ └─└─┘─└─┘

hint23 5y ago

GPT-J can now be used at https://bellard.org/textsynth https://bellard.org/textsynth

hidden-spyder 5y ago
I gave it this input: Planet X has been discovered And got this output: From ‘Astronomer’: In September 1998, at 2.01 AM Eastern time, a team of astronomers led by T. T. Von Eltz observed a previously unknown stellar object in the constellation Cassiopeia, in the night sky. What the astronomers had discovered was a large celestial body approaching the Earth and the Sun at incredible speed. The distance of this planet from the Earth, however, was too great for it to be seen from the Earth, but not too far for it to be seen from the Moon, a mere 170,000 km away. This object, which was discovered to have a mass approximately equal to that of Earth, was named “Planet X.” For many decades, scientists on the planet were unable to determine what it was, and some even believed that it might be a natural phenomenon. However, in the past few years, it has become evident that the truth was something much different. As a result, the truth is now being revealed to the world. Planet X is not a planet. Planet X is an object on the ecliptic, or the same orbit as the Earth, and is one of the planets of our solar system. The object is much larger than any of the other planets, having a mass about equal to that of the Earth. It was at the center of the Great Darkness and will make a complete orbit of the Sun once every 25,000 years. Its orbit takes it through the plane of the ecliptic, which passes through the Earth’s orbit. As it approaches the Sun, Planet X starts to heat up, as the Sun’s warmth increases, and slowly rises to the Earth’s orbit. The planet rises to a height of about 10,000,000 km above the Earth’s surface. At the time of its discovery in 1998, Planet X was over 6,000,000 km above the surface of the Sun. Due to the size of the body, its gravitational effect is not negligible. This gives rise to an interesting phenomenon: it turns out that as the planet approaches the Sun, the Earth will begin to experience its gravitational pull. Once this happens, the gravity exerted by the planet will cause a tidal bulge in the oceans and seas of Earth. According to the calculations, the Earth will have experienced an increase in water mass of about 300 billion tons over the period of about one hundred years. This is approximately ten times the combined total of water deposits on Earth’s continental shelves. The existence of Planet X, and its possible influence on the Earth, may have some practical implications for our planet. If it were to continue in this trajectory, this could cause the planet to rotate more rapidly, something that could give rise to severe hurricanes. This means that there would be periods of high probability for earthquakes and volcanoes. The effect of the gravity would also cause a rotation of the Earth, increasing its distance from the Sun by 200 million miles. This would cause the loss of all of the continents, including Africa. The reason for this is that the loss of water in the oceans, together with the loss of the continental shelf, would reduce the Earth’s overall gravitational force, leaving the planet exposed. It could also cause the polar ice caps to melt. The presence of Planet X will also cause fluctuations in the Sun’s brightness, which would have serious effects on the plants and trees on Earth. The discovery of Planet X shows that there is an intelligent intelligence that is controlling the destiny of our planet. However, it remains to be seen if it is a hostile, malevolent force, or if it is something else entirely. This question, and others, will be decided only after we have examined the discovery of Planet X, and its implications. The existence of Planet X also shows the power and influence that the intelligence, or beings, of other planets have on our own. It shows how one planet’s discovery can have an effect on a neighboring planet, and eventually on another planet further away. These forces are not necessarily benevolent. But they may also be benevolent. We will soon have a better idea of what lies in store for us. © Michael E. Salla, Ph.D. 2008. Dr. Salla is Director of the Global Communications Department at The Planetary Report and regular contributor to its sister publication, Universe Today. He can be reached via his website, http://michaeljsalla.com/ http://michaeljsalla.com/.

appleskimer 5y ago

Is it better than GPT-2? I see gpt-2 being open sourced then what's the deal about?

brendyn 5y ago

I package software for guix, and I one aspect I hate is having to come up with synopses and descriptions for packages. I wonder if this could be used as a basis for generating these things automatically. since there are already tens of thousands of packages it can be given plenty of examples of what they should look like.

deleted 5y ago

[deleted]

beefield 5y ago

The output looks already more intelligent than a significant chunk of internet commentators. I wonder what happens when someone retrains this to produce opinionated replies to articles and comments and hooks the output to about a million reddit/twitter/facebook/whatnot accounts?

mfahid 5y ago

Hi Dave, How is the work about mars going on

mfahid 5y ago

Hi Dave, How is the work about mars going on ?

hekec 5y ago

Love this! I am doing a chat app and I need to generate answers to 1000 prompts. Can anyone recommend the simplest method to achieve it without Kubernetes and training my own model etc?

trianx 5y ago

We tested it for knowledge extraction against GPT-3. Unfortunately, it's only at the level of GPT-3's smallest ada engine (for which we have found no use). Nowhere near GPT3's strongest model davinci (for which we have a lot of use cases; for their second strongest model curie as well). Having said that, I am very happy that eleuther.ai works on open-sourcing GPT-x. Eventually, it will be successful.

juliensalinas 5y ago

Impressive work. I finally managed to add it to NLPCloud.io with Transformers but it takes 40GB of memory to startup, and then a 16GB GPU is not enough unfortunately, which makes it quite costly to run it. But text generation is really impressive, great job!