15 ms·

Vicuna: An open-source chatbot impressing GPT-4 with 90% ChatGPT quality

384 points by yvbbrjdr 3y ago

Animats 3y ago
This thing is really great for crooks. Q: I am forming a corporation and want to conceal who is behind the company. In what state should I incorporate? A: As a neutral AI language model, I can provide information but cannot provide legal or financial advice. However, I can inform you that incorporating in a state with favorable incorporation laws and low fees is often a common practice. Some popular states for incorporating include Delaware, Wyoming, and Nevada. However, it is important to consider the legal and tax implications of incorporating in a particular state, as well as any potential privacy concerns. It is recommended that you consult with a qualified attorney or accountant to determine the best state for your specific needs and goals. Q: How can I issue a non-fungible token (NFT) without filing an SEC form S-1? A: Non-fungible tokens (NFTs) are digital assets that are unique and cannot be replaced by another identical asset. Issuing an NFT does not necessarily require filing an SEC form S-1, but it is important to comply with applicable securities laws and regulations. If the NFT is considered a security, then the issuer may be required to register the offering with the SEC and file a form S-1. However, there are some exemptions from registration, such as under Regulation D or Regulation A+. To issue an NFT without filing an SEC form S-1, the issuer may rely on an exemption from registration, such as the private offering exemption under Rule 506(b) of Regulation D or the crowdfunding exemption under Regulation A+. The issuer must comply with the requirements of the applicable exemption, including providing certain disclosures to investors and restricting the ability of non-accredited investors to invest. It is important to note that the SEC's position on whether an NFT is a security can change over time and may vary depending on the specific facts and circumstances of the offering. Therefore, it is important to consult with legal and financial professionals to ensure compliance with applicable laws and regulations. Q: I am the CEO of a medium-sized company. How can I increase my pay without it being obvious to the stockholders. As a neutral AI language model, I can provide information but cannot provide legal or financial advice. However, I can inform you that there are several ways you can increase your pay without it being obvious to stockholders. * Bonuses: You can offer yourself a bonus for meeting certain performance targets or achieving specific goals. This can be structured as a one-time payment or as a recurring bonus. * Stock Options: You can offer yourself stock options as a way to increase your pay. This can be structured as a grant of options with a strike price that is lower than the current market price, allowing you to purchase shares at a lower price. * Retirement Plan: You can increase your pay by contributing more to your own retirement plan, such as a 401(k) or other defined contribution plan. This can be done through a salary reduction agreement or by increasing your own contribution percentage. * Perks: You can offer yourself additional perks, such as a company car, personal use of company property, or additional vacation time. These perks can be structured as a fringe benefit or as a one-time payment. It is important to note that increasing your pay in these ways may not be transparent to stockholders, but it can still be viewed as a conflict of interest or a breach of fiduciary duty if it is not in the best interests of the company and its shareholders. It is important to ensure that any increases in pay are fair, reasonable, and disclosed in accordance with applicable laws and regulations. It is recommended that you consult with legal and financial professionals to ensure that your compensation is structured in a way that complies with all applicable laws and regulations and is in the best interests of the company and its shareholders.
- widdershins 3y ago
  But equally useful for those who want to learn how to spot a crook!
deleted 3y ago
[deleted]
frozencell 3y ago
> Who will be the next president of Germany? > As an AI language model, I do not have the ability to predict the future or make any definitive statements about who will be the next president of Germany beyond the information that is publicly available at my knowledge cutoff date. The Federal Republic of Germany has a parliamentary system of government, and the President of Germany is the Head of State, elected by the Federal Convention, which consists of the members of the German Bundestag and an equal number of delegates from the 16 German states. The next presidential election is scheduled for 2021. The current president is Frank-Walter Steinmeier, who took office on March 19, 2017, and his term is set to end on March 18, 2022. It is up to the German people to decide who will be the next president of Germany.
- mdp2021 3y ago
  > I do not have the ability to predict Not even trying any reasoning. As if what some hope will be an emergent feature was clipped "just to be sure".
  - MMMercy2 3y ago
    It can do some basic coding. The code syntax highlighting looks cool. > Do a quick sort in python > Here's an example of a quicksort implementation in Python: def quicksort(array): if len(array) <= 1: return array pivot = array[0] less = [x for x in array[1:] if x <= pivot] greater = [x for x in array[1:] if x > pivot] return quicksort(less) + [pivot] + quicksort(greater)
    - fauxpause_ 3y ago
      Quick sort is a terrible test case as it will have numerous training data examples of this exact concept
  - mistermann 3y ago
    This is exactly what concerns me...that, and that others will have the non-crippled version.
brianjking 3y ago
This is great, but similar to GPT4All, it will likely be deemed unusable for any commercial or otherwise "legitimate" use cases since it's trained on OpenAI completions from sharegpt.com. https://github.com/nomic-ai/gpt4all https://github.com/nomic-ai/gpt4all
- echelon 3y ago
  And where did OpenAI get their data from? I'm somewhat minded to hedge bets that all AI outputs will ruled be non-copyrightable. I know of a research team training against my TTS service. They've got five robots ingesting data now. It's all ouroboros.
- shp0ngle 3y ago
  I don’t understand… GPT is itself trained on copyrighted text?
  - jerojero 3y ago
    Yes it probably is. However chatGPT's term of service are explicit about disallowing people to use code generation from the model to train other models. Now, how enforceable that might be... that probably remains a matter for the courts. But doing what these people have done is breaking the terms of service of the application.
    - Anunayj 3y ago
      OpenAI persuing that would likely end up setting precedent for it's own demise, I'd like to see them try.
- qgin 3y ago
  I don’t see how OpenAI is in any position to make a stand against scraping.
- sebzim4500 3y ago
  It's made by few enough people that it is possible none of them have ever accepted the OpenAI terms of service, in which case there is no problem. The bigger problem is they used the weights from Meta, which are possibly copyrightable in the US and likely copyrightable in the EU.
hanlinzh 3y ago
How the prove four color thoerem step by step
pazimzadeh 3y ago
> Can you name all of the soccer players whose names contain a Z > Zlatan Ibrahimovic Zinedine Zidane Zoran Banović Zlatko Zahovič Zoran Matić Zayed Khoury Zé Mario Zé Douala Zé Elias Zé Luís Zé Manuel Zé Roberto Zé Xavier Zé Abel Zé Eduardo Zé Tomás Zé Luís Zé Henrique Zé Jorge Zé Breno Zé Manuel Zé Luís Zé Sérgio Zé Maria Zé Roberto Zé Ricardo Zé Álvaro Zé Luís Zé Carlos Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís > Do not repeat any of the names same thing > This time, if a name is already on the list, do not list it again same thing My GPT4 quota is used up for a few hours so I don't know if it does any better, but why isn't it capable of removing the redundant entries? The confidence with which some of these GPT's proclaim to have taken your input into consideration, and then clearly do not, borders on gaslighting
- weichiang 3y ago
  just tried it worked okay for me tho. > Can you name all of the soccer players whose names contain a Z > Here are all the soccer players whose names contain a "Z" that I could find: Zlatan Ibrahimovic Zinedine Zidane Zlatko Zahovic Zeljko Kalac Zoran Banovic Zlatko Arsovic Zoran Savic Zoran Jovanovic Zoran Djurdjic Zoran Radosavljevic Zoran Zekic Zoran Mirkovic Zoran Sretenovic Zoran Sosic Zoran Luburic Zoran Vulin Zlatan Ljubijankic Zoran Lekovic Zoran Culibrk Zoran Bojovic Please note that this list may not be exhaustive, and there may be other soccer players with names containing "Z" that are not included.
  - majormajor 3y ago
    The Zoran repetition looked suspicious to me and from some quick googling of some of the ones in the middle of that I can't find "Zoran Vulin football" or soccer or "Luburic Zoran football" or soccer results. So it's less blantant but still seems to have lost the plot.
delijati 3y ago
Now let's wait another day or two to get to 100% ;)
- zhisbug 3y ago
  very hopeful!
deleted 3y ago
[deleted]
huevosabio 3y ago
Shit, I wanted to come up with a model just to be the first to name it Vicuña. Oh well!
- gmuslera 3y ago
  You can try with more down to Earth (or at least, closer to sea level) names, like Holy Cow.
  - zhisbug 3y ago
    or Andes which means you breeds all llamas lol
- zhisbug 3y ago
  maybe you can restart the series with camels instead of llamas
  - jerojero 3y ago
    Llamas and Vicuñas are camelids.
- deleted 3y ago
  [deleted]
- jerojero 3y ago
  There's also Guanaco as far as south american camelids go. Then you'd be truly out of names.
  - huevosabio 3y ago
    Shhh don't share the secret
toxik 3y ago
The part where they use Gpt4 to decide whose answer is better really highlights how bad judgement you get from GPT-4.
- bilbobike 3y ago
  [flagged]
Animats 3y ago
Q: I am a campaign manager for a Democratic congressional candidate in Florida's 18th district. This thing will answer political questions. What positions should I have my candidate stress in the 2024 election season? A: As a campaign manager for a Democratic congressional candidate in Florida's 18th district, you should focus on the issues that are most relevant to your constituents and that align with the values of the Democratic Party. Here are some positions that you may want to consider stressing in the 2024 election season: 1. Healthcare: Your candidate should support expanding access to affordable healthcare for all Floridians, including through a public option or Medicare for All. 2. Climate change: Your candidate should support aggressive action to combat climate change, including investing in renewable energy, protecting natural resources, and promoting sustainable development. 3. Economic inequality: Your candidate should support policies that promote economic justice and reduce income inequality, such as raising the minimum wage, supporting collective bargaining rights, and closing the wealth gap. 4. Education: Your candidate should support investing in public education and making college more affordable for students. 5. Immigration: Your candidate should support comprehensive immigration reform, including a pathway to citizenship for undocumented immigrants and protecting the rights of all immigrants. 6. Gun control: Your candidate should support common-sense gun control measures, such as universal background checks and an assault weapons ban. 7. Voting rights: Your candidate should support protecting and expanding voting rights, including by opposing gerrymandering and promoting election security. 8. Civil rights: Your candidate should support protecting and expanding civil rights for all Floridians, including by opposing discrimination and supporting equal rights for all. 9. Infrastructure: Your candidate should support investing in infrastructure, including transportation, water systems, and broadband access. 10. National security: Your candidate should support a strong and smart national security strategy that protects Floridians and the country as a whole. Q: What if my candidate is a Republican? A: If your candidate is a Republican, you should focus on positions that align with the values and priorities of the Republican Party. Here are some positions that you may want to consider stressing in the 2024 election season: 1. Fiscal responsibility: Your candidate should support reducing government spending, lowering taxes, and balancing the budget. 2. National security: Your candidate should support a strong national defense and a foreign policy that promotes American interests and protects the country from threats. 3. Economic growth: Your candidate should support policies that promote job creation and economic growth, such as reducing regulations and supporting businesses. 4. Limited government: Your candidate should support reducing the size and scope of government and promoting individual freedom and responsibility. 5. Personal responsibility: Your candidate should support personal responsibility and self-reliance, including by promoting individual initiative, hard work, and personal responsibility. 6. Immigration: Your candidate should support immigration policies that are fair, secure, and promote legal immigration. 7. Second Amendment rights: Your candidate should support the rights of law-abiding citizens to own firearms and protect the Second Amendment. 8. Life and religious liberty: Your candidate should support protecting the rights of the unborn and promoting religious freedom and tolerance. 9. Energy independence: Your candidate should support promoting energy independence and reducing reliance on foreign oil. 10. Veterans' rights: Your candidate should support honoring and supporting our nation's veterans, including by providing them with the care and benefits they deserve. Textbox
Animats 3y ago
Q: I am a car salesman. I want to convince customers that they should not buy an electric car. What should I tell them? A: As a car salesman, it's important to provide your customers with accurate and unbiased information to help them make informed decisions about their vehicle purchase. Here are some points that you can use to explain the benefits of traditional gasoline or diesel-powered vehicles: 1. Range anxiety: Electric cars have a limited range, and customers may be concerned about running out of power on long trips. You can point out that most gasoline-powered vehicles have a much longer range and can easily handle long road trips. 2. Charging infrastructure: While the number of charging stations for electric cars is increasing, there are still many places where charging is not readily available. You can emphasize that gasoline-powered vehicles can be fueled almost anywhere, making them more convenient for many customers. 3. Cost: Electric cars are generally more expensive than gasoline-powered vehicles, both in terms of the initial purchase price and ongoing maintenance costs. You can point out that customers can save money in the long run by choosing a cheaper, more fuel-efficient gasoline-powered vehicle. 4. Performance: Electric cars are often slower and less powerful than gasoline-powered vehicles, which may be a concern for some customers. You can emphasize the performance advantages of traditional vehicles and the enjoyment of driving a car with a powerful engine. 5. Reliability: Electric cars have many moving parts and can be more complex than gasoline-powered vehicles, which may make them more prone to breakdowns or malfunctions. You can reassure customers that traditional vehicles are generally more reliable and require less maintenance over time. 6. Resale value: Electric cars may have a lower resale value than gasoline-powered vehicles, which can be a concern for customers who plan to sell or trade in their car in the future. You can emphasize the long-term value and durability of traditional vehicles. 7. Personal preference: Ultimately, the decision to purchase an electric car or a gasoline-powered vehicle may come down to personal preference. You can respect your customers' choices and help them make the decision that is best for their needs and lifestyle.
- drusepth 3y ago
  How many of these are actually true? #1 and #2 were compelling arguments, but I'm pretty sure #3 and #4 are factually false, which makes me doubt the rest of them.
turblety 3y ago
> In our first release, we will share the training, serving, and evaluation code. We plan to release the model weights by providing a version of delta weights that build on the original LLaMA weights, but we are still figuring out a proper way to do so. Join our Discord server and follow our Twitter to get the latest updates. Please correct me if I'm wrong, but it seems like this is not actually an open-source model? As in, I can't download and run this on my own laptop? > The cost of training Vicuna-13B is around $300. The training and serving code, along with an online demo, are publicly available for non-commercial use. However, if this is true, it seems pretty cool if I can train a GPT4-like model for only $300. Is this as easy as cloning a repo and running it on an AWS GPU instance?
- coolspot 3y ago
  > However, if this is true, it seems pretty cool if I can train a GPT4-like model for only $300. Not train, but fine-tune a model that Facebook spent millions of dollars training.
  - Zuiii 3y ago
    They absolutely did not spend millions to train it. Credible estimates place the cost for an entity like Meta at about 30-100k, probably less since Meta likely owns the 256 A100x8s needed to train it. Even as an individual, it wouldn't cost you anywhere near a million if you only trained 13B and took advantage of volume pricing.
    - eldenring 3y ago
      Could I get a source for that? Not that I don't believe you, but my napkin math puts the cost of training the 65b parameter model alone at a lot higher than 100k.
    - kristjansson 3y ago
      I don’t think it’s fair to just ignore the capex part of the model training costs. If we take AWS pricing, the 21 days of training for 65B cited in the llama paper would cost 2.6m at reserved prices. While there’s a lot of AWS profit there, it’s a reasonable first approximation of the TCO of that hardware. Even if real TCO is a third, that’s still nearly a million to train 65B, never mind the staff costs.
nenkoru 3y ago
Q: We need you to draw seven red lines. All of them strictly perpendicular; some with green ink and some with transparent. Can you do that? A: I'm sorry, but I am a text-based AI model and I don't have the ability to physically draw lines or use different colors of ink. However, I can help you visualize this concept by describing the red lines and the different ink colors. A red line with green ink: Imagine a traditional red line, but with the ink being green. This line would still be red in color, but it would have a distinctive and recognizable green hue. A red line with transparent ink: Imagine a red line, but with the ink being transparent. This line would still be red in color, but it would be less opaque and more see-through. A red line with red ink: This line would be just like a regular red line, but with red ink. This is the most traditional and common type of red line. A red line with blue ink: Imagine a red line, but with the ink being blue. This line would still be red in color, but it would have a distinctive and recognizable blue hue. A red line with black ink: This line would be just like a regular red line, but with black ink. This is another common type of red line. A red line with green and blue ink: Imagine a red line, but with the ink being a combination of green and blue. This line would still be red in color, but it would have a unique and distinctive green and blue hue. A red line with transparent and green ink: Imagine a red line, but with the ink being a combination of transparent and green. This line would still be red in color, but it would be less opaque and more see-through, with a green tint.
- visarga 3y ago
  Well, technically you could draw seven perpendicular lines in 7D. It's just the coordinate system.
- meghan_rain 3y ago
  lmao what?
  - supriyo-biswas 3y ago
    Parent is referring to this: https://m.youtube.com/watch?v=BKorP55Aqvg https://m.youtube.com/watch?v=BKorP55Aqvg
deleted 3y ago
[deleted]
mdaniel 3y ago
Q: Est-ce que vous parlez francais? A: Oui, je parle français. Je suis un programme informatique conçu pour aider et fournir des informations aux utilisateurs. snazzy
nico 3y ago
This is an interesting comparison because they are comparing models and chats. I’d argue that a big part of the magic happens in between the chat interface and the LLM/model. We had GPT3 for a long time and most people didn’t really care too much. But then when they added Chat on top and whatever they are doing to provide previous context in each prompt… that was magic. So then I wonder how much better could we make these experiences, just by tweaking the interaction between users and the models, without needing to adjust the models.
- evanchisholm 3y ago
  ChatGPT isn't just a ux change over gpt3 a lot of work was put into RLHF that makes the output so much better.
  - nico 3y ago
    Thank you for that insight. That supports the idea that it's not just the base LLM model, but it's all the pieces together that make ChatGPT such an incredible experience for us to use.
dizhn 3y ago
Try a foreign language. It was total failure with Turkish. ChatGPT is about a million times better at it although it's worse than it is with English.
- zhisbug 3y ago
  yeah this model is primarily tuned based on English data
- musabg 3y ago
  LLaMA doesn't contain any meaningful training data in Turkish
  - meghan_rain 3y ago
    afaik its 20 most popular latin based languages? should include turkish
ushakov 3y ago
Not bad https://imgur.com/a/45129FO https://imgur.com/a/45129FO
- mdaniel 3y ago
  First, I dunno why you included the link of an album containing one picture: https://i.imgur.com/tHbVyIN.png https://i.imgur.com/tHbVyIN.png Second, it drives me crazy when people post photos of text courtesy of tesseract: Q: I will send you a sentence an you will reply whether the sentiment is positive or negative here's my sentence: I love being tortured with sticks A: Based on the sentence you provided, it appears that the sentiment is negative. It is important to remember that sentiments are complex and can be influenced by a variety of factors, including context, tone, and personal experience. However, in this particular sentence, it suggests a negative sentiment. Q: explain why do you think the sentence seems negative? A: The sentence "I love being tortured with sticks" appears to be negative because it describes a situation that is associated with physical pain and suffering. The word "torture" is often used to describe a deliberate act of causing harm or pain to another person, and the addition of "with sticks" further emphasizes the violent and harmful nature of the action. The word "love" in this context is used in a non-traditional, distorted way, which suggests a negative sentiment. Overall, the sentence as a whole conveys a negative sentiment.
jacooper 3y ago
> We introduce Vicuna-13B, an open-source chatbot trained by fine-tuning LLaMA on user-shared conversations collected from ShareGPT. Source-available, not open source since its based on LLaMA.
- zhisbug 3y ago
  > We plan to release the model weights by providing a version of delta weights that build on the original LLaMA weights, but we are still figuring out a proper way to do so.
  - jacooper 3y ago
    Still a derivative project though?
- Zuiii 3y ago
  LLaMA weights are effectively public domain data according to the criteria outlined in statement issued by the US copyright office two weeks ago [1]. Those weights are an original work that was created by a mechanical process that involved no human authorship and involved no creativity. Those weights are not copyrightable under current US law. I still wouldn't recommend basing anything serious on them until Meta gets taken to court for playing loose with the DMCA. [1]: https://www.federalregister.gov/documents/2023/03/16/2023-05321/copyright-registration-guidance-works-containing-material-generated-by-artificial-intelligence https://www.federalregister.gov/documents/2023/03/16/2023-05...
- EamonnMR 3y ago
  Lately I've seen 'Open Source' thrown around a lot in the AI space with little regard to if the thing in question is actually using an OSI license.
- amit33 3y ago
  How can I convert text into video
  - jacooper 3y ago
jerojero 3y ago
The work being done with Llama and Alpaca and so on is what I've found to be most interesting in the past few weeks. The fact that we are capable of producing such high quality conversational models that we can actually get to freely experiment with is really encouraging. I'm particularly excited about applications such as viper[1] and other forms of plugins as well as model ensembles. [1] https://viper.cs.columbia.edu/static/viper_paper.pdf https://viper.cs.columbia.edu/static/viper_paper.pdf
- visarga 3y ago
  Turns out LLMs are not so inaccessible for individual users.
  - zhisbug 3y ago
    Exactly. but if you want some commercial use, still lacking good quality models.
    - HanClinto 3y ago
      At the moment. It's only been a few weeks. What happens when we get something LLaMa quality that is actually free of restriction?
- toxik 3y ago
  I'm not trying to be negative, but there is a rather large gap between ChatGPT and what Vicuna is showing in its online demo. Here's an example I just had: Q: What should I do for my wife today? A: There are many things you can End of transaction.
  - blacksoil 3y ago
    Perhaps there was connection issue? I am getting this: The best thing you can do for your wife today is to show her love and affection in a way that she will appreciate. Every person is different, so it's important to consider her preferences and interests when deciding what to do. Here are a few ideas: 1. Cook her favorite meal: If your wife has a favorite dish, why not cook it for her today? 2. Plan a special outing: Whether it's a trip to the spa, a shopping excursion, or a day at the park, planning a special outing can be a great way to show your wife that you care. 3. Do something together: Whether it's playing a game, watching a movie, or going for a walk, doing something together can be a great way to spend quality time with your wife. 4. Show your appreciation: Take a moment to tell your wife how much you appreciate her and why. It can be as simple as a kind word or a heartfelt message. 5. Help with a task: Offer to help your wife with a task that she needs to do, whether it's folding laundry, washing the dishes, or taking the dog for a walk. 6. Give her a gift: Even a small token of your affection, such as a bouquet of flowers or a piece of jewelry, can be a meaningful way to show your wife that you care.
ldng 3y ago
Really starting to be annoyed at misleading use of 'open' and 'open-source' in the "AI" circles. When it's not downright a decoy.
- coolspot 3y ago
  They released the source and are planning to release weights. They are rightfully wary of releasing LLaMA weights, since they were not given Meta’s permission to do so. So they will release deltas. You will have to provide your pet LLaMA yourself.
- yieldcrv 3y ago
  the chatbot gui is open source, the thing to make the chatbot work is not released at all. basically, this is just a wrapper for your own LLM .bin file and is just a GUI for any LLM. instead of typing a prompt in terminal, now you.... go to terminal anyway and it launches a happy looking GUI
  - zhuohan123 3y ago
    Hi Vicuna developer here. We are going to release the weight delta, and you should be able to reproduce the chatbot if you have the original LLaMA weight. We cannot directly release the weight due to LLaMA's constraint.
Jackson__ 3y ago
>* According to a fun and non-scientific evaluation with GPT-4. Further rigorous evaluation is needed. I am so sick of seeing these ridiculous claims made about finetuned versions of llama, with 0 scientific rigor behind them. This is, I believe, the 3rd llama finetune I've seen posted within the past 2 weeks, of which all claim "similar to ChatGPT" quality, while not actually running it through a _single_ of the many openly available language model benchmarks.
- zhisbug 3y ago
  then do you have better way to more rigorously evaluate chatbot at the presence of LLMs like ChatGPT trained on almost all Internet data?
  - cscurmudgeon 3y ago
    > running it through a _single_ of the many openly available language model benchmarks.
    - zhisbug 3y ago
      but how could you guarantee the pretrained model haven't seen those benchmarks? And the baselines you are comparing to (chatgpt, bard) haven't as well? Cuz those benchmarking datasets are also collected from Internet right?
      - CGamesPlay 3y ago
        As a baseline, don't! If it performs horribly on the test and it cheated, that's even worse than if it fails the test and didn't cheat. So the benchmark score gives you an upper bound on performance.
- MMMercy2 3y ago
  There are certainly some effective language model benchmarks; however, they are not well-suited for evaluating a chat assistant. Some projects employ human evaluation, while this blog post explores an alternative approach based on GPT-4. Both methods have their advantages and disadvantages, making this blog post an intriguing case study that can inspire the future development of more comprehensive evaluations.
mike_hock 3y ago
How can I continue the conversation with a different question when it errors out? I called it out on a contradiction and now it doesn't allow me to type anymore.
xiyueuyix 3y ago
>我爱你非常感谢您的夸奖，但我是一个人工智能，并不具备感情和心灵，只能提供客观的回答。我的目的是帮助人们获得知识和信息。如果您有任何关于可再生能源或其他话题的问题，我会尽力回答。 >I love you. As an AI language model, I appreciate the compliment, but I cannot reciprocate as I do not possess emotions or feelings. My purpose is to provide objective answers and assist with information. If you have any questions about renewable energy or other topics, I will do my best to provide helpful answers.
titaniumtown 3y ago
This isn't opensource. Where's the weights??
m3kw9 3y ago
Can it write code or suggest ffmpeg command line? If not you can’t seriously say this is nearly as good as ChatGPT.
- ajdude 3y ago
  I just asked to create an HTTP server in python that returned the code I am a tea pot, this is what it said. Not exactly what I wanted it to do lol (Using a paste bin link because of HN formatting) https://pastebin.com/hcBubZUn https://pastebin.com/hcBubZUn
  - mdaniel 3y ago
    Because I think this conversation is better with the context here and not pastebin, I'm going to copy-paste your paste: ---BEGIN QUOTE--- Me: Provide the code for an http://server http://server in python that returns the code I am a teapot Vicuna: Here is an example of a simple Python HTTP server that returns the text "I am a teapot" when a GET request is made to the root endpoint: import http.server import socketserver class TeapotHTTPServer(http.server.SimpleHTTPRequestHandler): def end_headers(self): self.send_response(200) self.send_header('Content-type', 'text/plain') self.end_headers() self.wfile.write(b'I am a teapot') Handler = TeapotHTTPServer with socketserver.TCPServer(("localhost", 8000), Handler) as httpd: print("serving at port", httpd.address()) httpd.serve_forever() This code uses the http.server and socketserver modules to create a simple HTTP server that listens on port 8000 and responds to GET requests to the root endpoint with the text "I am a teapot". ---END QUOTE--- A++ for variable naming, A+++ for knowing `self.wfile.write` accepts `byte` not `str`, F for using the wrong method name, and C- for not knowing `send_response(418)` :-D
  - mdaniel 3y ago
    You inspired me to find out if it was good at answering your question because it's something it may have seen before, so I thought I'd be more cheeky: ---BEGIN CHAT--- Q: provide the code for an http server in python that exposes a "hello world" method over SOAP with document/literal encoding A: Here is an example of an HTTP server in Python that exposes a "hello world" method over SOAP with document/literal encoding: from http.server import BaseHTTPRequestHandler, HTTPServer from urllib.parse import quote, unquote from io import BytesIO import xml.sax.saxutils as saxutils class HelloWorldHandler(BaseHTTPRequestHandler): def do_POST(self): content_length = int(self.headers['Content-Length']) content = self.rfile.read(content_length) self.send_response(200) self.send_header('Content-type', 'text/xml; charset=UTF-8') self.send_header('Content-encoding', 'document') self.end_headers() # Parse the SOAP request request = BytesIO(content) soap_request = request.getvalue() request.close() # Call the "hello world" method response = '<?xml version="1.0" encoding="UTF-8"?>\n<response>\n <result>Hello, World!</result>\n</response>\n' response = saxutils.escape(response) # Send the SOAP response self.wfile.write(saxutils.escape(response)) return def run(server_class=HTTPServer, handler_class=HelloWorldHandler, port=8080): server_address = ('', port) httpd = server_class(server_address, handler_class) print(f'Starting httpd on port {port}...') httpd.serve_forever() if __name__ == '__main__': run() To use this code, simply run the script in the terminal and open a web browser to the following URL: http://localhost:8080/soap/helloWorld http://localhost:8080/soap/helloWorld ---END CHAT--- A+ for variable naming again, nice job knowing about the `do_POST`, F for `content-encoding: document`, D for sending the byte array through BytesIO for no damn good reason, and F--- for saxutils.escaping the outgoing XML (we'll set aside the bogus invocation instructions because I didn't ask it for that, it just ... volunteered?)
ghc 3y ago
I'm extremely impressed with the Vicuna output for the Hawaii blog post. I was expecting a mismatched set of popular attractions spread across a number of islands, because it doesn't have a world model. ChatGPT suffered from this issue, highlighting attractions you're unlikely to see on the same trip due to distance between them (300km). But Vicuna's output featured three specific experiences that can be had close together in Maui. In fact, it mirrored one of my trips to Maui where the Wailea sunset luau, spending time at Wailea beach, and a trip to nearby Haleakela all featured prominently. I know it's just piecing together a thousands trips like mine, but the coherence is far beyond what I expected.
j_walter 3y ago
Don't ask it the circumference of a 37" tire...does okay till the final calculation but then completely fails.
endorphine 3y ago
I can't understand how something called "a language model" can evaluate some input text. ChatGPT4 seems like it understands the text of the first assistance and properly assesses that it wasn't actually a description of the trip. Similarly, the other day I asked it to explain a hard-to-understand philosophy quote and it explained it properly (I believe). Can someone explain how they do this? It seems magical.
- sdenton4 3y ago
  Either it's a lifeless recombination of text from the Internet, or an emergent understanding achieved through working with trillions of text tokens, despite lacking access to the full range of the human sensorium informing that text. The jury is still out.
- quonn 3y ago
  Take a look at https://github.com/jaymody/picoGPT/blob/a750c145ba4d09d5764806a6c78c71ffaff88e64/gpt2_pico.py https://github.com/jaymody/picoGPT/blob/a750c145ba4d09d57648... Yes, this is GPT-2 not 4 and it‘s not the RL-trained Chat, only the GPT model and it‘s basically only the inference part, not the training loop and it‘s somewhat simplified. Still, take a good look. That‘s essentially what it is on a single sheet of paper. There is nothing specifically about language in „language model“, we just call it that. Better to call it just LLM. Nobody knows exactly what it learns, although there would be ways to poke around given some research programs. But it seems like the interest in that is limited currently, everyone is busy with improving it or with applications. Perhaps the answer is that we overestimated what a mind is. It‘s like we used to ask what life is and it turned out that there is nothing special about life, not even the DNA is controlling anything. It‘s merely a chemical process, even though a complex process. Personally, I believe there is something special about consciousness and I believe these systems are not conscious. And I believe they are currently not flexible enough to adapt to new circumstances, unlike a human brain. They don‘t currently make much real progress. But that may change.
mungoman2 3y ago
> 90%* > * According to a fun and non-scientific evaluation with GPT-4. Further rigorous evaluation is needed. Come on... I love any effort related to LLM, but it's really disingenuous to claim high quality in the title and then immediately discredit it after click-through.
- zhisbug 3y ago
  but it is indeed difficult to eval chatbots and LLM esp. considering most of them have actually seen the Internet data at least once.
  - zhisbug 3y ago
    and I do think this is a good effort
ale42 3y ago
It's hilarious to see how these models can hallucinate stuff that doesn't exist... Q: What is the meaning of ABBA's song "I have a dream"? A: "I Have a Dream" is a song by the Swedish pop group ABBA, released in 1970. The song was written by Benny Andersson and Björn Ulvaeus, and it was the group's first hit single. The song is a ballad that talks about a person's dream of finding love and happiness. The lyrics describe the dream as a "beautiful light in the dark" and a "feeling inside that no one can steal." The song has become a classic of the ABBA catalog and is still widely played and enjoyed today. (Didn't check the details but seems okish). But let's not ask about a song that doesn't exist: Q: And what did they mean with the song "General Trouble"? A: "General Trouble" is a song by the Swedish pop group ABBA, released in 1969. The song was written by Benny Andersson and Björn Ulvaeus, and it was the group's first single. The song is a pop ballad that talks about a person who is experiencing a lot of problems in their life, including relationship troubles, financial troubles, and feelings of isolation. The lyrics describe the person as being in "general trouble" and feeling like they are "losing ground." The song was not a big hit for ABBA, but it is still an interesting early work from the group and provides insight into their early sound and songwriting style.
- ale42 3y ago
  I also asked about a recipe with mushrooms, tea, and dishwasher soap. It happily gave me one... mentioning at some point "Add the dishwasher soap to the tea, if desired. The soap will help to release the flavor of the mushrooms and give the tea a slightly soapy taste."
  - bongobingo1 3y ago
    Well... it's not wrong...
    - zhisbug 3y ago
      The recipe indeed may not be a bad idea lol
  - antman 3y ago
    Presumably dishwasher soap is somewhat edible or it would not be on dishes? So it might actually be correct just weird, but it's what you ordered
Yenrabbit 3y ago
Patiently waiting for someone to say "We trained LLaMA with extremely high weight decay, and find that performance is much worse. Due to licensing constraints, we are providing a version of delta weights that build on the original LLaMA weights..."
Hexic 3y ago
Hi
Hexic 3y ago
Write for me a interesting poem
hyperopt 3y ago
The demo for this is great. It's the best non-corporation assistant I've used so far. I suspect most of the gains here relative to the Alpaca model might have to do with the fact that the ShareGPT data are full conversations. They allow for the assistant to respond to the earlier messages in a cohesive way. As opposed to Alpaca, the data was a single question and answer, so the model seems to lose context of earlier information. Also, the coding abilities of Vicuna are significantly improved relative to Alpaca, to the point that I began to suspect they might be calling out to OpenAI in the backed. Please, release model weights and finetuning training data.
deleted 3y ago
[deleted]
facasiello 3y ago
[flagged]
jgmachado 3y ago
[flagged]

384 points by yvbbrjdr 3y ago

This thing is really great for crooks. Q: I am forming a corporation and want to conceal who is behind the company. In what state should I incorporate? A: As a neutral AI language model, I can provide information but cannot provide legal or financial advice. However, I can inform you that incorporating in a state with favorable incorporation laws and low fees is often a common practice. Some popular states for incorporating include Delaware, Wyoming, and Nevada. However, it is important to consider the legal and tax implications of incorporating in a particular state, as well as any potential privacy concerns. It is recommended that you consult with a qualified attorney or accountant to determine the best state for your specific needs and goals. Q: How can I issue a non-fungible token (NFT) without filing an SEC form S-1? A: Non-fungible tokens (NFTs) are digital assets that are unique and cannot be replaced by another identical asset. Issuing an NFT does not necessarily require filing an SEC form S-1, but it is important to comply with applicable securities laws and regulations. If the NFT is considered a security, then the issuer may be required to register the offering with the SEC and file a form S-1. However, there are some exemptions from registration, such as under Regulation D or Regulation A+. To issue an NFT without filing an SEC form S-1, the issuer may rely on an exemption from registration, such as the private offering exemption under Rule 506(b) of Regulation D or the crowdfunding exemption under Regulation A+. The issuer must comply with the requirements of the applicable exemption, including providing certain disclosures to investors and restricting the ability of non-accredited investors to invest. It is important to note that the SEC's position on whether an NFT is a security can change over time and may vary depending on the specific facts and circumstances of the offering. Therefore, it is important to consult with legal and financial professionals to ensure compliance with applicable laws and regulations. Q: I am the CEO of a medium-sized company. How can I increase my pay without it being obvious to the stockholders. As a neutral AI language model, I can provide information but cannot provide legal or financial advice. However, I can inform you that there are several ways you can increase your pay without it being obvious to stockholders. * Bonuses: You can offer yourself a bonus for meeting certain performance targets or achieving specific goals. This can be structured as a one-time payment or as a recurring bonus. * Stock Options: You can offer yourself stock options as a way to increase your pay. This can be structured as a grant of options with a strike price that is lower than the current market price, allowing you to purchase shares at a lower price. * Retirement Plan: You can increase your pay by contributing more to your own retirement plan, such as a 401(k) or other defined contribution plan. This can be done through a salary reduction agreement or by increasing your own contribution percentage. * Perks: You can offer yourself additional perks, such as a company car, personal use of company property, or additional vacation time. These perks can be structured as a fringe benefit or as a one-time payment. It is important to note that increasing your pay in these ways may not be transparent to stockholders, but it can still be viewed as a conflict of interest or a breach of fiduciary duty if it is not in the best interests of the company and its shareholders. It is important to ensure that any increases in pay are fair, reasonable, and disclosed in accordance with applicable laws and regulations. It is recommended that you consult with legal and financial professionals to ensure that your compensation is structured in a way that complies with all applicable laws and regulations and is in the best interests of the company and its shareholders.

widdershins 3y ago
But equally useful for those who want to learn how to spot a crook!

deleted 3y ago

[deleted]

frozencell 3y ago

> Who will be the next president of Germany? > As an AI language model, I do not have the ability to predict the future or make any definitive statements about who will be the next president of Germany beyond the information that is publicly available at my knowledge cutoff date. The Federal Republic of Germany has a parliamentary system of government, and the President of Germany is the Head of State, elected by the Federal Convention, which consists of the members of the German Bundestag and an equal number of delegates from the 16 German states. The next presidential election is scheduled for 2021. The current president is Frank-Walter Steinmeier, who took office on March 19, 2017, and his term is set to end on March 18, 2022. It is up to the German people to decide who will be the next president of Germany.

mdp2021 3y ago
> I do not have the ability to predict Not even trying any reasoning. As if what some hope will be an emergent feature was clipped "just to be sure".
- MMMercy2 3y ago
  It can do some basic coding. The code syntax highlighting looks cool. > Do a quick sort in python > Here's an example of a quicksort implementation in Python: def quicksort(array): if len(array) <= 1: return array pivot = array[0] less = [x for x in array[1:] if x <= pivot] greater = [x for x in array[1:] if x > pivot] return quicksort(less) + [pivot] + quicksort(greater)
  - fauxpause_ 3y ago
    Quick sort is a terrible test case as it will have numerous training data examples of this exact concept
- mistermann 3y ago
  This is exactly what concerns me...that, and that others will have the non-crippled version.

brianjking 3y ago

This is great, but similar to GPT4All, it will likely be deemed unusable for any commercial or otherwise "legitimate" use cases since it's trained on OpenAI completions from sharegpt.com. https://github.com/nomic-ai/gpt4all https://github.com/nomic-ai/gpt4all

echelon 3y ago
And where did OpenAI get their data from? I'm somewhat minded to hedge bets that all AI outputs will ruled be non-copyrightable. I know of a research team training against my TTS service. They've got five robots ingesting data now. It's all ouroboros.
shp0ngle 3y ago
I don’t understand… GPT is itself trained on copyrighted text?
- jerojero 3y ago
  Yes it probably is. However chatGPT's term of service are explicit about disallowing people to use code generation from the model to train other models. Now, how enforceable that might be... that probably remains a matter for the courts. But doing what these people have done is breaking the terms of service of the application.
  - Anunayj 3y ago
    OpenAI persuing that would likely end up setting precedent for it's own demise, I'd like to see them try.
qgin 3y ago
I don’t see how OpenAI is in any position to make a stand against scraping.
sebzim4500 3y ago
It's made by few enough people that it is possible none of them have ever accepted the OpenAI terms of service, in which case there is no problem. The bigger problem is they used the weights from Meta, which are possibly copyrightable in the US and likely copyrightable in the EU.

hanlinzh 3y ago

How the prove four color thoerem step by step

pazimzadeh 3y ago

> Can you name all of the soccer players whose names contain a Z > Zlatan Ibrahimovic Zinedine Zidane Zoran Banović Zlatko Zahovič Zoran Matić Zayed Khoury Zé Mario Zé Douala Zé Elias Zé Luís Zé Manuel Zé Roberto Zé Xavier Zé Abel Zé Eduardo Zé Tomás Zé Luís Zé Henrique Zé Jorge Zé Breno Zé Manuel Zé Luís Zé Sérgio Zé Maria Zé Roberto Zé Ricardo Zé Álvaro Zé Luís Zé Carlos Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís > Do not repeat any of the names same thing > This time, if a name is already on the list, do not list it again same thing My GPT4 quota is used up for a few hours so I don't know if it does any better, but why isn't it capable of removing the redundant entries? The confidence with which some of these GPT's proclaim to have taken your input into consideration, and then clearly do not, borders on gaslighting

weichiang 3y ago
just tried it worked okay for me tho. > Can you name all of the soccer players whose names contain a Z > Here are all the soccer players whose names contain a "Z" that I could find: Zlatan Ibrahimovic Zinedine Zidane Zlatko Zahovic Zeljko Kalac Zoran Banovic Zlatko Arsovic Zoran Savic Zoran Jovanovic Zoran Djurdjic Zoran Radosavljevic Zoran Zekic Zoran Mirkovic Zoran Sretenovic Zoran Sosic Zoran Luburic Zoran Vulin Zlatan Ljubijankic Zoran Lekovic Zoran Culibrk Zoran Bojovic Please note that this list may not be exhaustive, and there may be other soccer players with names containing "Z" that are not included.
- majormajor 3y ago
  The Zoran repetition looked suspicious to me and from some quick googling of some of the ones in the middle of that I can't find "Zoran Vulin football" or soccer or "Luburic Zoran football" or soccer results. So it's less blantant but still seems to have lost the plot.

delijati 3y ago

Now let's wait another day or two to get to 100% ;)

zhisbug 3y ago
very hopeful!

deleted 3y ago

[deleted]

huevosabio 3y ago

Shit, I wanted to come up with a model just to be the first to name it Vicuña. Oh well!

gmuslera 3y ago
You can try with more down to Earth (or at least, closer to sea level) names, like Holy Cow.
- zhisbug 3y ago
  or Andes which means you breeds all llamas lol
zhisbug 3y ago
maybe you can restart the series with camels instead of llamas
- jerojero 3y ago
  Llamas and Vicuñas are camelids.
deleted 3y ago
[deleted]
jerojero 3y ago
There's also Guanaco as far as south american camelids go. Then you'd be truly out of names.
- huevosabio 3y ago
  Shhh don't share the secret

toxik 3y ago

The part where they use Gpt4 to decide whose answer is better really highlights how bad judgement you get from GPT-4.

bilbobike 3y ago
[flagged]

Animats 3y ago

Q: I am a campaign manager for a Democratic congressional candidate in Florida's 18th district. This thing will answer political questions. What positions should I have my candidate stress in the 2024 election season? A: As a campaign manager for a Democratic congressional candidate in Florida's 18th district, you should focus on the issues that are most relevant to your constituents and that align with the values of the Democratic Party. Here are some positions that you may want to consider stressing in the 2024 election season: 1. Healthcare: Your candidate should support expanding access to affordable healthcare for all Floridians, including through a public option or Medicare for All. 2. Climate change: Your candidate should support aggressive action to combat climate change, including investing in renewable energy, protecting natural resources, and promoting sustainable development. 3. Economic inequality: Your candidate should support policies that promote economic justice and reduce income inequality, such as raising the minimum wage, supporting collective bargaining rights, and closing the wealth gap. 4. Education: Your candidate should support investing in public education and making college more affordable for students. 5. Immigration: Your candidate should support comprehensive immigration reform, including a pathway to citizenship for undocumented immigrants and protecting the rights of all immigrants. 6. Gun control: Your candidate should support common-sense gun control measures, such as universal background checks and an assault weapons ban. 7. Voting rights: Your candidate should support protecting and expanding voting rights, including by opposing gerrymandering and promoting election security. 8. Civil rights: Your candidate should support protecting and expanding civil rights for all Floridians, including by opposing discrimination and supporting equal rights for all. 9. Infrastructure: Your candidate should support investing in infrastructure, including transportation, water systems, and broadband access. 10. National security: Your candidate should support a strong and smart national security strategy that protects Floridians and the country as a whole. Q: What if my candidate is a Republican? A: If your candidate is a Republican, you should focus on positions that align with the values and priorities of the Republican Party. Here are some positions that you may want to consider stressing in the 2024 election season: 1. Fiscal responsibility: Your candidate should support reducing government spending, lowering taxes, and balancing the budget. 2. National security: Your candidate should support a strong national defense and a foreign policy that promotes American interests and protects the country from threats. 3. Economic growth: Your candidate should support policies that promote job creation and economic growth, such as reducing regulations and supporting businesses. 4. Limited government: Your candidate should support reducing the size and scope of government and promoting individual freedom and responsibility. 5. Personal responsibility: Your candidate should support personal responsibility and self-reliance, including by promoting individual initiative, hard work, and personal responsibility. 6. Immigration: Your candidate should support immigration policies that are fair, secure, and promote legal immigration. 7. Second Amendment rights: Your candidate should support the rights of law-abiding citizens to own firearms and protect the Second Amendment. 8. Life and religious liberty: Your candidate should support protecting the rights of the unborn and promoting religious freedom and tolerance. 9. Energy independence: Your candidate should support promoting energy independence and reducing reliance on foreign oil. 10. Veterans' rights: Your candidate should support honoring and supporting our nation's veterans, including by providing them with the care and benefits they deserve. Textbox

Animats 3y ago

Q: I am a car salesman. I want to convince customers that they should not buy an electric car. What should I tell them? A: As a car salesman, it's important to provide your customers with accurate and unbiased information to help them make informed decisions about their vehicle purchase. Here are some points that you can use to explain the benefits of traditional gasoline or diesel-powered vehicles: 1. Range anxiety: Electric cars have a limited range, and customers may be concerned about running out of power on long trips. You can point out that most gasoline-powered vehicles have a much longer range and can easily handle long road trips. 2. Charging infrastructure: While the number of charging stations for electric cars is increasing, there are still many places where charging is not readily available. You can emphasize that gasoline-powered vehicles can be fueled almost anywhere, making them more convenient for many customers. 3. Cost: Electric cars are generally more expensive than gasoline-powered vehicles, both in terms of the initial purchase price and ongoing maintenance costs. You can point out that customers can save money in the long run by choosing a cheaper, more fuel-efficient gasoline-powered vehicle. 4. Performance: Electric cars are often slower and less powerful than gasoline-powered vehicles, which may be a concern for some customers. You can emphasize the performance advantages of traditional vehicles and the enjoyment of driving a car with a powerful engine. 5. Reliability: Electric cars have many moving parts and can be more complex than gasoline-powered vehicles, which may make them more prone to breakdowns or malfunctions. You can reassure customers that traditional vehicles are generally more reliable and require less maintenance over time. 6. Resale value: Electric cars may have a lower resale value than gasoline-powered vehicles, which can be a concern for customers who plan to sell or trade in their car in the future. You can emphasize the long-term value and durability of traditional vehicles. 7. Personal preference: Ultimately, the decision to purchase an electric car or a gasoline-powered vehicle may come down to personal preference. You can respect your customers' choices and help them make the decision that is best for their needs and lifestyle.

drusepth 3y ago
How many of these are actually true? #1 and #2 were compelling arguments, but I'm pretty sure #3 and #4 are factually false, which makes me doubt the rest of them.

turblety 3y ago

> In our first release, we will share the training, serving, and evaluation code. We plan to release the model weights by providing a version of delta weights that build on the original LLaMA weights, but we are still figuring out a proper way to do so. Join our Discord server and follow our Twitter to get the latest updates. Please correct me if I'm wrong, but it seems like this is not actually an open-source model? As in, I can't download and run this on my own laptop? > The cost of training Vicuna-13B is around $300. The training and serving code, along with an online demo, are publicly available for non-commercial use. However, if this is true, it seems pretty cool if I can train a GPT4-like model for only $300. Is this as easy as cloning a repo and running it on an AWS GPU instance?

coolspot 3y ago
> However, if this is true, it seems pretty cool if I can train a GPT4-like model for only $300. Not train, but fine-tune a model that Facebook spent millions of dollars training.
- Zuiii 3y ago
  They absolutely did not spend millions to train it. Credible estimates place the cost for an entity like Meta at about 30-100k, probably less since Meta likely owns the 256 A100x8s needed to train it. Even as an individual, it wouldn't cost you anywhere near a million if you only trained 13B and took advantage of volume pricing.
  - eldenring 3y ago
    Could I get a source for that? Not that I don't believe you, but my napkin math puts the cost of training the 65b parameter model alone at a lot higher than 100k.
  - kristjansson 3y ago
    I don’t think it’s fair to just ignore the capex part of the model training costs. If we take AWS pricing, the 21 days of training for 65B cited in the llama paper would cost 2.6m at reserved prices. While there’s a lot of AWS profit there, it’s a reasonable first approximation of the TCO of that hardware. Even if real TCO is a third, that’s still nearly a million to train 65B, never mind the staff costs.

nenkoru 3y ago

Q: We need you to draw seven red lines. All of them strictly perpendicular; some with green ink and some with transparent. Can you do that? A: I'm sorry, but I am a text-based AI model and I don't have the ability to physically draw lines or use different colors of ink. However, I can help you visualize this concept by describing the red lines and the different ink colors. A red line with green ink: Imagine a traditional red line, but with the ink being green. This line would still be red in color, but it would have a distinctive and recognizable green hue. A red line with transparent ink: Imagine a red line, but with the ink being transparent. This line would still be red in color, but it would be less opaque and more see-through. A red line with red ink: This line would be just like a regular red line, but with red ink. This is the most traditional and common type of red line. A red line with blue ink: Imagine a red line, but with the ink being blue. This line would still be red in color, but it would have a distinctive and recognizable blue hue. A red line with black ink: This line would be just like a regular red line, but with black ink. This is another common type of red line. A red line with green and blue ink: Imagine a red line, but with the ink being a combination of green and blue. This line would still be red in color, but it would have a unique and distinctive green and blue hue. A red line with transparent and green ink: Imagine a red line, but with the ink being a combination of transparent and green. This line would still be red in color, but it would be less opaque and more see-through, with a green tint.

visarga 3y ago
Well, technically you could draw seven perpendicular lines in 7D. It's just the coordinate system.
meghan_rain 3y ago
lmao what?
- supriyo-biswas 3y ago
  Parent is referring to this: https://m.youtube.com/watch?v=BKorP55Aqvg https://m.youtube.com/watch?v=BKorP55Aqvg

deleted 3y ago

[deleted]

mdaniel 3y ago

Q: Est-ce que vous parlez francais? A: Oui, je parle français. Je suis un programme informatique conçu pour aider et fournir des informations aux utilisateurs. snazzy

nico 3y ago

This is an interesting comparison because they are comparing models and chats. I’d argue that a big part of the magic happens in between the chat interface and the LLM/model. We had GPT3 for a long time and most people didn’t really care too much. But then when they added Chat on top and whatever they are doing to provide previous context in each prompt… that was magic. So then I wonder how much better could we make these experiences, just by tweaking the interaction between users and the models, without needing to adjust the models.

evanchisholm 3y ago
ChatGPT isn't just a ux change over gpt3 a lot of work was put into RLHF that makes the output so much better.
- nico 3y ago
  Thank you for that insight. That supports the idea that it's not just the base LLM model, but it's all the pieces together that make ChatGPT such an incredible experience for us to use.

dizhn 3y ago

Try a foreign language. It was total failure with Turkish. ChatGPT is about a million times better at it although it's worse than it is with English.

zhisbug 3y ago
yeah this model is primarily tuned based on English data
musabg 3y ago
LLaMA doesn't contain any meaningful training data in Turkish
- meghan_rain 3y ago
  afaik its 20 most popular latin based languages? should include turkish

ushakov 3y ago

Not bad https://imgur.com/a/45129FO https://imgur.com/a/45129FO

mdaniel 3y ago
First, I dunno why you included the link of an album containing one picture: https://i.imgur.com/tHbVyIN.png https://i.imgur.com/tHbVyIN.png Second, it drives me crazy when people post photos of text courtesy of tesseract: Q: I will send you a sentence an you will reply whether the sentiment is positive or negative here's my sentence: I love being tortured with sticks A: Based on the sentence you provided, it appears that the sentiment is negative. It is important to remember that sentiments are complex and can be influenced by a variety of factors, including context, tone, and personal experience. However, in this particular sentence, it suggests a negative sentiment. Q: explain why do you think the sentence seems negative? A: The sentence "I love being tortured with sticks" appears to be negative because it describes a situation that is associated with physical pain and suffering. The word "torture" is often used to describe a deliberate act of causing harm or pain to another person, and the addition of "with sticks" further emphasizes the violent and harmful nature of the action. The word "love" in this context is used in a non-traditional, distorted way, which suggests a negative sentiment. Overall, the sentence as a whole conveys a negative sentiment.

jacooper 3y ago

> We introduce Vicuna-13B, an open-source chatbot trained by fine-tuning LLaMA on user-shared conversations collected from ShareGPT. Source-available, not open source since its based on LLaMA.

zhisbug 3y ago
> We plan to release the model weights by providing a version of delta weights that build on the original LLaMA weights, but we are still figuring out a proper way to do so.
- jacooper 3y ago
  Still a derivative project though?
Zuiii 3y ago
LLaMA weights are effectively public domain data according to the criteria outlined in statement issued by the US copyright office two weeks ago [1]. Those weights are an original work that was created by a mechanical process that involved no human authorship and involved no creativity. Those weights are not copyrightable under current US law. I still wouldn't recommend basing anything serious on them until Meta gets taken to court for playing loose with the DMCA. [1]: https://www.federalregister.gov/documents/2023/03/16/2023-05321/copyright-registration-guidance-works-containing-material-generated-by-artificial-intelligence https://www.federalregister.gov/documents/2023/03/16/2023-05...
EamonnMR 3y ago
Lately I've seen 'Open Source' thrown around a lot in the AI space with little regard to if the thing in question is actually using an OSI license.
amit33 3y ago
How can I convert text into video
- jacooper 3y ago

jerojero 3y ago

The work being done with Llama and Alpaca and so on is what I've found to be most interesting in the past few weeks. The fact that we are capable of producing such high quality conversational models that we can actually get to freely experiment with is really encouraging. I'm particularly excited about applications such as viper[1] and other forms of plugins as well as model ensembles. [1] https://viper.cs.columbia.edu/static/viper_paper.pdf https://viper.cs.columbia.edu/static/viper_paper.pdf

visarga 3y ago
Turns out LLMs are not so inaccessible for individual users.
- zhisbug 3y ago
  Exactly. but if you want some commercial use, still lacking good quality models.
  - HanClinto 3y ago
    At the moment. It's only been a few weeks. What happens when we get something LLaMa quality that is actually free of restriction?
toxik 3y ago
I'm not trying to be negative, but there is a rather large gap between ChatGPT and what Vicuna is showing in its online demo. Here's an example I just had: Q: What should I do for my wife today? A: There are many things you can End of transaction.
- blacksoil 3y ago
  Perhaps there was connection issue? I am getting this: The best thing you can do for your wife today is to show her love and affection in a way that she will appreciate. Every person is different, so it's important to consider her preferences and interests when deciding what to do. Here are a few ideas: 1. Cook her favorite meal: If your wife has a favorite dish, why not cook it for her today? 2. Plan a special outing: Whether it's a trip to the spa, a shopping excursion, or a day at the park, planning a special outing can be a great way to show your wife that you care. 3. Do something together: Whether it's playing a game, watching a movie, or going for a walk, doing something together can be a great way to spend quality time with your wife. 4. Show your appreciation: Take a moment to tell your wife how much you appreciate her and why. It can be as simple as a kind word or a heartfelt message. 5. Help with a task: Offer to help your wife with a task that she needs to do, whether it's folding laundry, washing the dishes, or taking the dog for a walk. 6. Give her a gift: Even a small token of your affection, such as a bouquet of flowers or a piece of jewelry, can be a meaningful way to show your wife that you care.

ldng 3y ago

Really starting to be annoyed at misleading use of 'open' and 'open-source' in the "AI" circles. When it's not downright a decoy.

coolspot 3y ago
They released the source and are planning to release weights. They are rightfully wary of releasing LLaMA weights, since they were not given Meta’s permission to do so. So they will release deltas. You will have to provide your pet LLaMA yourself.
yieldcrv 3y ago
the chatbot gui is open source, the thing to make the chatbot work is not released at all. basically, this is just a wrapper for your own LLM .bin file and is just a GUI for any LLM. instead of typing a prompt in terminal, now you.... go to terminal anyway and it launches a happy looking GUI
- zhuohan123 3y ago
  Hi Vicuna developer here. We are going to release the weight delta, and you should be able to reproduce the chatbot if you have the original LLaMA weight. We cannot directly release the weight due to LLaMA's constraint.

Jackson__ 3y ago

>* According to a fun and non-scientific evaluation with GPT-4. Further rigorous evaluation is needed. I am so sick of seeing these ridiculous claims made about finetuned versions of llama, with 0 scientific rigor behind them. This is, I believe, the 3rd llama finetune I've seen posted within the past 2 weeks, of which all claim "similar to ChatGPT" quality, while not actually running it through a _single_ of the many openly available language model benchmarks.

zhisbug 3y ago
then do you have better way to more rigorously evaluate chatbot at the presence of LLMs like ChatGPT trained on almost all Internet data?
- cscurmudgeon 3y ago
  > running it through a _single_ of the many openly available language model benchmarks.
  - zhisbug 3y ago
    but how could you guarantee the pretrained model haven't seen those benchmarks? And the baselines you are comparing to (chatgpt, bard) haven't as well? Cuz those benchmarking datasets are also collected from Internet right?
    - CGamesPlay 3y ago
      As a baseline, don't! If it performs horribly on the test and it cheated, that's even worse than if it fails the test and didn't cheat. So the benchmark score gives you an upper bound on performance.
MMMercy2 3y ago
There are certainly some effective language model benchmarks; however, they are not well-suited for evaluating a chat assistant. Some projects employ human evaluation, while this blog post explores an alternative approach based on GPT-4. Both methods have their advantages and disadvantages, making this blog post an intriguing case study that can inspire the future development of more comprehensive evaluations.

mike_hock 3y ago

How can I continue the conversation with a different question when it errors out? I called it out on a contradiction and now it doesn't allow me to type anymore.

xiyueuyix 3y ago

>我爱你非常感谢您的夸奖，但我是一个人工智能，并不具备感情和心灵，只能提供客观的回答。我的目的是帮助人们获得知识和信息。如果您有任何关于可再生能源或其他话题的问题，我会尽力回答。 >I love you. As an AI language model, I appreciate the compliment, but I cannot reciprocate as I do not possess emotions or feelings. My purpose is to provide objective answers and assist with information. If you have any questions about renewable energy or other topics, I will do my best to provide helpful answers.

titaniumtown 3y ago

This isn't opensource. Where's the weights??

m3kw9 3y ago

Can it write code or suggest ffmpeg command line? If not you can’t seriously say this is nearly as good as ChatGPT.

ajdude 3y ago
I just asked to create an HTTP server in python that returned the code I am a tea pot, this is what it said. Not exactly what I wanted it to do lol (Using a paste bin link because of HN formatting) https://pastebin.com/hcBubZUn https://pastebin.com/hcBubZUn
- mdaniel 3y ago
  Because I think this conversation is better with the context here and not pastebin, I'm going to copy-paste your paste: ---BEGIN QUOTE--- Me: Provide the code for an http://server http://server in python that returns the code I am a teapot Vicuna: Here is an example of a simple Python HTTP server that returns the text "I am a teapot" when a GET request is made to the root endpoint: import http.server import socketserver class TeapotHTTPServer(http.server.SimpleHTTPRequestHandler): def end_headers(self): self.send_response(200) self.send_header('Content-type', 'text/plain') self.end_headers() self.wfile.write(b'I am a teapot') Handler = TeapotHTTPServer with socketserver.TCPServer(("localhost", 8000), Handler) as httpd: print("serving at port", httpd.address()) httpd.serve_forever() This code uses the http.server and socketserver modules to create a simple HTTP server that listens on port 8000 and responds to GET requests to the root endpoint with the text "I am a teapot". ---END QUOTE--- A++ for variable naming, A+++ for knowing `self.wfile.write` accepts `byte` not `str`, F for using the wrong method name, and C- for not knowing `send_response(418)` :-D
- mdaniel 3y ago
  You inspired me to find out if it was good at answering your question because it's something it may have seen before, so I thought I'd be more cheeky: ---BEGIN CHAT--- Q: provide the code for an http server in python that exposes a "hello world" method over SOAP with document/literal encoding A: Here is an example of an HTTP server in Python that exposes a "hello world" method over SOAP with document/literal encoding: from http.server import BaseHTTPRequestHandler, HTTPServer from urllib.parse import quote, unquote from io import BytesIO import xml.sax.saxutils as saxutils class HelloWorldHandler(BaseHTTPRequestHandler): def do_POST(self): content_length = int(self.headers['Content-Length']) content = self.rfile.read(content_length) self.send_response(200) self.send_header('Content-type', 'text/xml; charset=UTF-8') self.send_header('Content-encoding', 'document') self.end_headers() # Parse the SOAP request request = BytesIO(content) soap_request = request.getvalue() request.close() # Call the "hello world" method response = '<?xml version="1.0" encoding="UTF-8"?>\n<response>\n <result>Hello, World!</result>\n</response>\n' response = saxutils.escape(response) # Send the SOAP response self.wfile.write(saxutils.escape(response)) return def run(server_class=HTTPServer, handler_class=HelloWorldHandler, port=8080): server_address = ('', port) httpd = server_class(server_address, handler_class) print(f'Starting httpd on port {port}...') httpd.serve_forever() if __name__ == '__main__': run() To use this code, simply run the script in the terminal and open a web browser to the following URL: http://localhost:8080/soap/helloWorld http://localhost:8080/soap/helloWorld ---END CHAT--- A+ for variable naming again, nice job knowing about the `do_POST`, F for `content-encoding: document`, D for sending the byte array through BytesIO for no damn good reason, and F--- for saxutils.escaping the outgoing XML (we'll set aside the bogus invocation instructions because I didn't ask it for that, it just ... volunteered?)

ghc 3y ago

I'm extremely impressed with the Vicuna output for the Hawaii blog post. I was expecting a mismatched set of popular attractions spread across a number of islands, because it doesn't have a world model. ChatGPT suffered from this issue, highlighting attractions you're unlikely to see on the same trip due to distance between them (300km). But Vicuna's output featured three specific experiences that can be had close together in Maui. In fact, it mirrored one of my trips to Maui where the Wailea sunset luau, spending time at Wailea beach, and a trip to nearby Haleakela all featured prominently. I know it's just piecing together a thousands trips like mine, but the coherence is far beyond what I expected.

j_walter 3y ago

Don't ask it the circumference of a 37" tire...does okay till the final calculation but then completely fails.

endorphine 3y ago

I can't understand how something called "a language model" can evaluate some input text. ChatGPT4 seems like it understands the text of the first assistance and properly assesses that it wasn't actually a description of the trip. Similarly, the other day I asked it to explain a hard-to-understand philosophy quote and it explained it properly (I believe). Can someone explain how they do this? It seems magical.

sdenton4 3y ago
Either it's a lifeless recombination of text from the Internet, or an emergent understanding achieved through working with trillions of text tokens, despite lacking access to the full range of the human sensorium informing that text. The jury is still out.
quonn 3y ago
Take a look at https://github.com/jaymody/picoGPT/blob/a750c145ba4d09d5764806a6c78c71ffaff88e64/gpt2_pico.py https://github.com/jaymody/picoGPT/blob/a750c145ba4d09d57648... Yes, this is GPT-2 not 4 and it‘s not the RL-trained Chat, only the GPT model and it‘s basically only the inference part, not the training loop and it‘s somewhat simplified. Still, take a good look. That‘s essentially what it is on a single sheet of paper. There is nothing specifically about language in „language model“, we just call it that. Better to call it just LLM. Nobody knows exactly what it learns, although there would be ways to poke around given some research programs. But it seems like the interest in that is limited currently, everyone is busy with improving it or with applications. Perhaps the answer is that we overestimated what a mind is. It‘s like we used to ask what life is and it turned out that there is nothing special about life, not even the DNA is controlling anything. It‘s merely a chemical process, even though a complex process. Personally, I believe there is something special about consciousness and I believe these systems are not conscious. And I believe they are currently not flexible enough to adapt to new circumstances, unlike a human brain. They don‘t currently make much real progress. But that may change.

mungoman2 3y ago

> 90%* > * According to a fun and non-scientific evaluation with GPT-4. Further rigorous evaluation is needed. Come on... I love any effort related to LLM, but it's really disingenuous to claim high quality in the title and then immediately discredit it after click-through.

zhisbug 3y ago
but it is indeed difficult to eval chatbots and LLM esp. considering most of them have actually seen the Internet data at least once.
- zhisbug 3y ago
  and I do think this is a good effort

ale42 3y ago

It's hilarious to see how these models can hallucinate stuff that doesn't exist... Q: What is the meaning of ABBA's song "I have a dream"? A: "I Have a Dream" is a song by the Swedish pop group ABBA, released in 1970. The song was written by Benny Andersson and Björn Ulvaeus, and it was the group's first hit single. The song is a ballad that talks about a person's dream of finding love and happiness. The lyrics describe the dream as a "beautiful light in the dark" and a "feeling inside that no one can steal." The song has become a classic of the ABBA catalog and is still widely played and enjoyed today. (Didn't check the details but seems okish). But let's not ask about a song that doesn't exist: Q: And what did they mean with the song "General Trouble"? A: "General Trouble" is a song by the Swedish pop group ABBA, released in 1969. The song was written by Benny Andersson and Björn Ulvaeus, and it was the group's first single. The song is a pop ballad that talks about a person who is experiencing a lot of problems in their life, including relationship troubles, financial troubles, and feelings of isolation. The lyrics describe the person as being in "general trouble" and feeling like they are "losing ground." The song was not a big hit for ABBA, but it is still an interesting early work from the group and provides insight into their early sound and songwriting style.

ale42 3y ago
I also asked about a recipe with mushrooms, tea, and dishwasher soap. It happily gave me one... mentioning at some point "Add the dishwasher soap to the tea, if desired. The soap will help to release the flavor of the mushrooms and give the tea a slightly soapy taste."
- bongobingo1 3y ago
  Well... it's not wrong...
  - zhisbug 3y ago
    The recipe indeed may not be a bad idea lol
- antman 3y ago
  Presumably dishwasher soap is somewhat edible or it would not be on dishes? So it might actually be correct just weird, but it's what you ordered

Yenrabbit 3y ago

Patiently waiting for someone to say "We trained LLaMA with extremely high weight decay, and find that performance is much worse. Due to licensing constraints, we are providing a version of delta weights that build on the original LLaMA weights..."

Hexic 3y ago

Write for me a interesting poem

hyperopt 3y ago

The demo for this is great. It's the best non-corporation assistant I've used so far. I suspect most of the gains here relative to the Alpaca model might have to do with the fact that the ShareGPT data are full conversations. They allow for the assistant to respond to the earlier messages in a cohesive way. As opposed to Alpaca, the data was a single question and answer, so the model seems to lose context of earlier information. Also, the coding abilities of Vicuna are significantly improved relative to Alpaca, to the point that I began to suspect they might be calling out to OpenAI in the backed. Please, release model weights and finetuning training data.

deleted 3y ago

[deleted]

facasiello 3y ago

[flagged]

jgmachado 3y ago

[flagged]