5 ms·

> Cautioning its own workers not to directly use code generated by Bard undermines Google's claims its chatbot can help developers become more productive The d

> Cautioning its own workers not to directly use code generated by Bard undermines Google's claims its chatbot can help developers become more productive The definitive definition of irony o_O This reminds me of two pictures that were circulating on social media a while ago. The first was Zuckerberg holding a book while billions of people are wasting their time in endless scrolling in the Metas; the second was that Turkish chef known as Burak eating a healthy lite salad. The bottom line is that we sell stuff that we are not convinced to consume.

deleted 3y ago
[deleted]
lynx23 3y ago
Drug dealers have figured this out long ago. Never get high on your own stuff... No surprise this is the same with tech.
- Lacerda69 3y ago
  Smoking crack /= using your own software. In fact i would recommend every dev to use their own stuff, "dogfooding" helps a lot to improve the product.
  - TeMPOraL 3y ago
    The thing you're dogfooding isn't exactly what's being sold - it's missing the layers of crack that get added so the product makes more money.
soultrees 3y ago
This is less of a productivity risk than a legal risk. Nobody is denying that these tools enable productivity like we haven’t seen before, but google is afraid of outputting protected code while the courts haven’t fully decided the stance on things. Imagine Google Bard outputs a piece of code for a major new feature, making them a shit ton of money. It’s then found out that code was actually taken copied, almost directly from average Joe’s hobby app, and now you’ve got a situation on your hands. Amazing for the average joe who’s expected to get a big pay day if the courts ruled in his favour but less so for the business owner. Frankly, I can’t wait to see what happens when all copyright is out of the window. Obviously within reason, but wait until you see the innovation that comes about when the gates are open. The free-for-all of ideas is a fun time to live in that’s for sure.
- martin8412 3y ago
  Imagine Bard outputs code that is licensed under AGPL.. Suddenly whatever services the code gets included in needs to have their source code made public.
  - giancarlostoro 3y ago
    That alone makes me never want to use any LLM for code specifics outside of asking concepts questions when the brains just not remembering.
- jfk13 3y ago
  > Obviously within reason Oh, so you don't really mean "when all copyright is out of the window", I guess. So.... how are we going to define "within reason"?
  - soultrees 3y ago
    I don’t know how to answer this other than I’m not smart enough to know what the solution is. The ‘within reason’ is simply to appease any unknowns that I’m not accounting for when I say we should just open the flood gates. What that looks like, who knows.
- sidewndr46 3y ago
  based on the code I've seen come out of these tools, I'm denying they enable any productivity gains. The code is either hilariously bad, filled with subtle bugs, or just something I could have googled then copy and pasted an OSS solution into my code. Is anyone's productivity really higher if we have to spend all our time correctness checking the output?
  - soultrees 3y ago
    We should test it and your ego at the same time. We might find some interesting results. Find someone at the same level of competency as you, set a goal, that person gets access to any LLM they want, you have to do it with no assistance other than reading stack or googling. And if you think your ego can handle it, find someone who you know is below you in competency (but not junior) and set the same goal and see if they surpass your own productivity or progress. We’d have to set up some controls but you see where I’m getting at. It would be an interesting experiment either way. It’s all in how you use the tools. If you’re zero shotting and hoping for the best then yeah, not great. But that’s where your domain expertise is supposed to come in.
ilitirit 3y ago
I asked Bard to suggest improvements to a SQL query I wrote. It suggested that I change my filter to improvement performance. The filter I had was: WHERE Column1 <> Column2 The suggested improvement: WHERE Column1 <> Column2 When I pointed out that it just gave me exactly the same code, it apologised and suggested this improvement instead: WHERE Column1 > Column2 I'm trying to understand how this helps me become more productive...
- deleted 3y ago
  [deleted]
- michaelt 3y ago
  > I'm trying to understand how this helps me become more productive... LLMs writing code have strange performance characteristics. Sometimes they produce working output many times faster than the fastest of senior developers, other times they can't understand a bug even when you point it out to them. Try giving your LLM this prompt: Please write a python script that loads a file named input.png, converts it to greyscale, plots a chart of the intensity of the pixels of the first row, and saves the chart as output.png . The chart should be a line chart with a blue line, and the lowest and highest points on the chart should be marked with a red cross. Thanks! So LLMs can sometimes produce working code very fast. Other times, as you've identified, they produce completely broken results.
- wkat4242 3y ago
  Wow I can't use bard yet as I'm in the EU. But ChatGPT has never done something like this to me.
  - cmrdporcupine 3y ago
    GPT4 produces dubious Rust code all the time for me, especially when I point out mistakes. It gets into a cycle of doubling down on mistakes and degrading into more and more bizarre solutions and gets into a loop of repeating itself. It gets the rough outline of a solution OK but very easily gets lost if the problem requires 2 or 3 levels of what I guess I'd call nested analysis. Or rather, it's fine for creating a surface solution, but it gets lost in the weeds quite easily for things like type system or borrow checker problems. Because it doesn't reason -- it (re)produces patterns.
freediver 3y ago
Eating your own dog food is an expression commonly used to characterize this.
dijit 3y ago
I don't think that's an accurate representation at all; and I say this as a person whom has also restricted the use of AI in their company. The last time I read the Github CoPilot agreement for example it used specific language such as: "Suggestions"; and seemed to indicate that copilot was merely inspiring developers through suggestion rather than creating code that could (or should) be copied verbatim. I think in reality most people are copying code verbatim and not using AI as inspiration, but it seems like (legally speaking) that it is the safest approach to treat AI generated code as non-copyable.. That notion is consistent with what Google is stating here.
giancarlostoro 3y ago
My guess is they dont want their own processes and code secrets leaked through Bard.
GuB-42 3y ago
There is nuance here. Google policy is "do not use code generated by Bard", not "do not use Bard". There are ways for developers to use chatbots that doesn't involve copy-pasting code into your own codebase. Just like there is a difference between not using stackoverflow (which is foolish) and not copy-pasting from stackoverflow (which is sensible). I have used ChatGPT a few times as a developer and found it useful, but never directly copied code from it. I had it document my code for me though, so maybe if you count comments as code, I did.

deleted 3y ago

[deleted]

lynx23 3y ago

Drug dealers have figured this out long ago. Never get high on your own stuff... No surprise this is the same with tech.

Lacerda69 3y ago
Smoking crack /= using your own software. In fact i would recommend every dev to use their own stuff, "dogfooding" helps a lot to improve the product.
- TeMPOraL 3y ago
  The thing you're dogfooding isn't exactly what's being sold - it's missing the layers of crack that get added so the product makes more money.

soultrees 3y ago

This is less of a productivity risk than a legal risk. Nobody is denying that these tools enable productivity like we haven’t seen before, but google is afraid of outputting protected code while the courts haven’t fully decided the stance on things. Imagine Google Bard outputs a piece of code for a major new feature, making them a shit ton of money. It’s then found out that code was actually taken copied, almost directly from average Joe’s hobby app, and now you’ve got a situation on your hands. Amazing for the average joe who’s expected to get a big pay day if the courts ruled in his favour but less so for the business owner. Frankly, I can’t wait to see what happens when all copyright is out of the window. Obviously within reason, but wait until you see the innovation that comes about when the gates are open. The free-for-all of ideas is a fun time to live in that’s for sure.

martin8412 3y ago
Imagine Bard outputs code that is licensed under AGPL.. Suddenly whatever services the code gets included in needs to have their source code made public.
- giancarlostoro 3y ago
  That alone makes me never want to use any LLM for code specifics outside of asking concepts questions when the brains just not remembering.
jfk13 3y ago
> Obviously within reason Oh, so you don't really mean "when all copyright is out of the window", I guess. So.... how are we going to define "within reason"?
- soultrees 3y ago
  I don’t know how to answer this other than I’m not smart enough to know what the solution is. The ‘within reason’ is simply to appease any unknowns that I’m not accounting for when I say we should just open the flood gates. What that looks like, who knows.
sidewndr46 3y ago
based on the code I've seen come out of these tools, I'm denying they enable any productivity gains. The code is either hilariously bad, filled with subtle bugs, or just something I could have googled then copy and pasted an OSS solution into my code. Is anyone's productivity really higher if we have to spend all our time correctness checking the output?
- soultrees 3y ago
  We should test it and your ego at the same time. We might find some interesting results. Find someone at the same level of competency as you, set a goal, that person gets access to any LLM they want, you have to do it with no assistance other than reading stack or googling. And if you think your ego can handle it, find someone who you know is below you in competency (but not junior) and set the same goal and see if they surpass your own productivity or progress. We’d have to set up some controls but you see where I’m getting at. It would be an interesting experiment either way. It’s all in how you use the tools. If you’re zero shotting and hoping for the best then yeah, not great. But that’s where your domain expertise is supposed to come in.

ilitirit 3y ago

I asked Bard to suggest improvements to a SQL query I wrote. It suggested that I change my filter to improvement performance. The filter I had was: WHERE Column1 <> Column2 The suggested improvement: WHERE Column1 <> Column2 When I pointed out that it just gave me exactly the same code, it apologised and suggested this improvement instead: WHERE Column1 > Column2 I'm trying to understand how this helps me become more productive...

deleted 3y ago
[deleted]
michaelt 3y ago
> I'm trying to understand how this helps me become more productive... LLMs writing code have strange performance characteristics. Sometimes they produce working output many times faster than the fastest of senior developers, other times they can't understand a bug even when you point it out to them. Try giving your LLM this prompt: Please write a python script that loads a file named input.png, converts it to greyscale, plots a chart of the intensity of the pixels of the first row, and saves the chart as output.png . The chart should be a line chart with a blue line, and the lowest and highest points on the chart should be marked with a red cross. Thanks! So LLMs can sometimes produce working code very fast. Other times, as you've identified, they produce completely broken results.
wkat4242 3y ago
Wow I can't use bard yet as I'm in the EU. But ChatGPT has never done something like this to me.
- cmrdporcupine 3y ago
  GPT4 produces dubious Rust code all the time for me, especially when I point out mistakes. It gets into a cycle of doubling down on mistakes and degrading into more and more bizarre solutions and gets into a loop of repeating itself. It gets the rough outline of a solution OK but very easily gets lost if the problem requires 2 or 3 levels of what I guess I'd call nested analysis. Or rather, it's fine for creating a surface solution, but it gets lost in the weeds quite easily for things like type system or borrow checker problems. Because it doesn't reason -- it (re)produces patterns.

freediver 3y ago

Eating your own dog food is an expression commonly used to characterize this.

dijit 3y ago

I don't think that's an accurate representation at all; and I say this as a person whom has also restricted the use of AI in their company. The last time I read the Github CoPilot agreement for example it used specific language such as: "Suggestions"; and seemed to indicate that copilot was merely inspiring developers through suggestion rather than creating code that could (or should) be copied verbatim. I think in reality most people are copying code verbatim and not using AI as inspiration, but it seems like (legally speaking) that it is the safest approach to treat AI generated code as non-copyable.. That notion is consistent with what Google is stating here.

giancarlostoro 3y ago

My guess is they dont want their own processes and code secrets leaked through Bard.

GuB-42 3y ago

There is nuance here. Google policy is "do not use code generated by Bard", not "do not use Bard". There are ways for developers to use chatbots that doesn't involve copy-pasting code into your own codebase. Just like there is a difference between not using stackoverflow (which is foolish) and not copy-pasting from stackoverflow (which is sensible). I have used ChatGPT a few times as a developer and found it useful, but never directly copied code from it. I had it document my code for me though, so maybe if you count comments as code, I did.