9 ms·

Gitlab’s AI-assisted code suggestions

168 points by sh_tomer 3y ago

whimsicalism 3y ago
editorialized title
dgrin91 3y ago
I wish them luck. While I'm not a huge fan of copilot coding AIs, I understand how they are going to be a killer feature for many and I worry that it would create a big moat for Github. I hope that Gitlab is able to close that gap because I really like Gitlab.
- cassianoleal 3y ago
  Codeium [0] seems like a worthy competitor to Copilot already. [0] https://codeium.com/ https://codeium.com/
  - luxurytent 3y ago
    Don't sleep on Cody either https://about.sourcegraph.com/cody https://about.sourcegraph.com/cody
  - michaelmior 3y ago
    I've been using Codeium with Vim for a few months and it works great :)
  - dustyharddrive 3y ago
    I’m spooked by this comment: https://news.ycombinator.com/item?id=34434995 https://news.ycombinator.com/item?id=34434995
  - moonchrome 3y ago
    I've tried it for a few days a month ago and it was noticably worse than copilot, copilot gave much more one shot completions sooner while codeium was line by line on same code. Way better than the AWS code whisperer or whatever it's called but still not worth switching. Especially since I trust GH way more than some random company (GH already has access to my code anyway)
    - ColonelPhantom 3y ago
      I should try Codeium then, Copilot's occasional "heap of code" was one of the most annoying parts of using it to me. Why? Because it looks plausible, so I accept it without actually understanding it. But using it as a better autocomplete is actually quite nice, for example when I have some more repetitive code to write. And a line-by-line AI completer will at least give me the time to read it piecemeal, which I find much easier. Just like I understood way better when my math teacher demonstrated how to solve a problem compared to reading it from the textbook.
      - moonchrome 3y ago
        I'm already in the flow with copilot so I'm pretty good at expecting when it's useful. Basically I use it as AI copy paste. When I have variations of methods or conditional logic copilot is great at one shot solutions and it's trivial to verify (if it's wrong somehow then I would have done the wrong thing as well - it just saves me a minute of copy/paste and refactoring) copilot was just better.
ases 3y ago
All the code examples on this page are doubling any brackets ([], {}, ()). How have they managed that? Not a fantastic first impression of its capabilities...
- mmcclure 3y ago
  I'm going to give them the benefit of the doubt and assume this is a marketing site problem and not a product problem, but either way, not a good look. Reminds me of Google's big Bard announcement including a wrong answer in the image.
  - SparkyMcUnicorn 3y ago
    Gitlab is using Google's Vertex Codey models/API for the code completions. Showcasing innacuracies in AI responses must be a Google requirement.
- Chiron1991 3y ago
  Not only that, but the Golang example is full of errors. Parameter definitions don't have colons between name and type. The map for seen elements is declared as a, but later referenced to as m. The append instruction references uniques, which is undefined. The return statement also references uniques.
  - blibble 3y ago
    ah yes, so a typical example of generative AI output
  - 0xjnml 3y ago
    The Go code shown now is both syntactically valid and semantically correct: https://go.dev/play/p/GDki9v78jMM https://go.dev/play/p/GDki9v78jMM I can't check how it looked 15 hours ago. Makes me wonder if it was manually corrected and is thus a fake example now.
- john_cogs 3y ago
  GitLab team member here. Thanks for flagging. Our web team is working to resolve this issue here: https://gitlab.com/gitlab-com/marketing/digital-experience/buyer-experience/-/merge_requests/2546 https://gitlab.com/gitlab-com/marketing/digital-experience/b...
RadiozRadioz 3y ago
Crucial: do they train on GPL code? If I am to use this tool, I must abide by the license terms of the training data. Even if it is found that the GPL does not cover LLM responses as derivative works, I would prefer to be on the safe side and refuse to use models trained on software with non-permissive licenses unless I am building Free Software.
- JoshTriplett 3y ago
  GitLab seems to be a lot more responsive to feedback. It'd be worth asking them if they can publicly document their training set and provide a list of licenses and copyright notices.
  - dnsmichi 3y ago
    GitLab team member here. The training data is documented in https://docs.gitlab.com/ee/user/project/repository/code_suggestions.html#training-data https://docs.gitlab.com/ee/user/project/repository/code_sugg... AI Transparency is important, all available AI features provide documentation for training data, and are built with privacy first. The GitLab Duo announcement adds more feature details and plans. https://about.gitlab.com/blog/2023/06/22/meet-gitlab-duo-the-suite-of-ai-capabilities/ https://about.gitlab.com/blog/2023/06/22/meet-gitlab-duo-the... The AI/ML blog series provides insights on experiments, and features being built. https://about.gitlab.com/blog/2023/04/24/ai-ml-in-devsecops-series/ https://about.gitlab.com/blog/2023/04/24/ai-ml-in-devsecops-...
    - JoshTriplett 3y ago
      "Codey was fine-tuned on a large dataset of high quality, permissively licensed code from external sources" is not sufficient information to be able to provide attribution and licensing information. Would it be possible to get a complete list of sources and licenses?
    - snickerbockers 3y ago
      Within this context, what is the definition of "permissively licensed"? More to the point, are the terms of licenses of the code in the training set being honored? There's not actually anything in GPL or any other major FOSS license that prohibits using it to train an AI. The controversy stems from Microsoft refusing to follow the terms of those licenses. If they could just fulfill their obligation to propagate copyright statements and license text there would be no controversy over copilot. Thus stating that it was trained only on "permissive" licenses doesn't actually answer the question without defining what a permissive license is.
Xeamek 3y ago
...something something do not editorialize titles something something...
Dudester230602 3y ago
So a follower, not a leader.
ianhawes 3y ago
This looks very interesting, but as an FYI using LLMs to do machine translation is a terrible waste of resources. The example on their landing page has a "translate.py" sample which, albeit handy, is not something I would do beyond basic string translations.
- lucasmullens 3y ago
  > using LLMs to do machine translation is a terrible waste of resources. Could you elaborate on that? LLMs seem perfectly suited for language tasks like translation. They don't seem particularly expensive either, especially compared to hiring a person.
  - nonameiguess 3y ago
    There are already machine-translation services trained and created specifically for that purpose. While it's an amazing realization that LLMs can do this and do it pretty well without having to be trained specifically for this one purpose, training something to do literally all text generation tasks is expensive compared to training something specifically to do language to language translation. For a maybe more obvious example, say that LLMs ever got good enough to do arbitrary precision arithmetic on numbers up to hundreds of digits. Would that be a good use of one when calculators can already do this and are far cheaper to produce? I guess it makes no difference from a free-tier consumer's perspective, but it's still more expensive even if you aren't personally paying the expense.
    - senko 3y ago
      LLMs are the machine-translation services created specifically for that purpose[0], it just turned out they're very good at many other things! Your analogy would be like saying why use a computer to multiply numbers if you can calculate them using calculator, which is much cheaper. Sure, but if you already have a computer, no need to use a dedicated calculator as wel. [0] https://nlp.seas.harvard.edu/annotated-transformer/#results https://nlp.seas.harvard.edu/annotated-transformer/#results
    - 3y ago
renewiltord 3y ago
Going to be interesting. Github had the advantage that they were offering Copilot into an audience of people who just defaulted to them. Gitlab on the other hand is for people who are into license wars, etc. so the AI product is going to be offered to a hostile audience.
- lolinder 3y ago
  These days Gitlab's main customer is the enterprise. They pushed away a lot of their open source users when they heavily curtailed the free tier.
frankgrecojr 3y ago
Have they said what LLM they use?
- SparkyMcUnicorn 3y ago
  Google's Codey API. https://docs.gitlab.com/ee/user/project/repository/code_suggestions.html#code-suggestions-data-usage https://docs.gitlab.com/ee/user/project/repository/code_sugg...
coolgoose 3y ago
So whos code is used for training?
- dnsmichi 3y ago
  GitLab team member here. The training data is publicly documented, and AI features are built with privacy first. For all URLs please check my comment in https://news.ycombinator.com/item?id=36526159 https://news.ycombinator.com/item?id=36526159
  - elforce002 3y ago
    Nice. I really like the transparency. That's one of the main reasons I didn't use github copilot.
    - remram 3y ago
      It's not very transparent. It was trained on "permissively licensed code from external sources" which might mean random GPL projects.
    - dnsmichi 3y ago
      Thanks! AI for self-managed instances is also focussing on privacy, while bringing more efficiency to teams. https://about.gitlab.com/blog/2023/06/15/self-managed-support-for-code-suggestions/#how-code-suggestions-uses-data https://about.gitlab.com/blog/2023/06/15/self-managed-suppor...
ezekg 3y ago
Is the Go example even valid? I've never seen that (arr: [[]]int) syntax.
- dharmab 3y ago
  Neither is the Python example
  - ezekg 3y ago
    Well, that's kind of embarrassing... the JS example is also all sorts of messed up.
- esafak 3y ago
  We need LLMs to respect the grammar of the language they are generating in.
  - ShamelessC 3y ago
    They do this naturally when they aren’t bad.
- deleted 3y ago
  [deleted]
phil917 3y ago
I just canceled my Copilot subscription last night. I definitely never saw the productivity boost that I’ve seen so many claim. I actually feel like the suggestions seemed to get worse during my month of using it for some reason. Towards the end, it started suggesting these large blocks of code (another issue I had with the interface as well) which were very not relevant to what I was attempting to write. All in all, I’m very underwhelmed from my first experience with “AI enhanced” coding.
- Tronno 3y ago
  My experience was very similar. Copilot became a sort of improved autocomplete, that would save copy/pasting repeated variable names into tests, logs, etc, but offered no other significant improvements. Its suggestions were frequently incorrect, often not obviously so.
  - benabbottnz 3y ago
    > that would save copy/pasting repeated variable names into tests, logs, etc This is all it takes for me to get value out of my subscription. It's saving me time by anticipating my next test / block of code based on what I've previously written, and that saved time really adds up. Sure it's not "Jesus take the wheel" level AI yet, but time is money, and so is my sanity.
- _ea1k 3y ago
  TBH, using this as a fancy smart autocomplete doesn't seem like the best idea. I can't believe that I'm suggesting something that sounds like Clippy... but... It really needs to be something where you can rattle off a list of requirements and have it build the code for you. The code context is necessary, but not sufficient.
  - bgarbiak 3y ago
    It’s coming up: https://github.com/features/preview/copilot-x https://github.com/features/preview/copilot-x
curtis3389 3y ago
Okay, but how much are they charging? It's getting really annoying how many sites force you into a trial just to find out how much it'll cost when it ends. EDIT: Is this even positioned to compete with Copilot? What editors are there plugins for? There is surprisingly little information on the site.
- deleted 3y ago
  [deleted]
- john_cogs 3y ago
  Code Suggestions can be used in GitLab's Web IDE and VS Code and Microsoft Visual Studio when you have the corresponding GitLab extension installed: https://docs.gitlab.com/ee/user/project/repository/code_suggestions.html#use-code-suggestions https://docs.gitlab.com/ee/user/project/repository/code_sugg... We offer experimental support for additional editors: https://docs.gitlab.com/ee/user/project/repository/code_suggestions.html#enable-code-suggestions-in-other-ides-and-editors https://docs.gitlab.com/ee/user/project/repository/code_sugg...
  - lolinder 3y ago
    That doesn't answer the question of what the pricing will be. I'm assuming it's not going to be free. Will it be a standalone subscription, or another thing that gets bundled into the main tiers to justify the ever-increasing prices?
    - john_cogs 3y ago
      Code Suggestions is free while in Beta. When GA, it will be included in our $9 per user per month AI add-on.
- ollieglass 3y ago
  Seems to be $99 per month. Signing up for the free trial funnels me to the trial of Gitlab Ultimate. So assuming that you need an Ultimate subscription to use it after the trial, that's the price. Pricing is here https://about.gitlab.com/pricing/ https://about.gitlab.com/pricing/ In contrast, Copilot is $10 per month https://github.com/features/copilot#pricing https://github.com/features/copilot#pricing
deleted 3y ago
[deleted]
ldehaan 3y ago
[dead]
smallpipe 3y ago
What nimrod added a chatbot to this page ? It’s downright unusable on mobile, it just keeps popping up with shit. Even closing it creates two more notifications that use up two thirds of the screen
lee101 3y ago
[dead]
HdS84 3y ago
I get that gitlab needs more features to continue to sell. But honestly for me it reduces the attractiveness of gitlab. Just adding half working features does not increase the value for me. Please finish and polish the existing Features before adding hype feature 93939. Maybe add a policy that your open feature and big count should be less than 20K or something. Examples: Security scanning has a god awful UI and UX. The whole project management stuff is barely usable Pipelines have loads of weird half features and missing QOL. E G. Adding saeif support instead of just your own warnings format.
- lvncelot 3y ago
  It's honestly doubly annoying when there's multiple times per month where I run into some issue / straight up bug which I then find an ongoing bug ticket from like 2018 for, that's still open but apparently not being worked on. I really like Gitlab, I think the CI experience is a lot better than, for instance, Github Actions, but I also don't ever see me using a good chunk of the stuff that's been shoveled on in the last few years and I don't really know anyone that does.
  - HdS84 3y ago
    At least for python and dotnet, I think github actions are superior. Gitlab did not support poetry for ages there where bugs with python galore and for dotnet they do not speak the same language - e.g. issues in dotnet are reported as saeif, but gl has it own weird format. And yes, you are totally right with the bug tracker. It's often funny how long the ticket is just because it was retagged sooo many times.
tallytarik 3y ago
GitLab was rather quick to board the AI hype train this year. Fine - they don't want to look like they're behind the curve, have to please investors, and whatever else. But IMO there are plenty of other places to add real value across the GitLab product with AI/ML features. Here, it just looks like they saw GitHub do something and felt a need to copy it. But two years late, and worse. As a longtime GitLab user (and onetime contributor!), I'm a bit disappointed they're spending so much time on this. I think they're just too far behind.
- dnsmichi 3y ago
  GitLab team member here. Thanks for your feedback. > But IMO there are plenty of other places to add real value across the GitLab product with AI/ML features. True, and after starting with ML experiments, the product and engineering teams have been working on new features for entire DevOps lifecycle. All AI workflows on the DevSecOps platforms are described in the GitLab Duo announcement blog post https://about.gitlab.com/blog/2023/06/22/meet-gitlab-duo-the-suite-of-ai-capabilities/ https://about.gitlab.com/blog/2023/06/22/meet-gitlab-duo-the... and website https://about.gitlab.com/gitlab-duo/ https://about.gitlab.com/gitlab-duo/ I'll share a few highlights that I am personally excited about - Explain and help fix security vulnerabilities. From my personal experience, I often find CVEs hard to read, especially when I am not the author of the code to fix. Getting help from AI can reduce entry barriers and make development for efficient. Security is everyone's responsibility these days. This follows the AI assisted feature to explain code in general. "What does this magic loop with memcpy do?" might not stay magic anymore, easing the path to code refactoring, improving performance, and reduce the resource usage footprint. - Summarize issue comments. Feature proposals or bug analysis can have long comment threads that require reading time. AI will help get the gist and better contribute to what has been discussed. - Summarize MR changes, to avoid reading long change diffs. This helps with faster (code) review cycles. I tested it this week with an MR for our handbook in https://gitlab.com/gitlab-com/www-gitlab-com/-/merge_requests/126426#note_1449388720 https://gitlab.com/gitlab-com/www-gitlab-com/-/merge_request... I'd also like to see AI helping fix CI/CD pipelines fast. Proposal in https://gitlab.com/gitlab-org/gitlab/-/issues/386863 https://gitlab.com/gitlab-org/gitlab/-/issues/386863 I shared some thoughts in a new talk "Observability for Efficient DevSecOps Pipelines", slides in https://go.gitlab.com/VDAvMw https://go.gitlab.com/VDAvMw (GitLab blog post coming soon, https://gitlab.com/gitlab-com/www-gitlab-com/-/issues/34296 https://gitlab.com/gitlab-com/www-gitlab-com/-/issues/34296) Additionally, I learned some new ideas at Cloudland last week, regarding product owner requirements list verification, and end-to-end test automation with AI. Need to create feature proposals :-) > As a longtime GitLab user (and onetime contributor!), Thanks for contributing. I'd like to invite you to share your ideas about AI features across the platform :) When you look at the DevOps lifecycle (image in https://about.gitlab.com/gitlab-duo/ https://about.gitlab.com/gitlab-duo/) from plan/manage to create, verify, secure, package, release, deploy, monitor, govern - where do you see yourself, and where do you spend the most time in? Second question: Which process feels the most inefficient? After identifying answers to the questions, please check the AI features https://docs.gitlab.com/ee/user/ai_features.html https://docs.gitlab.com/ee/user/ai_features.html and/or open new feature proposals for GitLab https://gitlab.com/gitlab-org/gitlab/-/issues/new?issuable_template=Feature%20Proposal%20-%20lean https://gitlab.com/gitlab-org/gitlab/-/issues/new?issuable_t... You can tag @dnsmichi so I can engage with your ideas. Thanks!
bilekas 3y ago
It seems I need to have a company to sign up for the trial ? I have a free gitlab account but can't proceed without entering business information as it seems to be tied to the gitlab premium feature set. Am I doing something wrong ?
moonchrome 3y ago
Funny how even the examples they have in the showcase are bad/mediocre code quality.
al_be_back 3y ago
Their service calls Google Vertex AI Codey APIs to generate suggestions [1]. It's fine to integrate Off-the-shelf solutions, but you don't have to implement every tech-hype going viral. I fear AI-assistant features are pushing more prescient features further down the backlog. [1] https://docs.gitlab.com/ee/user/project/repository/code_suggestions.html#code-suggestions-data-usage https://docs.gitlab.com/ee/user/project/repository/code_sugg...

168 points by sh_tomer 3y ago

editorialized title

I wish them luck. While I'm not a huge fan of copilot coding AIs, I understand how they are going to be a killer feature for many and I worry that it would create a big moat for Github. I hope that Gitlab is able to close that gap because I really like Gitlab.

cassianoleal 3y ago
Codeium [0] seems like a worthy competitor to Copilot already. [0] https://codeium.com/ https://codeium.com/
- luxurytent 3y ago
  Don't sleep on Cody either https://about.sourcegraph.com/cody https://about.sourcegraph.com/cody
- michaelmior 3y ago
  I've been using Codeium with Vim for a few months and it works great :)
- dustyharddrive 3y ago
  I’m spooked by this comment: https://news.ycombinator.com/item?id=34434995 https://news.ycombinator.com/item?id=34434995
- moonchrome 3y ago
  I've tried it for a few days a month ago and it was noticably worse than copilot, copilot gave much more one shot completions sooner while codeium was line by line on same code. Way better than the AWS code whisperer or whatever it's called but still not worth switching. Especially since I trust GH way more than some random company (GH already has access to my code anyway)
  - ColonelPhantom 3y ago
    I should try Codeium then, Copilot's occasional "heap of code" was one of the most annoying parts of using it to me. Why? Because it looks plausible, so I accept it without actually understanding it. But using it as a better autocomplete is actually quite nice, for example when I have some more repetitive code to write. And a line-by-line AI completer will at least give me the time to read it piecemeal, which I find much easier. Just like I understood way better when my math teacher demonstrated how to solve a problem compared to reading it from the textbook.
    - moonchrome 3y ago
      I'm already in the flow with copilot so I'm pretty good at expecting when it's useful. Basically I use it as AI copy paste. When I have variations of methods or conditional logic copilot is great at one shot solutions and it's trivial to verify (if it's wrong somehow then I would have done the wrong thing as well - it just saves me a minute of copy/paste and refactoring) copilot was just better.

ases 3y ago

All the code examples on this page are doubling any brackets ([], {}, ()). How have they managed that? Not a fantastic first impression of its capabilities...

mmcclure 3y ago
I'm going to give them the benefit of the doubt and assume this is a marketing site problem and not a product problem, but either way, not a good look. Reminds me of Google's big Bard announcement including a wrong answer in the image.
- SparkyMcUnicorn 3y ago
  Gitlab is using Google's Vertex Codey models/API for the code completions. Showcasing innacuracies in AI responses must be a Google requirement.
Chiron1991 3y ago
Not only that, but the Golang example is full of errors. Parameter definitions don't have colons between name and type. The map for seen elements is declared as a, but later referenced to as m. The append instruction references uniques, which is undefined. The return statement also references uniques.
- blibble 3y ago
  ah yes, so a typical example of generative AI output
- 0xjnml 3y ago
  The Go code shown now is both syntactically valid and semantically correct: https://go.dev/play/p/GDki9v78jMM https://go.dev/play/p/GDki9v78jMM I can't check how it looked 15 hours ago. Makes me wonder if it was manually corrected and is thus a fake example now.
john_cogs 3y ago
GitLab team member here. Thanks for flagging. Our web team is working to resolve this issue here: https://gitlab.com/gitlab-com/marketing/digital-experience/buyer-experience/-/merge_requests/2546 https://gitlab.com/gitlab-com/marketing/digital-experience/b...

RadiozRadioz 3y ago

Crucial: do they train on GPL code? If I am to use this tool, I must abide by the license terms of the training data. Even if it is found that the GPL does not cover LLM responses as derivative works, I would prefer to be on the safe side and refuse to use models trained on software with non-permissive licenses unless I am building Free Software.

JoshTriplett 3y ago
GitLab seems to be a lot more responsive to feedback. It'd be worth asking them if they can publicly document their training set and provide a list of licenses and copyright notices.
- dnsmichi 3y ago
  GitLab team member here. The training data is documented in https://docs.gitlab.com/ee/user/project/repository/code_suggestions.html#training-data https://docs.gitlab.com/ee/user/project/repository/code_sugg... AI Transparency is important, all available AI features provide documentation for training data, and are built with privacy first. The GitLab Duo announcement adds more feature details and plans. https://about.gitlab.com/blog/2023/06/22/meet-gitlab-duo-the-suite-of-ai-capabilities/ https://about.gitlab.com/blog/2023/06/22/meet-gitlab-duo-the... The AI/ML blog series provides insights on experiments, and features being built. https://about.gitlab.com/blog/2023/04/24/ai-ml-in-devsecops-series/ https://about.gitlab.com/blog/2023/04/24/ai-ml-in-devsecops-...
  - JoshTriplett 3y ago
    "Codey was fine-tuned on a large dataset of high quality, permissively licensed code from external sources" is not sufficient information to be able to provide attribution and licensing information. Would it be possible to get a complete list of sources and licenses?
  - snickerbockers 3y ago
    Within this context, what is the definition of "permissively licensed"? More to the point, are the terms of licenses of the code in the training set being honored? There's not actually anything in GPL or any other major FOSS license that prohibits using it to train an AI. The controversy stems from Microsoft refusing to follow the terms of those licenses. If they could just fulfill their obligation to propagate copyright statements and license text there would be no controversy over copilot. Thus stating that it was trained only on "permissive" licenses doesn't actually answer the question without defining what a permissive license is.

Xeamek 3y ago

...something something do not editorialize titles something something...

Dudester230602 3y ago

So a follower, not a leader.

ianhawes 3y ago

This looks very interesting, but as an FYI using LLMs to do machine translation is a terrible waste of resources. The example on their landing page has a "translate.py" sample which, albeit handy, is not something I would do beyond basic string translations.

lucasmullens 3y ago
> using LLMs to do machine translation is a terrible waste of resources. Could you elaborate on that? LLMs seem perfectly suited for language tasks like translation. They don't seem particularly expensive either, especially compared to hiring a person.
- nonameiguess 3y ago
  There are already machine-translation services trained and created specifically for that purpose. While it's an amazing realization that LLMs can do this and do it pretty well without having to be trained specifically for this one purpose, training something to do literally all text generation tasks is expensive compared to training something specifically to do language to language translation. For a maybe more obvious example, say that LLMs ever got good enough to do arbitrary precision arithmetic on numbers up to hundreds of digits. Would that be a good use of one when calculators can already do this and are far cheaper to produce? I guess it makes no difference from a free-tier consumer's perspective, but it's still more expensive even if you aren't personally paying the expense.
  - senko 3y ago
    LLMs are the machine-translation services created specifically for that purpose[0], it just turned out they're very good at many other things! Your analogy would be like saying why use a computer to multiply numbers if you can calculate them using calculator, which is much cheaper. Sure, but if you already have a computer, no need to use a dedicated calculator as wel. [0] https://nlp.seas.harvard.edu/annotated-transformer/#results https://nlp.seas.harvard.edu/annotated-transformer/#results
  - 3y ago

renewiltord 3y ago

Going to be interesting. Github had the advantage that they were offering Copilot into an audience of people who just defaulted to them. Gitlab on the other hand is for people who are into license wars, etc. so the AI product is going to be offered to a hostile audience.

lolinder 3y ago
These days Gitlab's main customer is the enterprise. They pushed away a lot of their open source users when they heavily curtailed the free tier.

frankgrecojr 3y ago

Have they said what LLM they use?

SparkyMcUnicorn 3y ago
Google's Codey API. https://docs.gitlab.com/ee/user/project/repository/code_suggestions.html#code-suggestions-data-usage https://docs.gitlab.com/ee/user/project/repository/code_sugg...

coolgoose 3y ago

So whos code is used for training?

dnsmichi 3y ago
GitLab team member here. The training data is publicly documented, and AI features are built with privacy first. For all URLs please check my comment in https://news.ycombinator.com/item?id=36526159 https://news.ycombinator.com/item?id=36526159
- elforce002 3y ago
  Nice. I really like the transparency. That's one of the main reasons I didn't use github copilot.
  - remram 3y ago
    It's not very transparent. It was trained on "permissively licensed code from external sources" which might mean random GPL projects.
  - dnsmichi 3y ago
    Thanks! AI for self-managed instances is also focussing on privacy, while bringing more efficiency to teams. https://about.gitlab.com/blog/2023/06/15/self-managed-support-for-code-suggestions/#how-code-suggestions-uses-data https://about.gitlab.com/blog/2023/06/15/self-managed-suppor...

ezekg 3y ago

Is the Go example even valid? I've never seen that (arr: [[]]int) syntax.

dharmab 3y ago
Neither is the Python example
- ezekg 3y ago
  Well, that's kind of embarrassing... the JS example is also all sorts of messed up.
esafak 3y ago
We need LLMs to respect the grammar of the language they are generating in.
- ShamelessC 3y ago
  They do this naturally when they aren’t bad.
deleted 3y ago
[deleted]

phil917 3y ago

I just canceled my Copilot subscription last night. I definitely never saw the productivity boost that I’ve seen so many claim. I actually feel like the suggestions seemed to get worse during my month of using it for some reason. Towards the end, it started suggesting these large blocks of code (another issue I had with the interface as well) which were very not relevant to what I was attempting to write. All in all, I’m very underwhelmed from my first experience with “AI enhanced” coding.

Tronno 3y ago
My experience was very similar. Copilot became a sort of improved autocomplete, that would save copy/pasting repeated variable names into tests, logs, etc, but offered no other significant improvements. Its suggestions were frequently incorrect, often not obviously so.
- benabbottnz 3y ago
  > that would save copy/pasting repeated variable names into tests, logs, etc This is all it takes for me to get value out of my subscription. It's saving me time by anticipating my next test / block of code based on what I've previously written, and that saved time really adds up. Sure it's not "Jesus take the wheel" level AI yet, but time is money, and so is my sanity.
_ea1k 3y ago
TBH, using this as a fancy smart autocomplete doesn't seem like the best idea. I can't believe that I'm suggesting something that sounds like Clippy... but... It really needs to be something where you can rattle off a list of requirements and have it build the code for you. The code context is necessary, but not sufficient.
- bgarbiak 3y ago
  It’s coming up: https://github.com/features/preview/copilot-x https://github.com/features/preview/copilot-x

curtis3389 3y ago

Okay, but how much are they charging? It's getting really annoying how many sites force you into a trial just to find out how much it'll cost when it ends. EDIT: Is this even positioned to compete with Copilot? What editors are there plugins for? There is surprisingly little information on the site.

deleted 3y ago
[deleted]
john_cogs 3y ago
Code Suggestions can be used in GitLab's Web IDE and VS Code and Microsoft Visual Studio when you have the corresponding GitLab extension installed: https://docs.gitlab.com/ee/user/project/repository/code_suggestions.html#use-code-suggestions https://docs.gitlab.com/ee/user/project/repository/code_sugg... We offer experimental support for additional editors: https://docs.gitlab.com/ee/user/project/repository/code_suggestions.html#enable-code-suggestions-in-other-ides-and-editors https://docs.gitlab.com/ee/user/project/repository/code_sugg...
- lolinder 3y ago
  That doesn't answer the question of what the pricing will be. I'm assuming it's not going to be free. Will it be a standalone subscription, or another thing that gets bundled into the main tiers to justify the ever-increasing prices?
  - john_cogs 3y ago
    Code Suggestions is free while in Beta. When GA, it will be included in our $9 per user per month AI add-on.
ollieglass 3y ago
Seems to be $99 per month. Signing up for the free trial funnels me to the trial of Gitlab Ultimate. So assuming that you need an Ultimate subscription to use it after the trial, that's the price. Pricing is here https://about.gitlab.com/pricing/ https://about.gitlab.com/pricing/ In contrast, Copilot is $10 per month https://github.com/features/copilot#pricing https://github.com/features/copilot#pricing

deleted 3y ago

[deleted]

ldehaan 3y ago

[dead]

smallpipe 3y ago

What nimrod added a chatbot to this page ? It’s downright unusable on mobile, it just keeps popping up with shit. Even closing it creates two more notifications that use up two thirds of the screen

lee101 3y ago

[dead]

HdS84 3y ago

I get that gitlab needs more features to continue to sell. But honestly for me it reduces the attractiveness of gitlab. Just adding half working features does not increase the value for me. Please finish and polish the existing Features before adding hype feature 93939. Maybe add a policy that your open feature and big count should be less than 20K or something. Examples: Security scanning has a god awful UI and UX. The whole project management stuff is barely usable Pipelines have loads of weird half features and missing QOL. E G. Adding saeif support instead of just your own warnings format.

lvncelot 3y ago
It's honestly doubly annoying when there's multiple times per month where I run into some issue / straight up bug which I then find an ongoing bug ticket from like 2018 for, that's still open but apparently not being worked on. I really like Gitlab, I think the CI experience is a lot better than, for instance, Github Actions, but I also don't ever see me using a good chunk of the stuff that's been shoveled on in the last few years and I don't really know anyone that does.
- HdS84 3y ago
  At least for python and dotnet, I think github actions are superior. Gitlab did not support poetry for ages there where bugs with python galore and for dotnet they do not speak the same language - e.g. issues in dotnet are reported as saeif, but gl has it own weird format. And yes, you are totally right with the bug tracker. It's often funny how long the ticket is just because it was retagged sooo many times.

tallytarik 3y ago

GitLab was rather quick to board the AI hype train this year. Fine - they don't want to look like they're behind the curve, have to please investors, and whatever else. But IMO there are plenty of other places to add real value across the GitLab product with AI/ML features. Here, it just looks like they saw GitHub do something and felt a need to copy it. But two years late, and worse. As a longtime GitLab user (and onetime contributor!), I'm a bit disappointed they're spending so much time on this. I think they're just too far behind.

dnsmichi 3y ago
GitLab team member here. Thanks for your feedback. > But IMO there are plenty of other places to add real value across the GitLab product with AI/ML features. True, and after starting with ML experiments, the product and engineering teams have been working on new features for entire DevOps lifecycle. All AI workflows on the DevSecOps platforms are described in the GitLab Duo announcement blog post https://about.gitlab.com/blog/2023/06/22/meet-gitlab-duo-the-suite-of-ai-capabilities/ https://about.gitlab.com/blog/2023/06/22/meet-gitlab-duo-the... and website https://about.gitlab.com/gitlab-duo/ https://about.gitlab.com/gitlab-duo/ I'll share a few highlights that I am personally excited about - Explain and help fix security vulnerabilities. From my personal experience, I often find CVEs hard to read, especially when I am not the author of the code to fix. Getting help from AI can reduce entry barriers and make development for efficient. Security is everyone's responsibility these days. This follows the AI assisted feature to explain code in general. "What does this magic loop with memcpy do?" might not stay magic anymore, easing the path to code refactoring, improving performance, and reduce the resource usage footprint. - Summarize issue comments. Feature proposals or bug analysis can have long comment threads that require reading time. AI will help get the gist and better contribute to what has been discussed. - Summarize MR changes, to avoid reading long change diffs. This helps with faster (code) review cycles. I tested it this week with an MR for our handbook in https://gitlab.com/gitlab-com/www-gitlab-com/-/merge_requests/126426#note_1449388720 https://gitlab.com/gitlab-com/www-gitlab-com/-/merge_request... I'd also like to see AI helping fix CI/CD pipelines fast. Proposal in https://gitlab.com/gitlab-org/gitlab/-/issues/386863 https://gitlab.com/gitlab-org/gitlab/-/issues/386863 I shared some thoughts in a new talk "Observability for Efficient DevSecOps Pipelines", slides in https://go.gitlab.com/VDAvMw https://go.gitlab.com/VDAvMw (GitLab blog post coming soon, https://gitlab.com/gitlab-com/www-gitlab-com/-/issues/34296 https://gitlab.com/gitlab-com/www-gitlab-com/-/issues/34296) Additionally, I learned some new ideas at Cloudland last week, regarding product owner requirements list verification, and end-to-end test automation with AI. Need to create feature proposals :-) > As a longtime GitLab user (and onetime contributor!), Thanks for contributing. I'd like to invite you to share your ideas about AI features across the platform :) When you look at the DevOps lifecycle (image in https://about.gitlab.com/gitlab-duo/ https://about.gitlab.com/gitlab-duo/) from plan/manage to create, verify, secure, package, release, deploy, monitor, govern - where do you see yourself, and where do you spend the most time in? Second question: Which process feels the most inefficient? After identifying answers to the questions, please check the AI features https://docs.gitlab.com/ee/user/ai_features.html https://docs.gitlab.com/ee/user/ai_features.html and/or open new feature proposals for GitLab https://gitlab.com/gitlab-org/gitlab/-/issues/new?issuable_template=Feature%20Proposal%20-%20lean https://gitlab.com/gitlab-org/gitlab/-/issues/new?issuable_t... You can tag @dnsmichi so I can engage with your ideas. Thanks!

bilekas 3y ago

It seems I need to have a company to sign up for the trial ? I have a free gitlab account but can't proceed without entering business information as it seems to be tied to the gitlab premium feature set. Am I doing something wrong ?

moonchrome 3y ago

Funny how even the examples they have in the showcase are bad/mediocre code quality.

al_be_back 3y ago

Their service calls Google Vertex AI Codey APIs to generate suggestions [1]. It's fine to integrate Off-the-shelf solutions, but you don't have to implement every tech-hype going viral. I fear AI-assistant features are pushing more prescient features further down the backlog. [1] https://docs.gitlab.com/ee/user/project/repository/code_suggestions.html#code-suggestions-data-usage https://docs.gitlab.com/ee/user/project/repository/code_sugg...