4 ms·

Is anyone else here building AI programming services based on models like this? I see a lot of comments saying the models can't do much programming. But I just

by ilaksh 4y ago

Is anyone else here building AI programming services based on models like this? I see a lot of comments saying the models can't do much programming. But I just suspect there must be a silent contingent that is also working on services like that. And maybe less likely to promote the abilities of these models because it encourages competition.

aunch 4y ago
We are at Codeium (codeium.com)! Not the SantaCoder model specifically, but the same types of LLM architectures. We've started with AI-based code autocomplete, but we think there is a lot more we can do. We wrote up some of our learnings so far in @swyx's blog recently: https://lspace.swyx.io/p/what-building-copilot-for-x-really https://lspace.swyx.io/p/what-building-copilot-for-x-really
- furyofantares 4y ago
  What I would really like is something I saw someone talking about here; I'd like the editor to brighten text it finds "unexpected" which could immediately alert to bugs, or to the fact that the code I'm writing looks weird in some way and might either be restructured or accompanied by a comment.
  - aunch 4y ago
    Yep, these kinds of applications are on our mind! We consider autocomplete to be the "baseline" task since there are plenty of benchmarks and research to compare our model's performance to, but there's lots of things like highlighting code, upgrading to new libraries/conventions, etc that we can do with a good base model.
- hooande 4y ago
  my unsolicited advice: pick an X. what is the one best use case for this other than code? law? finance? focus on that vertical. if you have no idea what that could be or if that market is too small, you're already in trouble. I don't use anticomplete at all. What I would like is something that can take my current, bad code and style transfer it into proper, modern code. best case, take code as I write it naturally and confirm it to the style guide of my organization.
videlov 4y ago
We built a semantic code search CLI tool (fully local and open source) using a similar model that I tuned https://github.com/sturdy-dev/semantic-code-search https://github.com/sturdy-dev/semantic-code-search
notwokeno 4y ago
I've been messing around some. Flan-T5 generates surprisingly close stuff occasionally for simple prompts like #square x or #sum the elements in the list.
morgante 4y ago
We're building tools like this at Grit: https://www.grit.io/ https://www.grit.io/ These kinds of models are particularly good at repetitive, boring work like refactoring legacy code and completing framework migrations. Unlike Copilot, we've specialized specifically in these areas and completing them end-to-end (instead of just sitting in the IDE, we open already-verified PRs).
- ilaksh 4y ago
  May I ask what model you are using?
  - morgante 4y ago
    We use a few depending on the task (Codex, fine-tuned T5, Bert models, etc.). Constantly experimenting with different variations. Since we focus on solving narrower problems in more depth, it leaves more room for optimizing accuracy.
    - ilaksh 4y ago
      Have you been able to get your rate limit increased for code-davinci-002? It defaults to a very small amount.
      - morgante 4y ago
        I recommend trying davinci-003 as well.
        ilaksh 4y ago
        Yeah, been messing with that a lot.
tluyben2 4y ago
Yes, we are building something that is somewhat like ILP/IFP (and other tried and tested but non scalable techniques) with the search space reduced by using modern ML language models. And indeed; the thing that works best in our system has not been done in the open yet. Of course we have no idea if it's viable for the masses; maybe if people see how well it works.
lmeyerov 4y ago
Yes we are incorporating into Graphistry as part of how we help sec/fraud/misinfo/crime/etc analyst teams investigate their data. Our platform does all sorts of GPU visual graph analytics & graph AI once data gets loaded in, and as part of our visual playbooks automation layer, this helps users make automations and fancier queries. Think Splunk, Spark, Neo4j, ... . IMO tough question of who can do codegen as a scalable standalone startup, but that's ok. Pretty darn easy & useful for many productivity platforms like ours where it's just a super nice feature as part of delivering a broader magical experience. Related: we are hiring a k8s/pydata person, ideally who has need a user & builder of investigation platforms, as we are working w co's like Nvidia to bring this kind of thing to some pretty major enterprise & gov teams. See gdoc linked on our careers page.
pryelluw 4y ago
Yup, as part of another thing. ML assisted everything is here to stay.
jillesvangurp 4y ago
I've been pretty impressed with chat gpt generating working implementations of various algorithms in different languages. Crucially, it actually knows about algorithms. I was trying to get it to generate some algorithm for calculating concave hulls the other day and ended up learning a thing or two about various algorithms for that in this space. Almost but not quite worked for my use case. It seems limited in the amount of code it can generate in one go. But otherwise, I was pretty impressed. So, we're not that far off from basically pair programming with an AI that will do most of the boring/tedious work we currently do manually. Something like chat gpt integrated into an IDE could be useful right now.
- ilaksh 4y ago
  Copilot or Codium
thakkarparth007 4y ago
There's replit. Constantly announcing new features around such models. They'd introduced "ghostwriter" a while back and yesterday or so they announced ghostwriter chat.

by ilaksh 4y ago

aunch 4y ago

We are at Codeium (codeium.com)! Not the SantaCoder model specifically, but the same types of LLM architectures. We've started with AI-based code autocomplete, but we think there is a lot more we can do. We wrote up some of our learnings so far in @swyx's blog recently: https://lspace.swyx.io/p/what-building-copilot-for-x-really https://lspace.swyx.io/p/what-building-copilot-for-x-really

furyofantares 4y ago
What I would really like is something I saw someone talking about here; I'd like the editor to brighten text it finds "unexpected" which could immediately alert to bugs, or to the fact that the code I'm writing looks weird in some way and might either be restructured or accompanied by a comment.
- aunch 4y ago
  Yep, these kinds of applications are on our mind! We consider autocomplete to be the "baseline" task since there are plenty of benchmarks and research to compare our model's performance to, but there's lots of things like highlighting code, upgrading to new libraries/conventions, etc that we can do with a good base model.
hooande 4y ago
my unsolicited advice: pick an X. what is the one best use case for this other than code? law? finance? focus on that vertical. if you have no idea what that could be or if that market is too small, you're already in trouble. I don't use anticomplete at all. What I would like is something that can take my current, bad code and style transfer it into proper, modern code. best case, take code as I write it naturally and confirm it to the style guide of my organization.

videlov 4y ago

We built a semantic code search CLI tool (fully local and open source) using a similar model that I tuned https://github.com/sturdy-dev/semantic-code-search https://github.com/sturdy-dev/semantic-code-search

notwokeno 4y ago

I've been messing around some. Flan-T5 generates surprisingly close stuff occasionally for simple prompts like #square x or #sum the elements in the list.

morgante 4y ago

We're building tools like this at Grit: https://www.grit.io/ https://www.grit.io/ These kinds of models are particularly good at repetitive, boring work like refactoring legacy code and completing framework migrations. Unlike Copilot, we've specialized specifically in these areas and completing them end-to-end (instead of just sitting in the IDE, we open already-verified PRs).

ilaksh 4y ago
May I ask what model you are using?
- morgante 4y ago
  We use a few depending on the task (Codex, fine-tuned T5, Bert models, etc.). Constantly experimenting with different variations. Since we focus on solving narrower problems in more depth, it leaves more room for optimizing accuracy.
  - ilaksh 4y ago
    Have you been able to get your rate limit increased for code-davinci-002? It defaults to a very small amount.
    - morgante 4y ago
      I recommend trying davinci-003 as well.
      - ilaksh 4y ago
        Yeah, been messing with that a lot.

tluyben2 4y ago

Yes, we are building something that is somewhat like ILP/IFP (and other tried and tested but non scalable techniques) with the search space reduced by using modern ML language models. And indeed; the thing that works best in our system has not been done in the open yet. Of course we have no idea if it's viable for the masses; maybe if people see how well it works.

lmeyerov 4y ago

Yes we are incorporating into Graphistry as part of how we help sec/fraud/misinfo/crime/etc analyst teams investigate their data. Our platform does all sorts of GPU visual graph analytics & graph AI once data gets loaded in, and as part of our visual playbooks automation layer, this helps users make automations and fancier queries. Think Splunk, Spark, Neo4j, ... . IMO tough question of who can do codegen as a scalable standalone startup, but that's ok. Pretty darn easy & useful for many productivity platforms like ours where it's just a super nice feature as part of delivering a broader magical experience. Related: we are hiring a k8s/pydata person, ideally who has need a user & builder of investigation platforms, as we are working w co's like Nvidia to bring this kind of thing to some pretty major enterprise & gov teams. See gdoc linked on our careers page.

pryelluw 4y ago

Yup, as part of another thing. ML assisted everything is here to stay.

jillesvangurp 4y ago

I've been pretty impressed with chat gpt generating working implementations of various algorithms in different languages. Crucially, it actually knows about algorithms. I was trying to get it to generate some algorithm for calculating concave hulls the other day and ended up learning a thing or two about various algorithms for that in this space. Almost but not quite worked for my use case. It seems limited in the amount of code it can generate in one go. But otherwise, I was pretty impressed. So, we're not that far off from basically pair programming with an AI that will do most of the boring/tedious work we currently do manually. Something like chat gpt integrated into an IDE could be useful right now.

ilaksh 4y ago
Copilot or Codium

thakkarparth007 4y ago

There's replit. Constantly announcing new features around such models. They'd introduced "ghostwriter" a while back and yesterday or so they announced ghostwriter chat.