10 ms·

Show HN: Use Third Party LLM API in JetBrains AI Assistant

102 points by Stream 1y ago

yekanchi 1y ago
does not gpt4free [g4all] do the same?
- Stream 1y ago
  The problem is that JetBrains AI Assistant only support Ollama and LM Studio as model provider for now.
aghilmort 1y ago
1. this is cool / innovative 2. had lots of fun with similar we should chain / tunnel this to this it can’t be done easily projects 3. fond memories reverse engineering JNI 1.0 to let Java only devs use C libs, get C# 1.0 to pipe via C++ to call C serial port driver for early drone microcontroller etc all to say / nice work / keep building / like that it solves your very specific AI anywhere problem in JetBrains even tho it seems like wait where to even start! It can’t be done! Like Microsoft once told me re: C# 1.0
- tymscar 1y ago
  Projects like this are at the heart of what it means to be a hacker. Its extremely fun to have a clear requirement that is not supported by a piece of software that you use and then work around it until it works
- pjmlp 1y ago
  Given that Managed C++ was part of .NET 1.0, replaced with C++/CLI with .NET 2.0, and one of the first things I did with .NET was a bindings library for a RPC framework written in C, via Managed C++, I wonder what Microsoft was telling you it cannot be done.
  - aghilmort 1y ago
    we had to call a serial port and according to MSFT compiler team, C# 1.0 (it was basically the first public release off C#) it couldn't talk to C control library we had to use from vendor, so somehow hacked a C# to C++ to C thing and back again at the time. Then they were like oh, ok ya, that could work after it was done timeframe was circa 2002-2004 give or take, the C lib we had to use was closed source
ulrikrasmussen 1y ago
I tried using the AI assistant when it came out but seemingly was too stupid to figure out how to use it correctly. I tried to get it to write single functions or short blocks of code for me, but it would always start rewriting the whole file from scratch which was way too slow. Has it gotten better since, or did I just do it wrong from the beginning?
- glzone1 1y ago
  The jetbrains ai assistant was truly terrible when it came out. They blocked the use of the major LLM’s and had supposedly a better European (?) LLM for coding, but I could not get it to work in any reasonable way. Now though they do allow you to use the major LLMs I believe and so it’s immediately much more useful. There is also Junie, which I’m not sure how that compares it’s a bit confusing but also seems better.
  - t-writescode 1y ago
    Recall that when LLMs were first coming out, there was a very loud and aggressive stance *against* using the standard models. Their choice of which LLM to use is probably related to those earlier, more cautious stances.
- rickette 1y ago
  It has gotten a lot better IMHO in the latest 2025 release. Ability to enable/disable per project. Ability to favor local models over online models (online as fallback), support for OpenAI, Claude, Gemini, etc. Beter integration overall.
- ta988 1y ago
  I have used Junie their new system quite a lot recently it is much much better. It solved problems that Claude code was struggling with (but couldn't solve some that Claude code did solve with max thinking). It is still slowish, but the quality is definitely getting there.
- surgical_fire 1y ago
  I have started to use it recently, and I am finding it useful as a code assistant. 1) It has been doing a good job reviewing my code, and could suggest some interesting performance improvements in the past. 2) It is good to generate things such as simple REST endooints, that sort of thing. Saves you a bunch of typing, even if you have to do some finishing touches yourself. 3) I think it generally does a decent job writing tests as well. 4) Current models have gotten really good at helping me explore libraries I am not very familiar with. The AI hype may be very annoying, and I am skeptical that in the long run it makes financial sense, but it is a productivity boost once you get the hang of it.
roandepoan 1y ago
There’s also https://github.com/xsharov/enchanted-ollama-openrouter-proxy https://github.com/xsharov/enchanted-ollama-openrouter-proxy
omneity 1y ago
Congrats for shipping! Have you seen LiteLLM Gateway before? As a bonus it supports a lot of different providers. https://docs.litellm.ai/docs/simple_proxy https://docs.litellm.ai/docs/simple_proxy
- Stream 1y ago
  Yes. Because my original purpose is to use third party LLMs in JetBrains AI Assistant, it doesn't provide Ollama or LM Studio endpoints as required. So I decide to develop one on myself.
bambax 1y ago
Neat! But instead of hitting each provider's API separately, managing their keys, etc., it may be easier to go through OpenRouter, which is a single endpoint, has hundreds of models, and doesn't add any cost on top of public prices from the providers.
- Lorean1 1y ago
  If they don't add any cost I wonder what exactly is their business model?
  - joshstrange 1y ago
    They charge you a service fee (5% + $0.35) when you reload credits. This is very well hidden and not publicly stated (or at least I couldn’t find it would logging in and trying to purchase).
  - bambax 1y ago
    They say they buy in bulk and get better prices than those available to the general public... IDK if that's the whole story, but could be.
- joshstrange 1y ago
  It adds 5% + $0.35 every time you reload, so it absolutely does add a cost on top of public prices.
  - tecleandor 1y ago
    What does 'reload' mean in this context? (haven't used OpenRouter)
    - joshstrange 1y ago
      You buy credits on OpenRouter, or rather you load money onto your account and then as you make API calls your account balance is deducted by the cost displayed on OpenRouter's website (market rates). So once money is loaded on your account you pay the same as if you used the LLM provider directly but you pay a premium to load money onto your account, it's effectively adding a 5% fee to the base LLM API prices (and $0.35/per reload, depending on reload amount this might be negligible but on a $10 reload it means you are effectively paying a 9% fee).
nullchan 1y ago
I literally was working on something similar. Emphasis on `was` since you beat me to it! :D
bionhoward 1y ago
One thing I thought was funny was remembering these AI services all have the customer noncompete clause and this morning realizing they can actually sue you (not just rug pull you, actually bankrupt you with legal fees) if you use em to develop competing stuff Does that imply using OpenAI/Anthropic/Gemini API/Grok/Copilot for anything but the most trivial jokish personal uses ought to be considered a serious breach of fiduciary duty because it exposes your business to potentially massive legal costs?
- ta988 1y ago
  Can you direct me to where you see that in the Jetbrains agreements? Same for the other services.
NoelJacob 1y ago
Wouldn't an extension be more suitable that runs on Jetbrains boot?
- Stream 1y ago
  Yes. I;m considering working on it. But a standalone version first.
linotype 1y ago
JetBrains really needs to support alternatives fast. Cursor, CoPilot, etc all need much better integration. VS Code is becoming almost mandatory for companies that force the use of LLM tools.
- ivape 1y ago
  Really? This stuff is mandatory for those who are dependent on it. Many people still just look at the code in a separate browser tab and make a meaningful decision of if they want to pull something in or not. Why even use an IDE is the better question. What exactly is an IDE for if the AI will search and determine where to add/remove code? I don't think Jetbrains needs to lose focus on their product to compete with products that are masquerading as an IDE but have no intention of fulfilling the role of an IDE. They want to re-invent it, so it's better they simply stay in their own lane as they will be cannibalized by their own genre as there are no rules in that wild-west. For example, why does my automatic code writing genie look like a fucking IDE? Is there a reason? Cursor and the like will have their own competition and they'll struggle to keep up with a blank screen that just magically outputs software with no visibility into the code. Then they'll be stuck trying to sell people why it's better to let magic be written inside of an IDE, both competing with new magic and old hard tools (like Jetbrains). Cursor is not the future, it's dead on arrival. Jetbrains and existing IDEs are much more focused for exactly what it is, manual code writing/inspection. How they integrate AI should be done meaningfully to improve the IDE experience. If they want to be in the business of reinventing code-writing, then you have to scrap the IDE altogether.
  - kurtis_reed 1y ago
    Huh?
  - sillysaurusx 1y ago
    That seems like a false dichotomy. AI doesn’t decide what kind of code to write. That’s still up to the human (for now). Jetbrains also has to have some kind of AI offering, if only because otherwise users will migrate to competitors that do. It’s an interesting question whether we’ll need an IDE at all in the future. I used to program in Vim with minimal help. Maybe one day I’ll program in Vim with maximal help from an AI. It would be nice to say "find where authentication happens" and it pulls up all instances of auth code. But that’s also another example of a feature that can go into an IDE: case sensitive search, regex search, natural language search. So I’m skeptical we won’t be using IDEs in the future. And if all of this is a part of an IDE, then that’s squarely in Jetbrains’ wheelhouse.
christkv 1y ago
What’s considered the best small local model for coding? Something that can run on an m1 max 32gb ram?

102 points by Stream 1y ago

yekanchi 1y ago

does not gpt4free [g4all] do the same?

Stream 1y ago
The problem is that JetBrains AI Assistant only support Ollama and LM Studio as model provider for now.

aghilmort 1y ago

1. this is cool / innovative 2. had lots of fun with similar we should chain / tunnel this to this it can’t be done easily projects 3. fond memories reverse engineering JNI 1.0 to let Java only devs use C libs, get C# 1.0 to pipe via C++ to call C serial port driver for early drone microcontroller etc all to say / nice work / keep building / like that it solves your very specific AI anywhere problem in JetBrains even tho it seems like wait where to even start! It can’t be done! Like Microsoft once told me re: C# 1.0

tymscar 1y ago
Projects like this are at the heart of what it means to be a hacker. Its extremely fun to have a clear requirement that is not supported by a piece of software that you use and then work around it until it works
pjmlp 1y ago
Given that Managed C++ was part of .NET 1.0, replaced with C++/CLI with .NET 2.0, and one of the first things I did with .NET was a bindings library for a RPC framework written in C, via Managed C++, I wonder what Microsoft was telling you it cannot be done.
- aghilmort 1y ago
  we had to call a serial port and according to MSFT compiler team, C# 1.0 (it was basically the first public release off C#) it couldn't talk to C control library we had to use from vendor, so somehow hacked a C# to C++ to C thing and back again at the time. Then they were like oh, ok ya, that could work after it was done timeframe was circa 2002-2004 give or take, the C lib we had to use was closed source

ulrikrasmussen 1y ago

I tried using the AI assistant when it came out but seemingly was too stupid to figure out how to use it correctly. I tried to get it to write single functions or short blocks of code for me, but it would always start rewriting the whole file from scratch which was way too slow. Has it gotten better since, or did I just do it wrong from the beginning?

glzone1 1y ago
The jetbrains ai assistant was truly terrible when it came out. They blocked the use of the major LLM’s and had supposedly a better European (?) LLM for coding, but I could not get it to work in any reasonable way. Now though they do allow you to use the major LLMs I believe and so it’s immediately much more useful. There is also Junie, which I’m not sure how that compares it’s a bit confusing but also seems better.
- t-writescode 1y ago
  Recall that when LLMs were first coming out, there was a very loud and aggressive stance *against* using the standard models. Their choice of which LLM to use is probably related to those earlier, more cautious stances.
rickette 1y ago
It has gotten a lot better IMHO in the latest 2025 release. Ability to enable/disable per project. Ability to favor local models over online models (online as fallback), support for OpenAI, Claude, Gemini, etc. Beter integration overall.
ta988 1y ago
I have used Junie their new system quite a lot recently it is much much better. It solved problems that Claude code was struggling with (but couldn't solve some that Claude code did solve with max thinking). It is still slowish, but the quality is definitely getting there.
surgical_fire 1y ago
I have started to use it recently, and I am finding it useful as a code assistant. 1) It has been doing a good job reviewing my code, and could suggest some interesting performance improvements in the past. 2) It is good to generate things such as simple REST endooints, that sort of thing. Saves you a bunch of typing, even if you have to do some finishing touches yourself. 3) I think it generally does a decent job writing tests as well. 4) Current models have gotten really good at helping me explore libraries I am not very familiar with. The AI hype may be very annoying, and I am skeptical that in the long run it makes financial sense, but it is a productivity boost once you get the hang of it.

roandepoan 1y ago

There’s also https://github.com/xsharov/enchanted-ollama-openrouter-proxy https://github.com/xsharov/enchanted-ollama-openrouter-proxy

omneity 1y ago

Congrats for shipping! Have you seen LiteLLM Gateway before? As a bonus it supports a lot of different providers. https://docs.litellm.ai/docs/simple_proxy https://docs.litellm.ai/docs/simple_proxy

Stream 1y ago
Yes. Because my original purpose is to use third party LLMs in JetBrains AI Assistant, it doesn't provide Ollama or LM Studio endpoints as required. So I decide to develop one on myself.

bambax 1y ago

Neat! But instead of hitting each provider's API separately, managing their keys, etc., it may be easier to go through OpenRouter, which is a single endpoint, has hundreds of models, and doesn't add any cost on top of public prices from the providers.

Lorean1 1y ago
If they don't add any cost I wonder what exactly is their business model?
- joshstrange 1y ago
  They charge you a service fee (5% + $0.35) when you reload credits. This is very well hidden and not publicly stated (or at least I couldn’t find it would logging in and trying to purchase).
- bambax 1y ago
  They say they buy in bulk and get better prices than those available to the general public... IDK if that's the whole story, but could be.
joshstrange 1y ago
It adds 5% + $0.35 every time you reload, so it absolutely does add a cost on top of public prices.
- tecleandor 1y ago
  What does 'reload' mean in this context? (haven't used OpenRouter)
  - joshstrange 1y ago
    You buy credits on OpenRouter, or rather you load money onto your account and then as you make API calls your account balance is deducted by the cost displayed on OpenRouter's website (market rates). So once money is loaded on your account you pay the same as if you used the LLM provider directly but you pay a premium to load money onto your account, it's effectively adding a 5% fee to the base LLM API prices (and $0.35/per reload, depending on reload amount this might be negligible but on a $10 reload it means you are effectively paying a 9% fee).

nullchan 1y ago

I literally was working on something similar. Emphasis on `was` since you beat me to it! :D

bionhoward 1y ago

One thing I thought was funny was remembering these AI services all have the customer noncompete clause and this morning realizing they can actually sue you (not just rug pull you, actually bankrupt you with legal fees) if you use em to develop competing stuff Does that imply using OpenAI/Anthropic/Gemini API/Grok/Copilot for anything but the most trivial jokish personal uses ought to be considered a serious breach of fiduciary duty because it exposes your business to potentially massive legal costs?

ta988 1y ago
Can you direct me to where you see that in the Jetbrains agreements? Same for the other services.

NoelJacob 1y ago

Wouldn't an extension be more suitable that runs on Jetbrains boot?

Stream 1y ago
Yes. I;m considering working on it. But a standalone version first.

linotype 1y ago

JetBrains really needs to support alternatives fast. Cursor, CoPilot, etc all need much better integration. VS Code is becoming almost mandatory for companies that force the use of LLM tools.

ivape 1y ago
Really? This stuff is mandatory for those who are dependent on it. Many people still just look at the code in a separate browser tab and make a meaningful decision of if they want to pull something in or not. Why even use an IDE is the better question. What exactly is an IDE for if the AI will search and determine where to add/remove code? I don't think Jetbrains needs to lose focus on their product to compete with products that are masquerading as an IDE but have no intention of fulfilling the role of an IDE. They want to re-invent it, so it's better they simply stay in their own lane as they will be cannibalized by their own genre as there are no rules in that wild-west. For example, why does my automatic code writing genie look like a fucking IDE? Is there a reason? Cursor and the like will have their own competition and they'll struggle to keep up with a blank screen that just magically outputs software with no visibility into the code. Then they'll be stuck trying to sell people why it's better to let magic be written inside of an IDE, both competing with new magic and old hard tools (like Jetbrains). Cursor is not the future, it's dead on arrival. Jetbrains and existing IDEs are much more focused for exactly what it is, manual code writing/inspection. How they integrate AI should be done meaningfully to improve the IDE experience. If they want to be in the business of reinventing code-writing, then you have to scrap the IDE altogether.
- kurtis_reed 1y ago
  Huh?
- sillysaurusx 1y ago
  That seems like a false dichotomy. AI doesn’t decide what kind of code to write. That’s still up to the human (for now). Jetbrains also has to have some kind of AI offering, if only because otherwise users will migrate to competitors that do. It’s an interesting question whether we’ll need an IDE at all in the future. I used to program in Vim with minimal help. Maybe one day I’ll program in Vim with maximal help from an AI. It would be nice to say "find where authentication happens" and it pulls up all instances of auth code. But that’s also another example of a feature that can go into an IDE: case sensitive search, regex search, natural language search. So I’m skeptical we won’t be using IDEs in the future. And if all of this is a part of an IDE, then that’s squarely in Jetbrains’ wheelhouse.

christkv 1y ago

What’s considered the best small local model for coding? Something that can run on an m1 max 32gb ram?