3 ms·

RubyLLM author here. I'm not sure where you got that. `chat.with_temperature(0.2)` https://rubyllm.com/chat/#controlling-response-behavior https://rubyllm.co

by earcar 3mo ago

RubyLLM author here. I'm not sure where you got that. `chat.with_temperature(0.2)` https://rubyllm.com/chat/#controlling-response-behavior https://rubyllm.com/chat/#controlling-response-behavior `chat.with_thinking(effort: :high, budget: 8000)` https://rubyllm.com/thinking/#controlling-extended-thinking https://rubyllm.com/thinking/#controlling-extended-thinking Max tokens is the only one of your list that require provider specific params: https://rubyllm.com/chat/#provider-specific-parameters https://rubyllm.com/chat/#provider-specific-parameters I'm one guy doing it for free. Happy to see your contribution!

techscruggs 3mo ago
And thank you! It is absolutely awesome and a true joy to work with.
mosselman 3mo ago
Hi! Valid challenge, I am probably misremembering. We were playing with various 'one-interface to all providers' solutions and I might have mixed up RubyLLM there. Sorry for that. I will have a deep dive into which things I felt we needed to adapt per provider. I didn't mean to imply that you have to solve all of our wants of course. One thing we did do was monkey-patch the spot where tool_calls are performed by RubyLLM. We had our own mechanism for that and were able to skip RubyLLM's and still extract the tool calls and run them through our own tool harness. That all worked beautifully. I don't know if that type of stuff is something you want PRs on or that you want to keep steering towards the route that does everything within RubyLLM classes. Happy to contribute some of that.
- earcar 3mo ago
  Interesting! What were you guys trying to achieve by running them in your own tool harness?
  - mosselman 3mo ago
    We had already implemented tool_calls in our own database and have a system that executes them and creates our conversation array, etc. So we wanted to leverage the providers that RubyLLM supports without having to change the tool execution in our platform.
- mosselman 3mo ago
  Update since the side thread blew up. There is some difference in how OpenAI and Anthropic handle 'max_tokens'. The OpenAI way raises errors in Anthropic for example. I will look at confirming this a bit more in depth once we've taken our RubyLLM based adapter in production and see if I can make some contribution. Thanks for all the work! It is incredibly impressive and I never meant any disrespect.
realty_geek 3mo ago
Well put!!! You work your arse off for free and the guy who made the disparaging comment didn't even bother to research to see if he had the details right. Hat's off to you Carmine for all your work. Many people really do appreciate it.
- jaredsohn 3mo ago
  FWIW, I don't think the GP post is disparaging (at least as I read it right now.) I think it is fair to list limitations from using a library that provides an abstraction; it can suggest why a tool isn't right for a person's use cases. But it also sounds like this API handles those pretty well.
  - katzgrau 3mo ago
    The issue is that it’s relatively low effort to make false and unverified claims. Defending and refuting it is a much higher effort task for the person doing the work to everyone else’s benefit. RubyLLM dev literally had to take time to provide code samples and doc links. No issue with listing legit limitations, but be a bro and fact check claims before wasting a volunteer’s time - and potentially leading other developers on a public board astray.
    - jaredsohn 3mo ago
      I completely agree that unverified claims create a heavy burden for maintainers. My only point was about the language used: 'disparaging' to me implies a bad-faith attack or a dismissive attitude, whereas this was just an honest technical mix-up that the poster immediately corrected. I think part of the confusion with that word comes from things like corporate non-disparagement clauses. In those contracts, lawyers write the terms so broad that "disparagement" means saying anything negative, regardless of malice or intent.
      - mosselman 3mo ago
        Thanks for the discourse. I never meant it to be disparagement nor do I think it really was. I checked and it turns out I remembered correctly that setting effort and some of its settings are not portable between providers. There are some different settings that each provider uses and in order for it to be portable, you have to force some defaults on provider A when using a setting that is almost only supported in provider B. In our implementation we decided to drop a certain setting when using OpenAI in one case and we decided we can just force some other setting when using Anthropic. But this 'solution', might not be what others expect. When you build an open-source library you can go this opinionated route and force these settings, or you might go the config route and force people to explicitly handle per-provider differences. I will have a look at what I am able to do in terms of a contribution and then in the PR Carmine can decide what they like.

by earcar 3mo ago

techscruggs 3mo ago

And thank you! It is absolutely awesome and a true joy to work with.

mosselman 3mo ago

Hi! Valid challenge, I am probably misremembering. We were playing with various 'one-interface to all providers' solutions and I might have mixed up RubyLLM there. Sorry for that. I will have a deep dive into which things I felt we needed to adapt per provider. I didn't mean to imply that you have to solve all of our wants of course. One thing we did do was monkey-patch the spot where tool_calls are performed by RubyLLM. We had our own mechanism for that and were able to skip RubyLLM's and still extract the tool calls and run them through our own tool harness. That all worked beautifully. I don't know if that type of stuff is something you want PRs on or that you want to keep steering towards the route that does everything within RubyLLM classes. Happy to contribute some of that.

earcar 3mo ago
Interesting! What were you guys trying to achieve by running them in your own tool harness?
- mosselman 3mo ago
  We had already implemented tool_calls in our own database and have a system that executes them and creates our conversation array, etc. So we wanted to leverage the providers that RubyLLM supports without having to change the tool execution in our platform.
mosselman 3mo ago
Update since the side thread blew up. There is some difference in how OpenAI and Anthropic handle 'max_tokens'. The OpenAI way raises errors in Anthropic for example. I will look at confirming this a bit more in depth once we've taken our RubyLLM based adapter in production and see if I can make some contribution. Thanks for all the work! It is incredibly impressive and I never meant any disrespect.

realty_geek 3mo ago

Well put!!! You work your arse off for free and the guy who made the disparaging comment didn't even bother to research to see if he had the details right. Hat's off to you Carmine for all your work. Many people really do appreciate it.

jaredsohn 3mo ago
FWIW, I don't think the GP post is disparaging (at least as I read it right now.) I think it is fair to list limitations from using a library that provides an abstraction; it can suggest why a tool isn't right for a person's use cases. But it also sounds like this API handles those pretty well.
- katzgrau 3mo ago
  The issue is that it’s relatively low effort to make false and unverified claims. Defending and refuting it is a much higher effort task for the person doing the work to everyone else’s benefit. RubyLLM dev literally had to take time to provide code samples and doc links. No issue with listing legit limitations, but be a bro and fact check claims before wasting a volunteer’s time - and potentially leading other developers on a public board astray.
  - jaredsohn 3mo ago
    I completely agree that unverified claims create a heavy burden for maintainers. My only point was about the language used: 'disparaging' to me implies a bad-faith attack or a dismissive attitude, whereas this was just an honest technical mix-up that the poster immediately corrected. I think part of the confusion with that word comes from things like corporate non-disparagement clauses. In those contracts, lawyers write the terms so broad that "disparagement" means saying anything negative, regardless of malice or intent.
    - mosselman 3mo ago
      Thanks for the discourse. I never meant it to be disparagement nor do I think it really was. I checked and it turns out I remembered correctly that setting effort and some of its settings are not portable between providers. There are some different settings that each provider uses and in order for it to be portable, you have to force some defaults on provider A when using a setting that is almost only supported in provider B. In our implementation we decided to drop a certain setting when using OpenAI in one case and we decided we can just force some other setting when using Anthropic. But this 'solution', might not be what others expect. When you build an open-source library you can go this opinionated route and force these settings, or you might go the config route and force people to explicitly handle per-provider differences. I will have a look at what I am able to do in terms of a contribution and then in the PR Carmine can decide what they like.