4 ms·

You can access Claude models with Google Cloud reliability via VertexAI. The caveat is that you cannot use your subscription, per-token pricing only. I persona

You can access Claude models with Google Cloud reliability via VertexAI. The caveat is that you cannot use your subscription, per-token pricing only. I personally prefer per-token, it makes you more thoughtful about your setup and usage, instead of spray and pray. You can also access the notable open weight models with VertexAI, only need to change the model id string.

joe_mamba 7mo ago
I saw a funny skit where if free Claude instance was down for you, you could just ask Rufus, Amazon's shopping AI assistant, your math/coding question phrased as a question about a product, and it would just answer lol.
- Tade0 7mo ago
  In my region a certain small bank had an AI assistant which someone neglected to limit, so you could put whatever there and not even phrase it as a question about a product.
Scene_Cast2 7mo ago
I also use them per-token (and strongly prefer that due to a lack of lock-in). However, from a game theory perspective, when there's a subscription, the model makers are incentivized to maximize problem solving in the minimum amount of tokens. With per-token pricing, the incentive is to maximize problem solving while increasing token usage.
- verdverm 7mo ago
  I don't think this is quite right because it's the same model underneath. This problem can manifest more through the tooling on top, but still largely hard to separate without people catching you. I do agree that Big Ai has misaligned incentives with users, generally speaking. This is why I per-token with a custom agent stack. I suspect the game theoretic aspects come into play more with the quantizing. I have not (anecdotally) experienced this in my API based, per-token usage. I.e. I'm getting what I pay for.
chewbacha 7mo ago
You mean Google Chaos Services as we call them?
perfmode 7mo ago
You can use your subscription for Anthropic-hosted Claude models?
- verdverm 7mo ago
  Don't know. I tried Anthropic directly a long time ago and was frustrated by their uptime issues. Seems it has not improved in the years since.
- lima 7mo ago
  No, unless you count tricks which are explicitly against ToS
lima 7mo ago
We tried this, but the quota for Opus models defaults to 0 on VertexAI and quota increase requests are auto-rejected. Any tips?
- polski-g 7mo ago
  What? There's no quota at all. You pay per token up to infinity.
  - verdverm 7mo ago
    There are in fact quotas and rate limits in VertexAI, albeit generous and automatically increased based on spend

joe_mamba 7mo ago

I saw a funny skit where if free Claude instance was down for you, you could just ask Rufus, Amazon's shopping AI assistant, your math/coding question phrased as a question about a product, and it would just answer lol.

Tade0 7mo ago
In my region a certain small bank had an AI assistant which someone neglected to limit, so you could put whatever there and not even phrase it as a question about a product.

Scene_Cast2 7mo ago

I also use them per-token (and strongly prefer that due to a lack of lock-in). However, from a game theory perspective, when there's a subscription, the model makers are incentivized to maximize problem solving in the minimum amount of tokens. With per-token pricing, the incentive is to maximize problem solving while increasing token usage.

verdverm 7mo ago
I don't think this is quite right because it's the same model underneath. This problem can manifest more through the tooling on top, but still largely hard to separate without people catching you. I do agree that Big Ai has misaligned incentives with users, generally speaking. This is why I per-token with a custom agent stack. I suspect the game theoretic aspects come into play more with the quantizing. I have not (anecdotally) experienced this in my API based, per-token usage. I.e. I'm getting what I pay for.

chewbacha 7mo ago

You mean Google Chaos Services as we call them?

perfmode 7mo ago

You can use your subscription for Anthropic-hosted Claude models?

verdverm 7mo ago
Don't know. I tried Anthropic directly a long time ago and was frustrated by their uptime issues. Seems it has not improved in the years since.
lima 7mo ago
No, unless you count tricks which are explicitly against ToS

lima 7mo ago

We tried this, but the quota for Opus models defaults to 0 on VertexAI and quota increase requests are auto-rejected. Any tips?

polski-g 7mo ago
What? There's no quota at all. You pay per token up to infinity.
- verdverm 7mo ago
  There are in fact quotas and rate limits in VertexAI, albeit generous and automatically increased based on spend