3 ms·
In my experience they are equally unusable due to compounding reliability issues. It’s a tech demo, not a platform, a data acquisition operation and alpha test
by gmerc 3y ago
In my experience they are equally unusable due to compounding reliability issues.
It’s a tech demo, not a platform, a data acquisition operation and alpha testing rather than anything seriously useful.
Interfaces are unstable.
Inference is intentionally non deterministic and control is crippled (e.g by choosing not to expose seed, denoising or image to image on dalle), to avoid PR backlash whole simultaneously hiding the true capabilities of the platform.
something as simple as using VITS to analyze an image fails 30% of the time because GPT5 decides it doesn’t have vision, wants to use pytesseract instead or writes hallucinatory pytorch code for a non existent vits library instead of just using inference.
One may create prompts that temporarily don’t fail at a high rate but constant silent finetuning, system prompt changes and unstable models / APIs make the whole thing a tech demo designed to get users to volunteer future training data
- bongodongobob 3y agoYeah, it's pretty exciting. I never thought this would be possible in my lifetime and it's actually accelerating so quickly it's hard to target.
- tomalaci 3y agoOh dear, looks like this seeming bot-comment is one of those reliability issues that GP was on about.
- bongodongobob 3y agoMmk.
- dr_kiszonka 3y agoEven OpenAI's plugins are unreliable. They recently removed some capabilities from Data Analyst without a word of warning.
- arthurcolle 3y agosnapshotted models don't really do this... But I agree, OpenAI is a seriously awful target for serious work. I've been pretty focused on a function calling mixtral funetuning dataset. The moment I can reliably do inference at a high speed with a finetuned model that can do gpt-4-32k function calling in an intelligent, hierarchically ordered big planner manner is the day that I use OpenAI a lot less. It's coming! miqu leaked over the last week! We're so close.
- throwuxiytayq 3y agoI've tried getting Custom GPT actions to work the other day. The workflow is terrible. The UI is broken and requires me to keep refreshing and re-navigating to the editing page after each change. Saving doesn't work every time. The actions themselves don't work either (I think they pushed an update while I was testing, turning them from "broken and usually doesn't work" to "utterly broken"). There is no sensible feedback about what went wrong. No error message, no logs. The "executing action" UI is broken, and I keep getting various versions of the UI randomly on different runs, for some god forsaken reason. Sometimes there is no UI at all and the bot dumps the raw command straight into chat. I've seen alpha-quality software, and this isn't even that.
- MOARDONGZPLZ 3y agoI experienced the same. My hack was to create a GPT to interact with the interface for me. It’s GPTs all the way down, and my prediction is that eventually every interface on the internet will be GPT-based communicating over their own GPT IR under the hood. Oh god.
- bigfishrunning 3y agoAt some point we'll have to throw this internet away and start a new one.
- behnamoh 3y agoshould we call it ipfs?
- stavros 3y agoIPFS works even less than the current internet.
- TeMPOraL 3y agoYou know how cyberpunk sci-fi talks about the cyberspace haunted by rogue AIs and daemons, being held back by hackers who fight them like knights fighting dragons? I always found it childlishly stupid anthropomorphization of systems and processes - basically dumbing down reality to something 5 year old can enjoy, and treating it as a prediction of the future. That is, until last year. With Internet becoming GPTs fronting for other GPTs, well, fuck it, rogue AIs and demons and cybersurfers it is. Reality itself got dumb.
- behnamoh 3y agoThere's a ridiculous 40 messages per 3 hours limit which gets burned quickly when designing a GPT. Imo they shouldn't include that in the limit.
- franze 3y agothey did not. And it was promptly misused to circumvent ChatGPT limits.
- infecto 3y agoWhile I agree ChatGPT is pretty flaky, calling the vision model via API I have not experienced these issues. Is it perhaps a tier issue with some of your API work, there is drastic performance differences between the service tiers.
- gmerc 3y agoThe fact that this is not debuggable itself is part of the problem.
- samatman 3y agoThe implementation is incredibly simplistic to the point where I consider it broken. I have a custom GPT where part of the shtick is that it speaks in rhyme. As an aside, I tried, and repeatedly failed, to get it to speak in iambic pentameter blank verse, but for whatever reason, that isn't a concept/constraint it can recognize and work with. So whatever, rhyming is ok. The point isn't about that, it's that when I talk with this GPT for long enough, it abruptly forgets about speaking in rhyme. The custom prompt is just initial token state, once it drops out of the context window, it's gone. This is a disaster for anyone trying to deploy a task-specific GPT, because it will appear to work for a few thousand tokens, after which it will just turn into vanilla GPT. There have to be a ton of ways to prevent this from happening but the system as implemented doesn't do any of them.