3 ms·

We tested several models and GPT-4o was the most accurate and makes the most sense for our initial launch That said: it's 100% our intention to add local model

We tested several models and GPT-4o was the most accurate and makes the most sense for our initial launch That said: it's 100% our intention to add local models. This is just our v1.

AlexErrant 2y ago
Yeah perhaps I came in with the wrong expectations. The ui/url made me expect a postgre playground, when instead I got this. Perhaps Postgres.ai? Also... the page title is "Postgres Sandbox". This is, at best, misleading.
auspiv 2y ago
The point is that the verbiage ("To prevent abuse we ask you to sign in before chatting with AI.") implies that only the AI won't work if you don't sign in, not the entire product/site.
- kiwicopple 2y ago
  good feedback, thanks we'll change the verbiage for now, and look at ways we can provide a 100% local experience without logins
vrmiguel 2y ago
I think the point is that many of us would like this without any AI in it, just a simpler Postgres playground in the browser, much like the Rust or Go playgrounds.
- cheema33 2y ago
  > just a simpler Postgres playground in the browser... Then you want pglite. This project uses it and provides a link to it.
  - vrmiguel 2y ago
    I obviously saw that since it's linked in the very first message in the chain of messages I responded to. pglite.dev/repl does not have the same level of visualizations as postgres.new. What I 'want' is exactly what I described in my previous message, postgres.new but without the required LLM integration, the same sentiment others had in this thread.
    - kiwicopple 2y ago
      we have something like that from an earlier launch: https://wasm.supabase.com/ https://wasm.supabase.com/ Be aware that the WASM file is 30Mb, but i think it fits what you're describing. details here: https://supabase.com/blog/postgres-wasm https://supabase.com/blog/postgres-wasm
underlines 2y ago
Benchmarks suggest otherwise. Toqan's sql benchmark shows other models way up in the ranking compared to gpt-4o [1] Open Weight models specifically fine-tuned on sql generation and modification also rank pretty well compared to SOTA proprietary models. If you want to eval alternative models, check out sqleval [2] 1 https://prollm.toqan.ai/leaderboard/stack-unseen?type=conceptual,version,debugging,implementation,optimization&level=intermediate,advanced,beginner&tag=sql https://prollm.toqan.ai/leaderboard/stack-unseen?type=concep... 2 https://github.com/defog-ai/sql-eval https://github.com/defog-ai/sql-eval

We tested several models and GPT-4o was the most accurate and makes the most sense for our initial launch That said: it's 100% our intention to add local models. This is just our v1.

AlexErrant 2y ago

Yeah perhaps I came in with the wrong expectations. The ui/url made me expect a postgre playground, when instead I got this. Perhaps Postgres.ai? Also... the page title is "Postgres Sandbox". This is, at best, misleading.

auspiv 2y ago

The point is that the verbiage ("To prevent abuse we ask you to sign in before chatting with AI.") implies that only the AI won't work if you don't sign in, not the entire product/site.

kiwicopple 2y ago
good feedback, thanks we'll change the verbiage for now, and look at ways we can provide a 100% local experience without logins

vrmiguel 2y ago

I think the point is that many of us would like this without any AI in it, just a simpler Postgres playground in the browser, much like the Rust or Go playgrounds.

cheema33 2y ago
> just a simpler Postgres playground in the browser... Then you want pglite. This project uses it and provides a link to it.
- vrmiguel 2y ago
  I obviously saw that since it's linked in the very first message in the chain of messages I responded to. pglite.dev/repl does not have the same level of visualizations as postgres.new. What I 'want' is exactly what I described in my previous message, postgres.new but without the required LLM integration, the same sentiment others had in this thread.
  - kiwicopple 2y ago
    we have something like that from an earlier launch: https://wasm.supabase.com/ https://wasm.supabase.com/ Be aware that the WASM file is 30Mb, but i think it fits what you're describing. details here: https://supabase.com/blog/postgres-wasm https://supabase.com/blog/postgres-wasm

underlines 2y ago

Benchmarks suggest otherwise. Toqan's sql benchmark shows other models way up in the ranking compared to gpt-4o [1] Open Weight models specifically fine-tuned on sql generation and modification also rank pretty well compared to SOTA proprietary models. If you want to eval alternative models, check out sqleval [2] 1 https://prollm.toqan.ai/leaderboard/stack-unseen?type=conceptual,version,debugging,implementation,optimization&level=intermediate,advanced,beginner&tag=sql https://prollm.toqan.ai/leaderboard/stack-unseen?type=concep... 2 https://github.com/defog-ai/sql-eval https://github.com/defog-ai/sql-eval