3 ms·

- magic.dev claimed 200M context window and it's been two years since and no real product yet. - They are admitting that this is built on top of a Chinese mode

by mohsen1 5mo ago

- magic.dev claimed 200M context window and it's been two years since and no real product yet. - They are admitting that this is built on top of a Chinese model[1] - They committed a huge chart crime with the Y axis of a chart comparing to Opus on their website that I can't find anymore (Too embarrassing to keep?). The delta between their score (81%) vs. Opus (87%) on SWE bench was hugely minimized - They named the company subquadratic but in parts they said O(1) linear scaling. At O(1) you could do much more than 12M tokens context window. At O(log n) even. I hope this is real but I doubt...

artisin 5mo ago
Ah, I nearly forgot about magic.dev. I took a quick peek to check up on them. Welp, last social/blog activity was in... 2024. But hey, their careers page still says they're hiring! So they must be doing just fine.
- shdh 5mo ago
  They did raise over $500M
alexsubq 5mo ago
The chart crime was not intentional! We will not make you wait two years. We are O(n), not O(1). O(1) would unfortunately be an impossibility. We may as well do infinite context at that point!
- esafak 5mo ago
  Good luck.
  - alexsubq 5mo ago
    Thanks!
- bbctr1 5mo ago
  What’s keeping you from releasing paper and access to the model?
  - alexsubq 5mo ago
    Model: - making sure it has been properly red-teamed, meets user preferences, etc. - it depends on what folks want in the model. Our original papers was mostly the technical blog post, but we decided to wait a little longer to see what else folks wanted and share more benchmarks
pvtmert 5mo ago
> not affiliated with subq, i see in the linked post they mention O(n) not O(1). O(1) would basically be impossible and instant. Something like no compute required, constant results... The name subquadratic is actually good and makes sense to me. Because today's models are usually O(n^2) or worse. Anything equals or less than O(n^1) is basically sub-quadratic. Meanwhile O(log n) would be logarithmic as the log name indicates. But we have a long way to go there. Maybe with double tokenizer plus extensive caching it may be possible... What I mean here is tokenizing the user input; then capturing intent; caching intent -> response. So that next time once you get the intent, you don't need to do full transformer inference compute. This can be logarithmic complexity in terms of time complexity.

by mohsen1 5mo ago

artisin 5mo ago

Ah, I nearly forgot about magic.dev. I took a quick peek to check up on them. Welp, last social/blog activity was in... 2024. But hey, their careers page still says they're hiring! So they must be doing just fine.

shdh 5mo ago
They did raise over $500M

alexsubq 5mo ago

The chart crime was not intentional! We will not make you wait two years. We are O(n), not O(1). O(1) would unfortunately be an impossibility. We may as well do infinite context at that point!

esafak 5mo ago
Good luck.
- alexsubq 5mo ago
  Thanks!
bbctr1 5mo ago
What’s keeping you from releasing paper and access to the model?
- alexsubq 5mo ago
  Model: - making sure it has been properly red-teamed, meets user preferences, etc. - it depends on what folks want in the model. Our original papers was mostly the technical blog post, but we decided to wait a little longer to see what else folks wanted and share more benchmarks

pvtmert 5mo ago

> not affiliated with subq, i see in the linked post they mention O(n) not O(1). O(1) would basically be impossible and instant. Something like no compute required, constant results... The name subquadratic is actually good and makes sense to me. Because today's models are usually O(n^2) or worse. Anything equals or less than O(n^1) is basically sub-quadratic. Meanwhile O(log n) would be logarithmic as the log name indicates. But we have a long way to go there. Maybe with double tokenizer plus extensive caching it may be possible... What I mean here is tokenizing the user input; then capturing intent; caching intent -> response. So that next time once you get the intent, you don't need to do full transformer inference compute. This can be logarithmic complexity in terms of time complexity.