espadrine

8 ms·

61.
by espadrine 2y ago
It is a neat approach, but one that comes with a tradeoff, IIUC: doubling the key heads. I wonder if a different approach without that issue exists. For instance, using max(0, exp(x)-1) instead of exp(x) in the softmax attention formula. Th
62.
by espadrine 2y ago
AWS: $0.024/GB/month (S3 standard, first 50 TB/month) GCP: $0.023/GB/month (standard storage, beyond 5 GB free limit) Azure: $0.019/GB/month (hot, first 50 TB/month) Cloudflare: $0.015/GB/mo
63.
by espadrine 2y ago
They are looking for a data structure that is robust against hash flooding attacks like https://www.cve.org/CVERecord?id=CVE-2011-4815 You mention that foldhash does not claim to provide HashDoS resistance against interacti
64.
by espadrine 2y ago
> New LLMs don't even rely that much on that aforementioned older architecture Don’t they all indicate being based on the transformer architecture? > not entirely because of transformers but because of the hardware Kaplan et
65.
by espadrine 2y ago
Short histories remove a lot of information, but it would be impractical to make it book-sized. There were numerous founders, and as another commenter mentioned, Elon Musk recruited Ilya, which soured his relationship with Larry Page. Hones
66.
by espadrine 2y ago
A short history: 1986: Geoffrey Hinton publishes the backpropagation algorithm as applied to neural networks, allowing more efficient training. 2011: Jeff Dean starts Google Brain. 2012: Ilya Sutskever and Geoffrey Hinton publish AlexNet, w
67.
by espadrine 2y ago
> I've been A/Bing 11B today with Molmo 72B How are you testing Molmo 72B? If you are interacting with https://molmo.allenai.org/ , they are using Molmo-7B-D.
68.
by espadrine 2y ago
The paper: https://molmo.allenai.org/paper.pdf > Our key innovation is a simple but effective data collection strategy that avoids these problems: we ask annotators to describe images in speech I see this as another ex
69.
by espadrine 2y ago
We can make informed guesses. Such attackers likely use the same TCP stack and datacenter within the attack, so perhaps the cipher suite, the latency (based off the Date header?), the OS, the UA, the set of HTTP headers sent, are some of th
70.
by espadrine 2y ago
> Use regular bigserial (64bit) PKs for internal table relations and UUIDs (128bit) for application-level identifiers and natural keys These recommendations are database-specific. It is definitely true of MySQL, since its row storage h
71.
by espadrine 2y ago
I don’t know Radix, so perhaps they use a different merge algorithm than the standard one. The article indicates, though, that nondeterminism was a bug that the company fixed. It does sound like it relies on validators, so perhaps they don’
72.
by espadrine 2y ago
It is deterministic: the local updates must be grouped into a block linking to the longest-known chain, and the block is broadcast; upon receiving a block, all nodes must locally revert operations that are no longer part of the deepest chai
73.
by espadrine 2y ago
How would you change the definition so that they do not?
74.
by espadrine 2y ago
I would say blockchains fit this definition. What I get away from that is that this definition is not restrictive enough. Similarly, a list of operations with Lamport timestamps fits this definition. However, just like the blockchain, it ma
75.
by espadrine 2y ago
Product presentation is a hard-to-develop skill. I agree that many aspects of the page are muddy. I personally find their compact mode the cleanest I have ever seen. This is the entire window: https://imgur.com/hhfyeVz To a
76.
by espadrine 2y ago
The technology is impressive; achieving such a level requires a lot of efforts in dataset creation, neural architectural costs, and GPU shepherding. What is the company’s ethical position though? It officially stemmed from Mr Musk’s objecti
77.
by espadrine 2y ago
First try, so no cherry picking: https://fal.ai/models/fal-ai/flux-pro?share=43b11da2-e008-4d...
78.
Gemini Live makes your mobile device a powerful AI assistant(blog.google)
3 points by espadrine 2y ago|0 comments
79.
by espadrine 2y ago
Do people search for applications in the App Store or on the Web? This 2023 study[0] indicates 48% of people discover apps by browsing through the App Stores, compared to 21% that discover them through Web search engines. Is this a trend
80.
by espadrine 2y ago
That is disallowed by section 4.2 of the App Review Guidelines[0]: > Your app should include features, content, and UI that elevate it beyond a repackaged website. [0]: https://developer.apple.com/app-store/review&
81.
by espadrine 2y ago
The problem is cultural: a growing population do not know what the Web is. When they are asked to search for a website, they open the app store and search there. If there is no result, they give up.
82.
by espadrine 2y ago
meta.ai is inaccessible in a large portion of world territories, but the Llama 3.1 70B and 405B are also available in https://hf.co/chat Additionally, all Llama 3.1 models are available in https://api.together.ai
83.
Apple Open-Sources LLM DCLM-7B(huggingface.co)
4 points by espadrine 2y ago|1 comments
84.
Stable Assistant: Chatbot with Stable Diffusion 3 Ultra, Stable Video(stability.ai)
2 points by espadrine 2y ago|0 comments
85.
by espadrine 2y ago
> it's cheaper but not cost-free to sustain a narrative of "independence" and "alternative to incumbent" The narrative is independent of the price. SpaceX has showed that they may refuse service in various situ
86.
by espadrine 2y ago
One extensive benchmark I like is https://bigcode-bench.github.io/ It places Claude 3.5 Sonnet in third position.
87.
by espadrine 2y ago
How big of a perf improvement would result from using the architectural tweaks that Llama3 and others have put in place since GPT-2?
88.
by espadrine 3y ago
Huh, that is true in both the model cards of Gemini 1.5 Pro and Gemini 1.0. That feels like it runs counter to this statement from the Gemini 1.0 technical report[0]: > Gemini models are trained to accommodate textual input interleaved
89.
by espadrine 3y ago
That also matches my estimation which places AlphaGo Zero 40-blocks 40-days at 5185, and KataGo at 6262: https://thefiletree.com/espadrine/%E2%9A%92/elo.html (Details of the estimation in the source code.)
90.
by espadrine 3y ago
> Google's Gemini LLM model was not to blame for the image generation weirdness. That would be like blaming DALL-E weirdness on GPT-4. The way I read the Gemini technical report, it seemed like, unlike GPT-4 vs DALL-E, Gemini was

More ›