samus

32 ms·

601.
by samus 3y ago
But it does - it just needs an application to retrieve the buffer and do something with it. For example pushing it to storage.
602.
by samus 3y ago
CPUs and DSPs are not primarily designed for graphics work, therefore they don't count as GPUs. CPU are general-purpose, DSPs might be abused for graphics work. The "G" in GPU doesn't imply that they have to render direc
603.
by samus 3y ago
The amount of possibilities you see might be influenced by your emotional state, and what you call "reasonable" might not be the same thing as other people consider it. A lot of our reasoning are applications of observations and b
604.
by samus 3y ago
I think you might doing some disservice by not pointing out such things. Outsider's perspective can be very valuable to improve processes. Even if there are good reasons why things are how they are, you'll find out much faster why
605.
by samus 3y ago
Intel could just as well become a pure fab and give up on designing processors. Switching to another ISA might tie them up for years before they can deliver state-of-the-art processors again. Intel can't afford that after the recent ye
606.
by samus 3y ago
No, they really can't get off x86, the architecture they built their business on. The best they can do is add new extensions to provide modern alternatives to crufty instructions, while still having to support the old ones. NPUs are a
607.
by samus 3y ago
It's admittedly less common, but also in Chinese lots of characters have multiple pronunciations. Sometimes they are associated with different meanings. Even if the other pronunciation is just considered formal or poetic, it can carry
608.
by samus 3y ago
AOT whole program optimization has two limits: * It is possible with `dlopen()` to load code objects that violate the assumptions made during compilation. * The presence of runtime configuration mechanisms and application input can make it
609.
by samus 3y ago
1. They need it for style and language, not necessarily for the facts 2. Since GPT-4 is seen as the very best general-purpose LLM in existence, it makes sense to emulate its performance with less resources. 3. Phi models are also trained wi
610.
by samus 3y ago
It's a tech report. Fair enough to include the whole lab.
611.
by samus 3y ago
This is one of the cases where JIT compiling can shine. You can use a bazillion interfaces to decouple application code, and the JIT will optimize the calls after it found out which implementation is used. This works as long as there is onl
612.
by samus 3y ago
Devirtualization maybe not necessarily, but inlining might make code fail to fit into instruction caches.
613.
by samus 3y ago
In these situation photons are either bouncing off matter or are getting absorbed and emitted by matter. In the former case they don't travel a straight path, in the latter case there is a short time lag between absorption and emission
614.
by samus 3y ago
Surely it might still work, but training a model into true overfitting territory is a waste of resources. An overtrained model is usually a worse model than an earlier checkpoint. That's the main reason one strives to avoid that.
615.
by samus 3y ago
Many MoE models like Mixtral were not created from scratch. Somebody could still upcycle that 8B into an 8x8B. It should ideally be done by Meta since they have access to the original training data.
616.
by samus 3y ago
Probably nothing specific. By definition, an overfitted model has only learned to deal with the training input and will perform worse on anything that is different. In training metrics it shows as a large gap between the performance metrics
617.
by samus 3y ago
Depends what you want and whether you have new data or just train for more epochs. More relevant data is probably always a good thing, repeating the same data not necessarily. Repeating data is probably good though if you really want the mo
618.
by samus 3y ago
Any business idea built almost exclusively on AI, without adding much value, is doomed from the start. AI is not good enough to make humans obsolete yet. But a well finetuned model can for sure augment what individual humans can do.
619.
by samus 3y ago
So far, people have come up with many alternatives for quadratic attention. Only recently have they proven their potential.
620.
by samus 3y ago
RAM is simply too useful.
621.
by samus 3y ago
64GB is not GPU RAM, but system RAM. Consumer GPUs have 24GB at most, those with good value/price have way less. Current generation workstation GPUs are unaffordable; used can be found on ebay for a reasonable price, but they are quite
622.
by samus 3y ago
It misses the crucial detail that every transformer layer chooses the experts independently from the others. Of course they still indirectly influence each other since each layer processes the output of the previous one.
623.
by samus 3y ago
There is Qwen1.5-MoE-A2.7B, which was made by upcycling the weights of Qwen1.5-1.8B, splitting it and finetuning it.
624.
by samus 3y ago
Almost :) the model chooses experts in every block. For a typical 7B with 8 experts there will be 8^32=2^96 paths through the whole model.
625.
by samus 3y ago
It should increase quality since those layers can specialize on subsets of the training data. This means that getting better in one domain won't make the model worse in all the others anymore. We can't really tell what the router
626.
by samus 3y ago
Unsurprisingly, you will find people who argue even Lambdas went to far, and others who are willing to endure Lombok, multiple annotation processors, or precompilers to work with more advanced features.
627.
by samus 3y ago
We have both moved as far from the original topic as possible.
628.
by samus 3y ago
We at least have in common with them that we are mammals. Therefore, we can very much assume that our brain is more similar to theirs than, say, an octopus' brain. Apart from that, we very much know how certain parts of the human brain
629.
by samus 3y ago
Such corporate structures are often set up for the express purpose of optimizing tax. Replacing corporate tax with dividend tax would remove incentives from setting up such structures, which is a good thing. Apart from that, double taxation
630.
by samus 3y ago
One simple reason might be this over: people make errors over time, but they become familiar with the case. That familiarity is lost whenever they hand over a patient. And you can't solve that with documentation since people would be b

More ›