lumost

5 ms·

61.
by lumost 4mo ago
Context is more available locally. You can have the LLM operate for arbitrarily long periods, use your credentials to access services (if desired), store memory locally etc. Whether such a model exists or not is a different question.
62.
by lumost 4mo ago
The big question for local LLMs is whether there is a 100 tok/s model which requires less than 16 GB of memory and is competitive on most tasks with the cloud models. There is some signal that this is possible through both hardware inn
63.
by lumost 4mo ago
The s curve won’t inflect until it becomes difficult to allocate additional resources due to economic limitations. There is no sign that training a model on 10x the compute won’t lead to at least an equivalent improvement as the last order
64.
by lumost 4mo ago
I don't think this is really true, there are plenty of engineers/managers who rotate through major tech companies. Many Meta folks will head off to new companies which would pay at similar levels.
65.
by lumost 4mo ago
This is true if you take the ai market as equal to the market for labor discounted to 5-10% penetration. It’s not a totally unreasonable assertion, it’s the implication of the assertion that we are uncomfortable with. There is no reason for
66.
by lumost 4mo ago
The abstraction of capital and money get a bit funny when wealth is sufficiently concentrated. If there is a monopsomy (one buyer), then they can largely dictate the price of anything. If they also control violent coercion via a captured st
67.
by lumost 4mo ago
its worse at code compared to qwen 3.6 coder.
68.
by lumost 4mo ago
This just looks like a capex problem. There is no evidence that Anthropic has secret sauce above and beyond access to capital. If there is secret sauce, it's unclear that it changes the required amount of capital by all that much. Chin
69.
by lumost 4mo ago
bear in mind, elon musk now has the wealth of 10k "100 millionaires" - we truly lack comprehension of how wealthy the wealthy have gotten.
70.
by lumost 4mo ago
I really think the environmental movements were a red herring. It was always impossible to make a meaningful dent in your personal emissions while still existing in your location. There was never any reduction proposal which could mitigate
71.
by lumost 4mo ago
This is either a complete own goal by Amazon… a play to consolidate compute/model access. Will Chinese models be allowed on the market… at all? Will startups be banned from training models of equivalent capacity?
72.
by lumost 4mo ago
No, reusing segments of the kv cache for different purposes in an order independent manner is an active research area.
73.
by lumost 4mo ago
The KV cache is order dependent and dependent on the context of tokens which exist before the KV cache. There are some transformation approaches to re-use the kv cache across inferences, but none are in wide use due to accuracy concerns fol
74.
by lumost 4mo ago
Is this the inevitable outcome of frontier labs who own their hardware? the GPUs and datacenters are the major cost. The inference and training a higher tier value proposition, if the company gets nervous that the investment in hardware won
75.
by lumost 4mo ago
Is it really "wear and tear?" or is it an evolved mechanism to keep genetic drift and natural selection alive? Alternatively, it could be an evolved mechanism to avoid genetic bottlenecks caused by highly reproductive individuals
76.
by lumost 4mo ago
Why do we think frontier model vendors are high margin?
77.
by lumost 4mo ago
TSLA has been around for many years, whether I agree with its value or not. It has been able to retain its valuation in the public market. An IPO with massive insider selling counterbalanced by a flood of index fund rebalancing is entirely
78.
by lumost 4mo ago
It's quite clear that there is an effort to engineer mega financial vehicles that index tracking funds are forced to buy. The incentive to do so is massive, and there is nothing illegal about it. As a holder of index funds such as the
79.
by lumost 4mo ago
the math is odd. Paying 2k for something that you use 100 hours of is quite expensive. Having the capability built into your existing silicon which you would buy for 1k is cheap. Paying 200 dollars a month for 2 years give a present value o
80.
by lumost 4mo ago
The current theme is that agi may not be definable, and an ai which matches humans on all economically relevant tasks is close enough for business purposes. Billions spent on RL may be good enough to beat human performance.
81.
by lumost 5mo ago
I can't put a 10 page narrative on how my specific motor should work into a hacker news post ;) you can also imagine the above where the goal is to have the ai exceed the performance of stock motors.
82.
by lumost 5mo ago
Laptops/desktops are cheaper per flop than any datacenter hardware by a good order of magnitude. The problem is that expectations rise in datacenters, hardware/power/security/availability guarantees cost real money. Then
83.
by lumost 5mo ago
"Design me a 3d printable rocket engine for a hobby rocket project. Verify it's design in a full simulation. Iterate until it works reliably in simulation based on a verified printable design on a consumer laser sintering device (
84.
by lumost 5mo ago
I actually can’t wait for the future where I upgrade hardware in order to upgrade my ai as an alternative to an expensive subscription. There are many problems I want to work on which require billions of tokens. These are completely inacces
85.
by lumost 5mo ago
Taste, architecture, new innovations. These are all streams of tokens which are subject to the same scaling laws as code, language, and basic classification. We are going to see a new generation of models which effectively “solves” these pr
86.
by lumost 5mo ago
The author’s implied argument is that capital control’s the entrepreneurship game already. There is only 1 top law firm, financiers of law firms have no interest in starting a race to the bottom. Foundation model labs will take a significan
87.
by lumost 5mo ago
They are likely also starting to realize that the end result of their anthropic contract is that nobody but anthropic knows how to run their business. Why would anthropic not treat their business like a utility in the future?
88.
by lumost 5mo ago
The critical view is that these IPOs are bumping up against physics. How many trillion dollar companies can the economy support? The US GDP is roughly 32 Trillion. A company with 100 billion dollars in revenue and 10x annual growth would be
89.
by lumost 5mo ago
Which also leaves OpenAI vulnerable to NVidia's aggressive pricing. To my knowledge Anthropic is relatively well positioned across multiple compute vendors/hardware providers.
90.
by lumost 5mo ago
I find it to be a fundamental hinderance as they use a cheap model. The cheap model gets confused based on conflicting reddit threads and then I get a wrong answer. It's better to just ask codex to do the search for me, but this is muc

More ›