5 ms·
>> You no longer need to review the code. Or instruct the model at the level of files or functions. You can test behaviors instead. I think this is where thing
by asdff 10mo ago
>> You no longer need to review the code. Or instruct the model at the level of files or functions. You can test behaviors instead.
I think this is where things will ultimately head. You generate random code, purely random in raw machine readable binary, and simply evaluate a behavior. Most random generated code will not work. some, however, will work. and within that working code, some will be far faster and this is the code that is used.
No different than what a geneticist might do evaluating generated mutants for favorable traits. Knowledge of the exact genes or pathways involved is not even required, one can still select among desired traits and therefore select for that best fit mechanism without even knowing it exists.
- asdev 10mo agoall fun and games until you need to debug the rats nest that you've been continually building. I am actually shocked people who have coded before have been one-shotted into believing this
- asdff 10mo agoIf a bug rears its head it can be dealt with. Again, this is essentially already practiced by humans through breeding programs. Bugs have come up, such as deleterious traits, and we have either engineered solutions to get around them or worked to purge the alleles behind the traits from populations under study. Nothing is ever bug free. The question is if the bugs are show stoppers or not. And random walk iteration can produce more solutions that might get around those bugs.
- SilenN 10mo agoAgree! Don't let it get too bad. See my other comment on debugging and refactoring being a core part of your workflow.
- madrox 10mo agoYou're describing genetic algorithms: https://en.wikipedia.org/wiki/Genetic_algorithm https://en.wikipedia.org/wiki/Genetic_algorithm
- hollowturtle 10mo agoWhy should we throw away decades of development in determistic algorithms? Why tech people mentions "geneticists"? I would never select an algorithm with a "good" flying trait for making an airplane works, that's nuts
- phist_mcgee 10mo agoGreat rule of business: sell a solution that causes more problems, requiring the purchase of more solutions.
- hollowturtle 10mo agoCustomers are tired of getting piles of shit, look at the Windows situation
- lenerdenator 10mo agoOr don't sell the solution. When you have monopolies, regulatory capture, and endless mountains of money, you can more or less do what you'd like.
- hollowturtle 10mo agoThat's a lie, people will eventually find a way out, it was always like that, being it open source or by innovating and eventually leave the unable to innovate tech giants dying. We have Linux and this year will be the most exciting for the Linux desktop given how bad the Windows situation is
- bradlys 10mo agoOnly been hearing that for twenty years and these tech giants are bigger than they’ve ever been. I remember when people said Open Office was going to be the default because it was open source, etc etc etc. It never happened. Got forked. Still irrelevant.
- stavros 10mo agoMaybe, but you won't be able to test all behaviors and you won't have enough time to try a million alternatives. Just because of the number of possibilities, it'll be faster to just read the code.
- asdff 10mo agoEventually the generation and evaluation will be quite fast where testing a million alternatives will be viable. Impressive you suggest that there might be a million alternatives but it would be faster to just read the code and settle on one. How might that be determined? Did the author who wrote the standard library really come up with the best way when writing those functions? Or did they come up with something that seemed alright to ship relative to other ideas people came up with? I think we need to think outside the box here and realize ideas can be generated, evaluated, and settled upon far faster than any human operates. The idea of doing what a trillion humans evaluating different functions can do is actually realistic with the path of our present technology. We are at the cusp of some very remarkable times, even more remarkable than the innovations of the past 200 years, should we make progress on this effort.
- stavros 10mo agoIf this were viable, we'd all be running Haiku ten times in the time it took to run Opus once, but nobody does.
- asdff 10mo agoWe don't have the compute for this today. We will in several centuries, if compute growth continues.
- hiimkeks 10mo agoWe definitely would need centuries for this, because Moore's law has been dead for a while, while the number of possible programs grows exponentially in it's length. But I hope we have more efficient ways to do this in a century.
- formerly_proven 10mo agoThis is a recurring fantasy in LLM threads but makes little sense. Writing machine code is very difficult (even writing byte code for simple VMs is annoying and error-prone). Abstractions are beneficial and increase productivity (per human, per token). It makes essentially no sense to throw away seven decades of productivity increasing technologies to have neural nets punch cards again, and it's not going to happen unless tokens become unimaginably cheap.
- asdff 10mo agoCompute is always increasing on this planet. It makes no sense to stick with seven decade old paradigms from the time when we were simply transmuting mathematical proofs into computational functions. We should be exploring the void, we will have the compute for this. Randomness will take away the difficulty as we increase compute to parse over these random functions in reasonable time frames. The idea of limiting technological development to what our ape brain can conceive of on its own on human biological timescales is quite a shackle, honestly.
- themafia 10mo ago> You generate random code, Code derived from a training set is not at all "random."
- asdff 10mo agoNot saying training set or present LLM. Truly random binary generator left to its own device. Lets evaluate what spits out from that iterated several trillion times over with the massive compute capability we will have. I am not thinking of this happening in the next couple years, but in the next couple of centuries.
- themafia 10mo ago> Truly random binary generator left to its own device. This has been tried. It tends to overfit to unseen test environment conditions. You will not produce what you intend to produce. > the massive compute capability we will have Just burn carbon in the hopes something magical will happen. This is the mentality of a cargo cult.
- asdff 10mo agoWe also burn carbon to feed the brain. Compute is what is increasing in capability on the scale of orders of magnitudes just within our own lifetimes. Brainpower is not increasing in capability. If you want future capabilities and technological advancement to occur at the fastest pace possible, eventually we have to leave the slow ape brain behind in favor of sources of compute that can evaluate functions several orders of magnitude faster.
- lunar_mycroft 10mo agoFor this to work, you'd have to fully specify the behavior of your program in the tests. Put another way, at that point your tests are the program. So the question is, which is a more convenient way to specify the behavior of a program: a traditional programming language, or tests written in that language. I think the answer should be fairly obvious.
- asdff 10mo agoBehavior does not need to be fully specified at the outset. It could be evaluated after the run. We've actually done this before in our own technology. We studied birds and their flight characteristics, and took lessons from that for airplane development. What is a bird but the output of a random walk algorithm selected by constraints bound by so many latent factors we might never fully grasp?
- lunar_mycroft 10mo ago> Behavior does not need to be fully specified at the outset. It could be evaluated after the run. This doesn't work when the software in question is written by competent humans, let alone the sort of random process you describe. A run of the software only tells you the behavior of the software for a given input, it doesn't tell you all possible behaviors of the software. "I ran the code and the output looked good" is no where near sufficient. > We've actually done this before in our own technology. We studied birds and their flight characteristics, and took lessons from that for airplane development. There is a vast chasm between "bioinspiration is sometimes a good technique" and "genetic algorithms are a viable replacement for writing code".
- asdff 10mo agoGenetic algorithms created our species, which are far more complex than anything we have written in computer science. I think they have stood up to the tests of creating a viable product for a given behavior. And with future compute, you will be able to evaluate behavior across an entire range of inputs for countless putative functions. There will be a time when none of this is compute bound. It is today, but in three centuries or more?
- bossyTeacher 10mo ago> ou generate random code, purely random in raw machine readable binary, and simply evaluate a behavior. Most random generated code will not work. some, however, will work. and within that working code, some will be far faster and this is the code that is used. Humans are expensive but this approach seems incredibly inefficient and expensive. Even a junior can make steady progress against implementing a function, with your approach, just monkey coding like that could take you ages to write a single function. Estimates in software are already bad, they will get worse with your approach
- asdff 10mo agoToday it might not work given what a junior could do against the cost of compute through random walk, but can you say the same in three centuries? We increase compute by the year, but our own brainpower does not increase on those terms. Estimates are that we are actually losing brainpower over time.
- Dilettante_ 10mo agoWe would of course need to specify the behaviors to test for. The more precisely we specify these behaviors, the more complexly our end product would be able to behave. We might invent a formal language for writing down these behaviors, and some people might be better at thinking about what kind of tests would need to be written to coax a certain type of end result out of the machine. But that's future music, forgive a young man for letting his imagination run wild! ;)
- asdff 10mo agoIf we consider other fields such as biology, behaviors of interest are specified but I'm not sure a formal language is currently being used per say. Data are evaluated on dimensional terms that could be either quantitative or qualitative. meta analysis of some sort might be used to reduce dimensionality to some degree but that usually happens owing to lack of power for higher resolution models. One big advantage of this future random walk paradigm is you would not be bound by the real world constraints of sample collection of biological data. datasets could be made arbitrarily large and cost to do so will follow an inverse relationship with compute gains.
- mitemte 10mo agoI can see a lot of negatives in relation to removing the human readable aspect of software development. Thorough testing would be virtually impossible because we’d be relying on fuzzing to iron out potential edge cases or bugs. In this situation, AI companies are incentivised to host the services their tooling generates. If we don't get source code, it is much easier for them to justify not sharing it. Plus, who is to say the machine code even works on consumer hardware anyway? It leads to a future where users specify inputs while companies generate programs and handle execution. Everything becomes a black box. No thank you.
- asdff 10mo agoAll these questions are true for agriculture, yet you say "yes thank you, and please continue" for that industry I am sure, which seeks to improve product through random walk and unknown mechanisms. Maybe take a step back and examine your own biases.
- troupo 10mo ago> All these questions are true for agriculture, yet you say "yes thank you, and please continue" for that industry I am sure, which seeks to improve product through random walk and unknown mechanisms. Tell me you know nothing about modern agriculture without telling me that
- Fire-Dragon-DoL 10mo agoHow do you handle the larger amounts of tests? I did this but my PRs are larger because more tests are needed
- asdff 10mo agoI'm not sure. My thinking is this will occur on the scale of the next several hundred years, not the next few.
- lenerdenator 10mo agoI can tell you why this won't go this way: Customers. When you sell them a technological solution to their problem, they expect it to work. When it doesn't, someone needs to be responsible for it. Now, maybe I'm wrong, but I don't see any of the current AI leaders being like, "Yeah, you're right, this solution didn't meet your customer's needs, and we'll eat the resulting costs." They didn't get to be "thought leaders" in the current iteration of Silicon Valley by taking responsibility for things that got broken, not at all. So that means you will need to take responsibility for it, and how can you make that work as a business model? Well, you pay someone - a human - who knows what they're looking at to review at least some of the code that the AI generates. Will some of that be AI-aided? Of course. Can you make a lot of the guesswork go away by saying "use commonly-accepted design patterns" in your CLAUDE.md? Sure. But you'll still need someone to enforce it and take responsibility at the end of the day if it screws up.
- asdff 10mo agoYou are thinking in terms of the next few years not the next few centuries. Plenty of software sold today fails to meet expectations and no one eats costs.
- dns_snek 10mo agoAnd how exactly do you foresee probabilistic systems working out in real life? Nobody wants software that seldom does what they expect, and which tends to trend toward desirable behavior over time (where "desirable" behavior is determined by the sum of global feedback and revenue/profit of the company producing it). Today you send some money to your spouse but it's received by another person with the same name. Tomorrow you order food but your order gets mixed up with someone else's. Tough luck, the system is probabilistic and you can only hope that the evolutionary pressures influence the behavior to change in desirable ways. This fantasy is a delusion.
- asdff 10mo agoI think you misunderstand. Once established a function found through random walk is no different than a function found in any other way. If it works it works, if it doesn't it doesn't.
- dns_snek 10mo agoI didn't misunderstand. I'm talking about all the exciting unintended behaviors you're adding and all the invariants that you're not preserving by arriving at solutions through randomness rather than careful design and engineering. Yes a function will always do "the exact same thing" at runtime, but that "thing" isn't guaranteed to be free from race conditions and other types of bugs.
- quixoticaxolotl 10mo agoYou're thinking of probabilistic systems at run time. People are talking about probabilistic systems at compile time. Whatever gets generated, if it passes tests and is observably in compliance with the spec, is accepted and made permanent. It's the clay we're talking about Jackson Pollocking, not the sculpture.
- dns_snek 10mo agoI'm not, I'm thinking of poorly engineered systems that display buggy, unintended behaviors at runtime. > observably in compliance with the spec That's so easy to say and so incredibly hard to implement! Most unintended behaviors will never end up being prohibited/defined in the specification written by non-programmers. The act of translating requirements from human language into well-defined semantics is what programming is.
- jimbohn 10mo agoThe problem is that programming logic/state is discrete and not continous so you can't assume similar behaviour given "similar state", and that possible states grow exponentially. Selecting the desired state will mean writing an extremely detailed spec that is akin to a programming language, which is what Dijkstra hinted at in the past.