MoonGhost

12 ms·

91.
by MoonGhost 2y ago
I think the problem is with positional encoding. If model cannot clearly separate tokens in context window they overlap which leads to mess. That encoding matters and actual position does not.
92.
by MoonGhost 2y ago
> Anecdotally, I feel MoE models sometimes exhibit slightly less “deep” thinking Makes sense to compare apples with apples. Same compute amount, right? Or you are giving less time to MoE model and then feel like it underperforms. Shouldn
93.
by MoonGhost 2y ago
It probably depends on how much A and B overlap. If it's say English sci-fi and Chinese poetry two different models may be better.
94.
by MoonGhost 2y ago
> Late 2025, "its PhD-level knowledge of every field". I just don't think you're going to get there. You think too much of PhDs. They are different. Some of them are just repackaging of existing knowledge. Some are ju
95.
by MoonGhost 2y ago
They already can do calculations by using tools and without pretending. Why not to make them write code for logic too. This will extend their 'range'. End user can be provided only summary to keep it look simple.
96.
by MoonGhost 2y ago
> when viewed from the other side? Nobody has done it so far. We have only theories and hypothesis.
97.
by MoonGhost 2y ago
Regardless, are there any good examples of projects generated by LLMs? There was a game like Angry Birds. But that was long time ago. I did some simple 'games'. If it's easy there should be a lot of open source projects, righ
98.
by MoonGhost 2y ago
> Full self replication of a machine either from sand to silicon or through some yet to be developed process inspired by biology will be a fundamental change in how we do everything. Not any time soon. This requires the whole economy to
99.
by MoonGhost 2y ago
Noise is an artistic effect, it changes the look and feel. A) It adds something in the blanked spaces. B) improves visual sharpness of blurry images. C) It works in video as well. Also increases image file size and/or introduces compre
100.
by MoonGhost 2y ago
We know how it ended for Rasputin.
101.
by MoonGhost 2y ago
Even average papers can have nice overview of the problem and references.
102.
by MoonGhost 2y ago
> obvious mistakes and completely uninspired, sloppy software and service design. That's something you can work on to improve. A few years back I wanted to work for FAANG big company. Now I don't after working for smaller but w
103.
by MoonGhost 2y ago
I'm working on sort of 'graph' library. It's litcoding all the way. There are many separate containers and algorithms. The problem to a) write them b) optimize for memory c) optimize for performance d) find a 'good
104.
by MoonGhost 2y ago
> I hear we hit a plateau with current approaches Then we need new. The important last year step was distillation as mainstream. In my opinion. Now using old models to train new is normal or even necessary. That was done before, but it
105.
by MoonGhost 2y ago
If you recall how it was just 4 years back. Since then more progress in AI than in previous 40. Which in turn better than prev 400. It's accelerating and unstoppable. It will be very different world in 10 years from now. I hope I live
106.
by MoonGhost 2y ago
According to this cheap food is good because people just start eating more. Actually cheap imports can be really bad for local businesses. Now imagine dystopian world where AI can solve most data / engineering / science problems f
107.
by MoonGhost 2y ago
Coding is actually a hard skill which requires practice. Regular litcoding for the sake of it should help. The problem with that is it takes the whole brain and breaks other thoughts chain. I'm thinking about to dedicate full days for
108.
by MoonGhost 2y ago
Google is not alone. Intel halfheartedly develops GPUs. It doesn't work without strategy.
109.
by MoonGhost 2y ago
Can't be. "It was launched by Anthropic back in November 2024," about MCP
110.
by MoonGhost 2y ago
Caligula et Messaline 1981 exist in two close but significantly different versions played by the same actors at the same time.
111.
by MoonGhost 2y ago
> found it strange that more people weren't talking about it. Some simply dislike everything OpenAI. Just like everything Musk or Trump.
112.
by MoonGhost 2y ago
Well, probably you can try to go down to simpler models to get the idea. (they are almost useless) From my experience better model like Claude or o3 can do things that others simply cannot. At some complexity they start going circles making
113.
by MoonGhost 2y ago
And when it finally works..? I suspect (vibe) coding can be significantly improved by multi-step structural approach. Including coding, review, testing in a loop. Fully automatic. Like Chain of Thoughts helps solving logical tasks. This can
114.
by MoonGhost 2y ago
BTW, Copilot is not the best at coding. With the quality LLM return exponentially grows. Bigger chunks, fewer bugs, less time checking. From my experience LLMs do not impress on complex algorithms and shine on small utilities. They can use
115.
by MoonGhost 2y ago
Could it be that model just uses latent space for thinking while generating almost garbage? Interesting to check if adding repeating something at the end of prompt helps. I.e. model uses it for 'thinking'.
116.
by MoonGhost 2y ago
The opposite of it would be an agent which deliberately generates expensive but useless requests. Like search. If it detects labyrinth.
117.
by MoonGhost 2y ago
> most humans will fail their first time it's easier on 4 legs
118.
by MoonGhost 2y ago
Cool! Next step roller skating. It could be even useful in real tasks like delivery as it's more energy efficient and fast in urban environment.