Vetch

9 ms·

61.
by Vetch 3y ago
Was it via gemma.cpp or some other library? I've seen a few people note that gemma performance via gemma.cpp is much better than llama.cpp, possible that the non-google implementations are still not quite right?
62.
by Vetch 3y ago
Which is why I say it is contextual and depends on the task. I'll note that it's not only programming ability that is empowered but learning math, electronics, history, physics and so on up to the university level. As long as you
63.
by Vetch 3y ago
> thought should feel empowered? This is a strange question since augmentation can be objectively measured even as its utility is contextual. With MidJourney I do not feel augmented because while it makes pretty images, it does not make
64.
by Vetch 3y ago
In this specific instance, I think it is necessary to make a distinction because given the direction things are headed, spied upon is tautological in the context of AI. AI-as-a-service requires sending private details to gain utility and th
65.
by Vetch 3y ago
Don't take tacit knowledge for granted.
66.
by Vetch 3y ago
I'd also wager that "I don't know Timmy" is more thematically related. I feel most of the discussion in this thread glosses over what is most unsettling about Permutation City. It isn't just a book about what it cou
67.
by Vetch 3y ago
If we're talking about open-source LLMs, among the best embedding, multimodal, pure and coding LLMs are Chinese (attested and not just benchmarks).
68.
by Vetch 3y ago
For a drone, an LLM derived solution is far too slow, unreliable, heavy and not fit for purpose. Developments in areas like optical flow, better small CNNs for vision, adaptive control and sensor fusion are what's needed. When neural n
69.
by Vetch 3y ago
An AI that would be like an Illustrated Primer or the AIs from Fire Upon Deep is the dream from which we are currently far, doubly so for open source models. I wouldn't trust one with a sauerkraut recipe, let alone the instructions for
70.
by Vetch 3y ago
This issue is overstated. Given the material is in Chinese, it stands to reason the bulk (but of course not all) of it will not be in violation of whatever policies. Furthermore, there have been a series of open-weights Chinese models that
71.
by Vetch 3y ago
Let's assume that these LLMs are very useful and provide boosts to their users. It follows that anyone not leveraging them where applicable would be at an economic disadvantage. Without opensource models, everyone would have to pay a G
72.
by Vetch 3y ago
It makes sense to make a purchase decision based on others opinions yes, but what I don't understand is why sillysaurusx seems to have already decided based on one response, and without knowing how (dis)similar Drybones's taste in
73.
by Vetch 3y ago
One of the best games ever made, Outer Wilds, was done in Unity. Dyson Sphere Program is an excellent factorio style game that's very well optimized. I do not share your same trepidation on seeing the logo (doesn't move me in any
74.
by Vetch 3y ago
How does developing another engine help at all? It is very likely its graphics are so under-optimized because they spent most of their time on the simulation aspect, which is more than challenging enough. Dyson Sphere Program is a game with
75.
by Vetch 3y ago
Ah you're right. That makes sense. The autocomplete, informal proofs, translation or autoformalization, reference, search, feedback interactive assistant use-cases do seem promising though.
76.
by Vetch 3y ago
I don't think the right way to think about its utility is as a replacement. I see it in terms of a NNUE for Stockfish type augmentation, where a small neural network supercharges search. Small neural network because no LLM, not even GP
77.
by Vetch 3y ago
Don't forget that Vannevar Bush also inspired Douglas Engelbart (it saddened him that Bush would not accept his and his team's digital computer innovations). The Mother of All Demos, by Engelbart's Augmentation Research Cente
78.
by Vetch 3y ago
We are not anywhere near 160 IQ assistants, otherwise there'd have been a blooming of incredible 1-person projects by now. By 160 IQ, there should have been people researching ultra-safe languages with novel reflection types enhanced b
79.
by Vetch 3y ago
Knowledge is power true, but even more powerful and rare is tacit knowledge. A vast collection of minor steps that no one bothers to communicate, things locked in the head of the greybeards of every field that keep civilizations running. It
80.
by Vetch 3y ago
My opinion is that if you're already well into a project, it makes far more sense to donate to Godot's development and remain in unity while also developing as much as possible in an engine agnostic manner. Moving to Unreal makes
81.
by Vetch 3y ago
Something to realize is that different models require different prompting styles. You can't prompt non-gpt4 models with GPT4 tuned stylistic ticks and expect similar results. I've gotten great performance from llama2 derivatives.
82.
by Vetch 3y ago
That is stretching arguably too far. If you are taking 1 sample path, you are not in any meaningful sense searching a tree. In the context of sampling a probability distribution, which is what LLMs do in effect, there is extra depth to this
83.
by Vetch 3y ago
Whether or not backtracking is needed is really down to the grammar's ambiguity. The auto-regressive nature of LLMs is actually something that counts against them, at least as some tell it. Although, really, the root problem is generat
84.
by Vetch 3y ago
idbfs has posted this link already but did not explain that Shalizi provides a deep theoretical explanation for why universal source coding (does not require information about symbol distribution or statistics) such as Lempel ziv derived co
85.
by Vetch 3y ago
Neural network weights are better viewed as source code because they specify what function the network computes. As we're operating purely on feed-forward networks, there are no loops. Therefore, weights fully describe everything relev
86.
by Vetch 3y ago
The tokenizer differences are major as LLMs are sensitive to whitespace handling. If I am reading the github page properly, OpenLLama failed to learn how to model code properly? Code contains many implicit reasoning tasks. What other differ
87.
by Vetch 3y ago
> Meh it's poorly supported by both PyTorch and TF. This does not match my experience. Most new model architectures port fine to ONNX from pytorch, only occasionally having to fill in rare functions. > Not even by a long-shot - f
88.
by Vetch 4y ago
Base model performance is what's most important and also impacts fine-tuning quality. Practically, a model that's good out of the box with minimal fine-tuning is also useful to more people. Since they focused on being training com
89.
by Vetch 4y ago
This is interesting. What sizes are you seeing this for?
90.
by Vetch 4y ago
It's not just that it's accessible, it's also significantly higher in quality than previous local runnable causal LMs. I suspect people saying it's not good are prompting it like ChatGPT, not realizing how much trickier

More ›