4 ms·
Source?
by refulgentis 3y ago
Source?
- ShamelessC 3y agoJust theorizing from the top-level post here. No clue if it's legitimate.
- toxik 3y agoTo be clear, you just made up MoE details while MoE is actually well established and hails from decades old research?
- natpalmer1776 3y agoThis is common behavior for inference based learners who don’t hail from strong academic backgrounds. Many developers who are self taught utilize a similar method of learning, essentially using pattern recognition to make “educated guesses” that are then internalized as potential facts and tested at the earliest opportunity. In this instance the test was to project the incorrect information out onto a public forum containing experts and using the presence of contradiction as weak evidence to the validity of their newfound knowledge. Yes, this is done in lieu of actually looking up extended details on what something means. It has it’s advantages though.
- reaperman 3y ago> It has its advantages though. Seems the advantage is somewhat localized to the individual inference-based learner; it doesn't seem like a pro-social strategy which would optimize benefit to the group. Overall this seems like it would generalize to widespread misinformation if the majority of uses adopted this behavior. I'm guessing it's in the best interests of the wider group to try to minimize the occurrence of this type of participation.
- natpalmer1776 3y agoThe advantages in a social setting lie in the introduction of entropy, that is _creativity_, to a community. In a rigorous academic setting and with proper training these individuals are more likely identify links between ideas or information that may not seem obvious at first, and tend to be your more 'eccentric' academics. For the interests of the wider group, the best outcome is to help these individuals refine their communication to make it clear when information they present is unsubstantiated inference as opposed to verified knowledge. Once the proper 'rules of engagement' are outlined the contributions of these individuals is an oftentimes useful 'ingredient' to the success of many enterprises.
- chaos_emergent 3y agoGiven that this is an online forum, another advantage is that a conversational trail is left for others to discover. The inferences these types of individuals make are often based on a structure of knowledge and reality that others share, so the most common preconceived and incorrect notions tend to have the most documentation on how to ameliorate the incorrectness (given that these individuals are allowed to state their inferences out loud).
- doctor_eval 3y agoThis had got to be the best thread I’ve ever (inadvertently) started.
- refulgentis 3y agoIts a good thread, particularly Exuma's comment, but a memento mori from the root node: My one word "Source?" meant: "You are adding a new episode of fictional info to a discussion about fictional info." Even in that fictional world the post was wrong. Even if it was right in some allegorical sense, the simplistic allegory adds nothing. "Mixture of experts is like a group of experts where you pick the right expert to ask a question" isn't some hard-won cross-domain self-taught knowledge. It's something a bright 6th grader would pull off. The self-taught feeling-stuff-out stuff matters when you're making useful connections that get practical results. When you're just wiring stuff together online, and the wiring together is meaningless, you're doing nothing and taking the consequences of the negative signals
- jstarfish 3y ago> Yes, this is done in lieu of actually looking up extended details on what something means. If they were capable of understanding the extended details, they would already have an academic background in the subject. Laymen aren't going to have a clue what MoE means even if they went to the trouble of digging up the paper. > Many developers who are self taught utilize a similar method of learning, essentially using pattern recognition to make “educated guesses” that are then internalized as potential facts and tested at the earliest opportunity. Using pattern recognition skills to make an educated guess that is internalized as a potential fact sounds an awful lot like what LLMs do. At least when humans do it, we bother with the verification step instead of just acting like we know what we're talking about.
- jprd 3y ago> If they were capable of understanding the extended details, they would already have an academic background in the subject. Laymen aren't going to have a clue what MoE means even if they went to the trouble of digging up the paper. This is, hopefully, an accidental thought experiment gone awry. "IF THEY WERE CAPABLE of understanding the extended details, they would already have an academic background in the subject" can and should == "I spent a ton of time in the library", and an follow-up apology for putting "capable" and "academic background" in the same sentence. The whole friggin' point of this glorified LAN is that we can break down those dumb walled gardens and let kids learn from random BBS textfiles, MIT YT videos and the gathered wisdom of HN. If you are going to just dismiss auto-didacts, you're going to have to re-write the complete higher education History in Western society. I won't even begin to try and validate how wrong this is for Eastern History as well.
- mvkel 3y ago> At least when humans do it, we bother with the verification step instead of just acting like we know what we're talking about We do??
- lobocinza 3y agoWe ask Google (and now ChatGPT) if it's true. It goes round.
- ShamelessC 3y agoI don’t appreciate the many bad-faith assumptions made. in particular the assertion that I’m not from a “strong academic background”. For what it’s worth, I made a mistake in a public forum, admitting to this twice. I’ve sought accurate versions of my response which no one provided. Nevertheless I have continued to educate myself on the subject and only feel more confident that I wasn’t misinformed to the degree you all indicated. You may not realize it but not everyone on the internet is nefarious and if you were to speak in this analytical way about say a classmate while they were in ear shot - that person would likely be quite upset.
- natpalmer1776 3y agoI take it all back. The comment I responded to appears to be both correct in assertion and implication.
- ShamelessC 3y agoOkay then. Best of luck with all of that. There’s a forum called LessWrong you would likely be interested in.
- natpalmer1776 3y agoI'm not the biggest fan of LessWrong, I do however refer new developers who seem interested to the "Rationality: From AI to Zombies" sequences to help refine conscious development of rational thinking. Also, I absolutely loved the fanfic "Harry Potter and the Methods of Rationality" written by the forum's creator. ** Edit ** And for the record, my comment wasn't intended to be an accurate depiction of you specifically, which honestly wasn't very effectively conveyed. It was to highlight a common 'type of person' who make authoritative statements on areas they're not 'experts' in, through no malice on their part, rather as a function of their default mode of behavior. As others who replied to me highlighted, there are at least two of these people in the world and I wanted others to at least be aware of their existence and point of view; the end goal of this being they might offer others the benefit of the doubt and perhaps some constructive feedback instead of unproductive criticism in similar exchanges. In essence, I thought other comments were being too hard on you and wanted to point out a potential scenario in which their critiques were at at best unproductive.
- drumttocs8 3y agoAre you saying that this was a form of Socratic questioning- intentionally presenting an incorrect statement in order to obtain the correction?
- natpalmer1776 3y agoA bit like that, but without prior knowledge of whether the information is incorrect, rather an intuition that it is correct.
- toxik 3y agoSorry but that is just ludicrous. You do not answer a factual question with a mostly made-up answer without saying clearly "this is pure speculation" at some point, preferably early on.
- natpalmer1776 3y agoWhen I was younger (hah, I'm only 26 now) I sure did do exactly what you say. If your statement is intended to say that "People should not" then you're absolutely correct, however it is a learned behavior that some people must adopt after being corrected by their peers.
- deleted 3y ago[deleted]
- ShamelessC 3y agoCan you detail what mistakes were made? I’m brushing up on it currently but having trouble grokking it.
- rdlecler1 3y agoI believe it’s called ‘hallucination’
- drumttocs8 3y agoI would have presented it as a question versus a statement, in that case.