3 ms·

can someone explain why the AI or language model community circles around arxiv? I really hate the pseudo-academic gatekeeping in the AI/ML community, Google s

can someone explain why the AI or language model community circles around arxiv? I really hate the pseudo-academic gatekeeping in the AI/ML community, Google said you have no moat, we all know you have no moat, including that degree. we can all fine tune with consumer hardware we already have or even better cheaply on readily accessible clouds for this specific purpose. why are they still doing this fake academic junk.

SimplyUnknown 3y ago
I mean, you can't just share the weights of the model and call it a day, right? You have to share details on what and why you are doing. You must communicate this somehow. In theory, you might be able to do this in a github readme, but a paper-style document on arxiv is nicely suited for this.
- yieldcrv 3y ago
  > I mean, you can't just share the weights of the model and call it a day, right? you can't?
  - SimplyUnknown 3y ago
    Obviously you can, but in the grand scheme of things people should share more details about their method so people can improve on it in the future, no?
- selfhoster11 3y ago
  People release models as just the weights all the time. HuggingFace makes it pretty easy to do that.
deleted 3y ago
[deleted]
selfhoster11 3y ago
I am very confused by this comment. There is no gatekeeping in the ML/AI community. Ideas flow freely (albeit within the confines of several major Discord servers, or so it seems). Whether the author of an idea has formal training in ML and adjacent disciplines or not, whether it's published on arXiv or not, it doesn't matter - it'll be adopted if it works and/or makes it easier for people to run their GPT waifu/ baby AGI prototype. That said, new open foundation models sized 7B and over are still a fairly rare thing to see. If someone goes through the effort of creating one of those, and especially if it has some sort of an edge against Llama 2 7B, it's not unreasonable to expect an arXiv paper to be released about it.
- LoganDark 3y ago
  Isn't token completion incapable of representing AGI? AGIs need the ability to perform internal thought and deliberation, and they probably also need to be bottom-up rather than top-down.
  - stavros 3y ago
    You can't judge whether something is AGI or not from how it works, that just leads to goalpost-moving. AGI is AGI if it can do certain things, no matter whether it's token-based or top-down or anything.
  - dragonwriter 3y ago
    > Isn't token completion incapable of representing AGI? Given the absence of a validated model (or even usable operational definition) of general intelligence, who knows? AGI might as well be an empty marketing buzzword, it isn't something about which falsifiable fact claims can be made. > AGIs need the ability to perform internal thought and deliberation Systems built around LLMs can do something like this, via reflection, a technique in constructing agent-based simulations using LLMs as the engine.
  - selfhoster11 3y ago
    I see what you mean. In this case, I was talking about the LLM being a component of the AGI rather than the whole AGI all by itself.

SimplyUnknown 3y ago

I mean, you can't just share the weights of the model and call it a day, right? You have to share details on what and why you are doing. You must communicate this somehow. In theory, you might be able to do this in a github readme, but a paper-style document on arxiv is nicely suited for this.

yieldcrv 3y ago
> I mean, you can't just share the weights of the model and call it a day, right? you can't?
- SimplyUnknown 3y ago
  Obviously you can, but in the grand scheme of things people should share more details about their method so people can improve on it in the future, no?
selfhoster11 3y ago
People release models as just the weights all the time. HuggingFace makes it pretty easy to do that.

deleted 3y ago

[deleted]

selfhoster11 3y ago

I am very confused by this comment. There is no gatekeeping in the ML/AI community. Ideas flow freely (albeit within the confines of several major Discord servers, or so it seems). Whether the author of an idea has formal training in ML and adjacent disciplines or not, whether it's published on arXiv or not, it doesn't matter - it'll be adopted if it works and/or makes it easier for people to run their GPT waifu/ baby AGI prototype. That said, new open foundation models sized 7B and over are still a fairly rare thing to see. If someone goes through the effort of creating one of those, and especially if it has some sort of an edge against Llama 2 7B, it's not unreasonable to expect an arXiv paper to be released about it.

LoganDark 3y ago
Isn't token completion incapable of representing AGI? AGIs need the ability to perform internal thought and deliberation, and they probably also need to be bottom-up rather than top-down.
- stavros 3y ago
  You can't judge whether something is AGI or not from how it works, that just leads to goalpost-moving. AGI is AGI if it can do certain things, no matter whether it's token-based or top-down or anything.
- dragonwriter 3y ago
  > Isn't token completion incapable of representing AGI? Given the absence of a validated model (or even usable operational definition) of general intelligence, who knows? AGI might as well be an empty marketing buzzword, it isn't something about which falsifiable fact claims can be made. > AGIs need the ability to perform internal thought and deliberation Systems built around LLMs can do something like this, via reflection, a technique in constructing agent-based simulations using LLMs as the engine.
- selfhoster11 3y ago
  I see what you mean. In this case, I was talking about the LLM being a component of the AGI rather than the whole AGI all by itself.