5 ms·

I've been at two startups that have done genuine world first fundamental research. The first tried to publish novel results for 3 years in tier 1 journals befo

• NetOpWibby 2mo ago
  Can you share a link to the preprint?
  • noosphr 2mo ago
    https://www.biorxiv.org/content/10.1101/2021.12.02.471005v2.full https://www.biorxiv.org/content/10.1101/2021.12.02.471005v2....
    • NetOpWibby 2mo ago
      Sweet, thanks! The intro reminds me of Tron: Ares, when Ares was put through endless battle training cycles until he was a solider. Fascinating stuff.
• eikenberry 2mo ago
  Why wouldn't they just skip the journals and publish the papers themselves? Sharing the research is the important part unless these are academics who are trying to get tenure, grants or such.
  • Forgeties79 2mo ago
    Same reason movies released on YouTube aren’t treated with the same respect as those released in theaters.
  • noosphr 2mo ago
    The article is complaining that research isn't peer reviewed and research is turning into blogs. I'm explaining why that's the best outcome possible and why in a field as hot as large dl models you won't even get that.
    • eikenberry 2mo ago
      I thought parent post was saying that they didn't publish them at all after they had so much trouble with the journals. If they did publish the papers themselves then I don't see the problem. The article itself read to me as your standard journal rent seeking and I wasn't really responding to it.
      • fluoridation 2mo ago
        >Why wouldn't they just skip the journals and publish the papers themselves? Sharing the research is the important part To answer your question, sharing the research isn't the important part. These are researchers in the private sector, in a highly competitive field. Arguably, doing the research isn't the important part either. The important part is making money, and the research is a means to that end. Publishing the research doesn't further that goal, and, according to GP, it's so difficult that it may not be worthwhile trying.
  • londons_explore 2mo ago
    For the individual employees, having your name on a peer reviewed paper in a prestigious journal makes you far more employable elsewhere. Moreso if your current employer is a stealth AI startup which hasn't yet produced much notable. It also can encourage employees to work for fewer wages... 'If you do great work you can publish and then get a $$$$$$ job offer at OpenAI, or you can stay here and get equity in a fast growing startup. If however your work doesn't do well, you walk away from a bankrupt startup with worthless shares and minimum wage'
    • sieabahlpark 2mo ago
      [dead]
• cortesoft 2mo ago
  > That and avoiding openAI and Anthropic copying our results and leaving us with nothing to show for six months of work. Isn't the POINT of publishing research because you want others to copy it?
  • willmadden 2mo ago
    Anthropic and OpenAI have proven themselves to be aspiring monopolists through their positions and actions. Why would you want to share your research with them?
    • deleted 2mo ago
      [deleted]
  • notRobot 2mo ago
    And crucially, you should want to others to build upon it, like you build upon so many that came before you.
    • sigmoid10 2mo ago
      If your perception of a moat is worth more than the GDP of most countries, publishing anything must seem like a bad idea. Much less helping others build upon your work.
  • xienze 2mo ago
    There seems to be this mentality among a large number of people who publish open data/research/software that everything is fine as long as no one else makes money using it. The second that happens, they're furious and act as though someone unfairly profited off of their "unpaid labor" (read: that thing they did purely for the love of the game). I think it has to be some sort of feeling of resentment and/or stupidity that _they_ were the person with a million dollar idea that let it slip through their fingers.
    • fluoridation 2mo ago
      It's not that difficult. You invest effort into research in the hope it'll give you a competitive edge. If someone else can come along and reproduce your results practically at a much lesser cost, then you lose that competitive edge. You're back at square one, except you've lost the cost of the research.
• ACCount37 2mo ago
  "Drop a preprint like it's a mic" is ML's new "publish".
• fooker 2mo ago
  Publishing CS papers at the top venues requires using in-group language and formalism that is pretty much inaccessible to someone who has not done a PhD in that specific narrow field. LLMs are pretty good at this, hilariously. This means a genuinely good paper by an outsider has significantly less chance of getting good reviews than AI slop.
  • DonHopkins 2mo ago
    If an LLM can present good research in the field's required dialect better than a human outsider can, calling the result "AI slop" tells us nothing about its quality -- only about your prejudice against how it was produced. And when that prejudice degrades the quality of your own thinking and writing, you're producing "human slop", which is considerably more tragic: you ought to be capable of doing better than the AI. You're holding it wrong.
    • phoghed 2mo ago
      > genuinely good content, dismissed without deep engagement because it didn’t use the correct in group language, would have otherwise passed muster > genuinely bad content, dismissed after deep engagement You have to be misreading the GP on purpose. Maybe feed it to your LLM of choice next time before you rant at someone. The crux of their point is that outsiders have no mastery of this supposed specialized dialect, and LLMs do.
      • fooker 2mo ago
        Exactly right. Not only is it specialized, it also has its own field specific memes* and trends that evolve over time. The purpose is mostly to signal "I'm one of you". *memes as in the actual meaning of memes, not internet memes.
    • fooker 2mo ago
      I think, maybe you misunderstood my comment? I'm all for LLMs producing good research. That's obviously happening right now. What I said was people with good research being penalized for not knowing in-group language. It's not a zero sum game.
• jandrewrogers 2mo ago
  There is almost no benefit in publishing frontier research as a startup. People can try to argue it but it is not defensible. Publishing that kind of thing is a flex that companies risking nothing can do. Cool for Google. Problematic if you are a startup.
  • make3 2mo ago
    I think talent would definitely take a startup that publishes more seriously, and it would be a smaller risk for top research talent to go somewhere where they can also publish than have to bear the full risk of their work at a no name startup not going anywhere
  • mkolodny 2mo ago
    Google has been publishing frontier research since it was an early-stage startup (https://research.google/pubs/the-anatomy-of-a-large-scale-hypertextual-web-search-engine/ https://research.google/pubs/the-anatomy-of-a-large-scale-hy...).
    • silver_sun 2mo ago
      Looks like that was published in 1998, but that's also the year they submitted their patent application for PageRank. So you couldn't use it because they didn't want to risk losing their budding search business to, say, Microsoft. In later years Google generally published the details of a system after they moved to the next one, to stay ahead of their competition, or if they didn't see it as important to their business (most infamously the Transformer language model). It was still generous of them to share their research at all though, and this allowed their scientists to talk publicly about their research which is a huge plus.
  • ozgrakkurt 2mo ago
    You can define benefit in a lot of different ways. Not everything is measured in money.
  • dgellow 2mo ago
    Could be a good way to attract talents. Could be a good way to establish your name for a potential acquisition
• marcus_holmes 2mo ago
  Is this the "AI startups blog about discoveries rather than submitting to journals" phenomenon that TFA noted? Are journals just too slow, hidebound, and predatory to be useful to this industry any more?
• YeGoblynQueenne 2mo ago
  Tier 1 journals have high standards (that's why they're tier 1) and they also have hundreds of submissions every cycle. It's common to have a paper rejected. On the upside you get your work seen by some of the most experienced researchers in AI, who are also specialists in your paper's subject. For example, here's the editorial board of the Journal of Machine Learning Research: https://jmlr.org/editorial-board.html https://jmlr.org/editorial-board.html Scroll down where it says "JMLR Editorial board of reviewers" and where you can find the names of reviewers and their areas of expetise. The advice you get from such reviewers, even if your paper is rejected, is an invaluable tool to help you improve your work and its presentation. You eventually learn to not be sore about it and iterate until your work gets accepted. Of course nobody's forcing you to publish, especially in a journal (in AI and CS conference proceedings tend to have much quicker turn around times and you naturally get more feedback if you get to present at a conference) but the alternative is to sit alone in your room hacking away at your code until you discover the perpetual motion machine.
  • okaleniuk 2mo ago
    The alternative is to use your network directly. The very notion of "I'll publish my work and someone will notice me" is wrong. Most people in academia are too concentrated on their own work to care about yours. The only minority that cares is already in your contact list.
    • YeGoblynQueenne 2mo ago
      Not always. AFAICT my papers have often been reviewed by people I have never met and who have expertise that overlaps with, but does not fully match, mine, and I couldn't possibly contact those people out of the blue to ask for feedback, particularly because I'd never get the chance to meet them outside of peer review (they wouldn't be in the same conferences as me etc). And that's a good thing because that way I get a sort of "outsider looking in" view. I've sure given and taken informal feedback in the past from people I know and that's great but peer review also makes the whole process more formal and less sloppy; at least sometimes and depending on the venue and to my experience.
• m000 2mo ago
  May I ask how did you try to publish, and if you have done this before? Tier 1 is HARD. Universiy labs (where publishing is their bread and butter) and have multi-year streaks of Tier 1 papers, still seek out collaborations when publishing. Because you still need this extra angle on your work that will make it stand against the proverbial Reviewer 2, or that extra evaluation paragraph that will make it stand out. And don't forget that each Tier 1 publication is probably the life of at least 1 PhD student for several months. So if you don't put the time and effort in it, forget Tier 1. This is like believing that because you have excellent voice and singing skills, all you have to do to get a top 10 hit is to send your demo tape to a recording company.
• accurrent 2mo ago
  This. The circus that is academic publishing is just not worth the effort. Ultimately, companies do have to make money.
• tanker_war 2mo ago
  The publication model in academia is predatory and elitist, but have you considered that your paper just wasn't that good? Almost every scientific paper published in every field is "world first" research, and the job of tier 1 journals is to decide what deserves to be signal boosted, and what doesn't. This response to rejection to me confirms the fears implicit in the article linked above. Not only is private AI research not transparent and not documenting its work, it also has a deep rooted culture issue.