12 ms·

Why there isn’t an Apache Arrow article in Wikipedia

111 points by riboflavin 7y ago

deleted 7y ago
[deleted]
ggggtez 7y ago
I've never heard of it. Add my vote to removing the article. Cry more, company I never heard of either.
mistrial9 7y ago
sadly, I can jump in on the "Wikipedia fails" train here, also. In about five attempts to really change an article (different ones) in about five years, every single change was rejected, as far as I know. The changes were different, one was writing style and order of facts on a public historical event in this century; one was adding a lot of detail to the description of a popular fantasy fiction series; one was removing a controversial and provocative one-liner at the top of a page about people at the edge of (western) society; and another .. hmm I forget now, because I just gave up ! My aging colleague tells me, just keep doing the changes, they cant stop everything. However, my direct (and limited) experience is.. they do stop everything (that I try). I was logged in twice and used anonymous three times, and added citation a bit, too. To the point of the article, FOSS projects in wikipedia ? hmm maybe there could be a clear category for that ? software projects are proliferating rapidly.. dunno
- wffurr 7y ago
  Did you read the reasons for rejection and try to modify subsequent submissions to better comply with Wikipedia's published guidelines?
  - mistrial9 7y ago
    yes, I did, and I feel that this revert behavior was more hazing/article control than substantive in all cases but one, and that one I dont personally agree.
    - ErrantX 7y ago
      Have you got an example?
- zozbot234 7y ago
  The way around mindless reverts is to first detail the proposed change on the talk page, then wait for anyone interested to object. If no one does, you can make the change live, pointing out that no objections were raised. Even if someone does object, such objections should ultimately be made actionable, i.e. it should be made clear how to address them to the other party's satisfaction.
tetromino_ 7y ago
See https://en.wikipedia.org/wiki/Wikipedia:Notability https://en.wikipedia.org/wiki/Wikipedia:Notability - all you need to show is that Apache Arrow has received significant coverage in reliable sources that are independent of the subject. So: find conference papers/talks by people not affiliated with Apache or the Apache Arrow project and that discuss Apache Arrow. Figure out how to incorporate the tidbits about Arrow from those papers into the article text. Add sources in footnotes. Done.
- dajohnson89 7y ago
  And to be clear -- this is the job of the article author, not the editor who rejects the article without doing a little bit of homework first?
  - tetromino_ 7y ago
    Of course. Similarly, when you are submitting a PR to an open source project or a manuscript to an academic journal, it's your job as the author to take note of contributor guidelines.
  - skywhopper 7y ago
    Yes. The author is supposed to cite the sources.
  - wolfgang42 7y ago
    Yes; Wikipedia's expectation is that authors have researched the topic about which they are writing, and therefore they are in the best position to provide the sources from which they got their information. The editors' job is to ensure that Wikipedia's standards are met, not to re-do the research that the author should already have performed. If the case is that no research was performed because the author is already an expert in the area, they are still expected to provide citations so that the same standard can be applied to all authors.
- sb057 7y ago
  A version which was rejected included the following (non-Apache-affiliated [afaik]) references: https://www.xenonstack.com/insights/what-is-apache-arrow/ https://www.xenonstack.com/insights/what-is-apache-arrow/ https://link.springer.com/chapter/10.1007%2F978-1-4842-1311-7_5 https://link.springer.com/chapter/10.1007%2F978-1-4842-1311-... https://www.biorxiv.org/content/biorxiv/early/2016/08/23/071092.full.pdf https://www.biorxiv.org/content/biorxiv/early/2016/08/23/071... http://delivery.acm.org/10.1145/3110000/3103003/p138-Maas.pdf http://delivery.acm.org/10.1145/3110000/3103003/p138-Maas.pd... https://www.theregister.co.uk/2016/02/17/apache_arrow_toplevel_project/ https://www.theregister.co.uk/2016/02/17/apache_arrow_toplev... https://www.cio.com/article/3034279/big-data-gets-a-new-open-source-project-apache-arrow.html https://www.cio.com/article/3034279/big-data-gets-a-new-open... https://www.infoworld.com/article/3033446/hadoop/apache-arrow-aims-to-speed-access-to-big-data.html https://www.infoworld.com/article/3033446/hadoop/apache-arro... https://sdtimes.com/apache/guest-view-first-release-apache-arrow/ https://sdtimes.com/apache/guest-view-first-release-apache-a... https://www.infoq.com/news/2016/12/le-dem-apache-arrow/ https://www.infoq.com/news/2016/12/le-dem-apache-arrow/ http://dbmsmusings.blogspot.com/2017/10/apache-arrow-vs-parquet-and-orc-do-we.html http://dbmsmusings.blogspot.com/2017/10/apache-arrow-vs-parq... https://dbmsmusings.blogspot.com/2017/10/apache-arrow-vs-parquet-and-orc-do-we.html https://dbmsmusings.blogspot.com/2017/10/apache-arrow-vs-par...
TallGuyShort 7y ago
Having worked on commercially-resold Apache projects, can't say I argue with Wikipedia a whole lot on this. It seems to me it should be let in, but it's a bit silly to go and call them out like this on a corporate blog, IMO. Dremio does benefit from Apache Arrow publicity and notoriety, even if they don't profit directly. Having a de-facto standard data format and open-source engines is a selling point for some. That's why Dremio explicitly calls it out on their own website. It also never hurts in the recruiting department. (edit: there's a reason the article was submitted by someone working in marketing & strategy) >> I’m wondering if Wikipedia can continue to be considered a reliable source of information for technical folks who want to learn more about the vast system of Apache open source software projects. Sign up for the Olympics, because that's a hell of a leap. You didn't get your page in, it's really not much of a reflection on the rest of Wikipedia. It's an open-source project. It should have it's own freely available documentation that fills much the same purpose anyway. If I want to learn about Apache X, I go straight to x.apache.org. They concede that it's not an end-user product anyway, so I'd think their key audience knows how to find an open-source project website. Lower the bar too far the other way, and there are plenty of semi-open-source project's marketing departments would be all over using Wikipedia to their own ends - I've seen my own former employer do this for their Apache projects.
- duskwuff 7y ago
  > I’m wondering if Wikipedia can continue to be considered a reliable source of information for technical folks who want to learn more about the vast system of Apache open source software projects. I'm confused why the writer thinks it should be! The Apache Foundation is a big tent. There's some clearly notable projects in there (like Apache httpd), but there's also a lot of really obscure crap that basically nobody outside ASF cares about (like Apache Creadur or Apache Pony Mail). Expecting Wikipedia to document every Apache project is ridiculous. Is this particular project notable enough for a Wikipedia article? I don't know a lot about it, so I can't say for sure. But the article drafts that I've seen don't convince me that it is.
  - lallysingh 7y ago
    > Expecting Wikipedia to document every Apache project is ridiculous. Wikipedia has a lot of rather obscure entries. Long, long lists that I think easily under-rank the Apache project. I'm not saying you're wrong, but the bar for notability is kinda vague. Lists of every episode of a series, every kind of kim-chee, etc.
oefrha 7y ago
https://en.wikipedia.org/wiki/Draft:Apache_Arrow https://en.wikipedia.org/wiki/Draft:Apache_Arrow > REVIEWERS: Please note that the submitting editor is the chief marketing officer and vice president of strategy at this company. Yeah, sorry, big no no there. Disclosure: consider myself a Wikipedian to some extent, got a couple hundred edits on Wikipedia.
- foota 7y ago
  This is wrong though, Apache Arrow is an open source spec, the fact that they happen to work at a company working on it seems secondary.
  - oefrha 7y ago
    1. Conflict of interest has nothing to do with whether it’s open source. If I submit a Wikipedia article about my completely uncommercial personal project it’s still a huge conflict of interest. Rule of thumb: don’t submit Wikipedia articles about yourself, your project, your employer or your employer’s project. If it’s notable enough eventually someone without a conflict of interest will do it. 2. Tons of companies use open source for marketing. This one is no different as far as I can tell. Even had the chief marketing officer submit a Wikipedia article for their project.
    - mxfh 7y ago
      The problem with these rules are, that they are so selectively enforced, it is a farce. They selectively assume bad faith, where, by all objective means is none, and brush over others, as long as it is backed up by random arguably non-neutral publications. It's near impossible to put article's up over prolific female journalist for example, because all this can be enforced since, the publications are all from the same source (the publisher they work for) or are interviews or some sort of talk or award, where it's almost never for men, which get away* by linking to a podcast. tldr: *WP:OTHERSTUFFEXISTS
    - foota 7y ago
      But it's not _their_ project, it's a project they contribute to. Google contributing an article about k8s would be a completely different thing from Google contributing an article on say Hadoop.
mxfh 7y ago
It was a pain to get gitlab in 5 years ago after a "controversial" deletion, so it wasn't available for simple undeletion. Domain specific knowledge has it notouriously hard with wikilawyers who, at large, seemingly stopped adding new things to their world view 15 years ago. Then it becomes a game of jumping through hoops and hoping you end up with a kind wiki-landlord or knowing a friendly wikipedia admin. Doing the latter by anouncing your concern on social media and hoping a sympathetic admin picks it up, might be the easiest on human time and resources, just let them copy your reasonably well sourced article draft from your personal space and see what happens.
- sien 7y ago
  At one point about 10 years ago the entry for Atlasssian was deleted for not being notable.
  - Geimfari 7y ago
    It was deleted in 2005 for being a blatant advert, and again in 2010 for the same reason. I doubt it was actually a notable company in 2005. [1] https://en.wikipedia.org/wiki/Wikipedia:Articles_for_deletion/Atlassian_Software_Systems https://en.wikipedia.org/wiki/Wikipedia:Articles_for_deletio... [2] https://en.wikipedia.org/wiki/Special:Log?type=delete&page=Atlassian&wpdate=2010-01-19 https://en.wikipedia.org/wiki/Special:Log?type=delete&page=A...
    - yellowapple 7y ago
      2005 was 15 years ago, not 10.
    - mxfh 7y ago
      Even if it was a clumsy self-promotion or over-ambitious fans with no clue on wikipedia inner mechanics shouldn't set back a viable interest on information about a given company or other entitity by a multitude of years. After a deletion it's just magnitudes harder for anyone to get an article restored, compared to an entity which didn't have the "luck" to get added to wikipedia too early. Deletion history shouldn't have that much of a say on actively developing entities as it has now.
Ninjaneered 7y ago
Here's the link to the draft: https://en.wikipedia.org/wiki/Draft:Apache_Arrow https://en.wikipedia.org/wiki/Draft:Apache_Arrow And some possible additional sources: * https://www.forbes.com/sites/forbestechcouncil/2019/09/24/dremio-helps-reduce-proximity-to-your-data/amp/ https://www.forbes.com/sites/forbestechcouncil/2019/09/24/dr... * https://www.businesswire.com/news/home/20180906005114/en https://www.businesswire.com/news/home/20180906005114/en * https://thesiliconreview.com/2016/02/apache-arrow-is-the-new-open-source-project-for-big-data https://thesiliconreview.com/2016/02/apache-arrow-is-the-new...
- tptacek 7y ago
  The first article is a paid promotion piece, which WP won't accept as an RS. The second is a press release by Arrow's sponsoring company, which, obviously, WP won't accept as an RS. I have no idea what "The Silicon Review" is; this is the first time I've ever seen it. To the extent it's not a pay-to-play trade publication, it might qualify as a notability-establishing source. The fact that the "Review" does not itself have a WP page might make it harder to claim it's reliable, since it suggests nobody else knows what it is, either.
  - Ninjaneered 7y ago
    Looks like my lateral reading was sub-par (actually I didn't even try, just a quick Google/post). The "Silicon Review" one looks like a pay-to-play as well after further review, it's used in citation on a few other Wikipedia articles, but as far as I can tell, and due to some anecdotal stories, it doesn't look good. * https://www.reddit.com/r/PublicRelations/comments/bha6hs/silicon_review_emailed_me_about_an_offer_to/ https://www.reddit.com/r/PublicRelations/comments/bha6hs/sil... * https://arpr.com/blog/4-pay-for-play-scams/ https://arpr.com/blog/4-pay-for-play-scams/ Good catch, thanks for spending the time to review my links. Reading your comments above, I largely agree. It's a high bar (mostly) to get an article on Wikipedia, and that's a good thing. It allows us to read the majority of content on Wikipedia without too much suspicion.
- SquishyPanda23 7y ago
  I read the draft. Maybe this is an unpopular opinion, but it's obvious advertising and has no place on Wikipedia. Maybe a Medium post would be more appropriate. Wikipedia already has a problem with bad software articles like this.
pradn 7y ago
It looks like there aren't enough independent, non-commercial articles to use as references. This is somewhat common for many newish technical projects. Add some academic papers, some usage numbers, some summary blog posts that aren't related to the project. Wiki editors are very suspicious of people from companies editing articles related to their work.
- PeterCorless 7y ago
  I'm not so sure it's falling afoul of 'not-notable' so much as WP:COI.
foota 7y ago
Can HN create a draft that would be accepted? :)
- thebooktocome 7y ago
  Nope, this violates Wikipedia's policy against "meat puppetry".
  - ErrantX 7y ago
    Not at all! The main thrust of that policy is for discussions. Improving Wikipedia by conspiring to write a compliant article is explicitly allowed!
    - thebooktocome 7y ago
      Tell your deletionist editors that.
zeveb 7y ago
One could perhaps be forgiven for wishing that the deletionists would … delete themselves. Seriously, though, bytes are cheap, and an article sitting somewhere in Wikipedia doing nothing and bothering no-one is pretty damned cheap too.
- SpicyLemonZest 7y ago
  I dunno, this kind of thing seems like exactly the canonical argument for deletionism. Maybe there's no cost to a page sitting on Wikipedia describing, like, some guy's special attack from Naruto. There are reasonable arguments that allowing things like that would set a bad precedent and encourage behavior that doesn't help the project, but I admit it's pretty tenuous. There are obvious and important costs if Wikipedia articles start being perceived as promotional material rather than encyclopedia entries.
  - CharlesColeman 7y ago
    > Maybe there's no cost to a page sitting on Wikipedia describing, like, some guy's special attack from Naruto. There is a cost, but it's measured in hours of maintenance labor not bytes of storage. If Wikipedia wants to maintain a semblance of accuracy [1] in the face of declining participation, it needs to concentrate its labor resources rather than spread them out. [1] which IMHO is vital given its unwise prestige as arbiter or truth
    - sgift 7y ago
      Since Wikipedias concentration of labor itself is a source of declining participation[1] it's doubtful that continuing this behavior will result in something else than a death spiral with even fewer people ready to do the work, more concentration, even fewer .. and so on. [1] Among other reasons, more here: https://en.wikipedia.org/wiki/Wikipedia:Why_is_Wikipedia_losing_contributors_-_Thinking_about_remedies https://en.wikipedia.org/wiki/Wikipedia:Why_is_Wikipedia_los...
      - CharlesColeman 7y ago
ForHackernews 7y ago
I've literally never heard of this piece of software, and it's fair to say I'm much more interested in FLOSS than the average person on the internet. Why should this thing have its own article and not just appear in a list of Apache foundation projects?
- m_ke 7y ago
  Because it's actually a pretty big deal for the (python) data science ecosystem.
xibalba 7y ago
As a strategy for getting Dremio on the front page of HN and thus on the radar of a large group of tech people (i.e. Dremio's prospects), this is article is very clever. As a critique of Wikipedia, not so much.
- jessaustin 7y ago
  This seems like a good reason to flag it.
tptacek 7y ago
Open source projects are particularly tricky for Wikipedia. There are tens of thousands of them. Their owners are often passionate. They compete with each other, so there's incentive to write hard-to-adjudicate competing claims. Many have commercial backing, which further warps incentives. The projects themselves are highly technical; many, like Arrow, are software development tools and components. There are few authoritative sources that reliably track open source projects. Keeping up involves directly following bug trackers and message boards and then synthesizing a narrative, which is the definition of "original research", forbidden in the encyclopedia. It's likely that Arrow does deserve a WP article. But Arrow's sponsors misunderstand more about Wikipedia than Wikipedia does about Arrow. Writing a defensible article about their project will require work; in particular, they're going to need to spend the time tracking down authoritative sources for why Arrow is notable, and those claims will probably need to be something more persuasive than "hundreds of companies use it"; hundreds of companies use all sorts of things that don't, and shouldn't, be featured in their own encyclopedia articles. I understand the impulse behind "this project is important; it should have a Wikipedia article". But when you take a step back and accept what Wikipedia actually is, rather than what you think it should be, you're left with the question: do we really need to feature this particular piece of software in its own encyclopedia article? 20 years from now, will people still be getting value from it? Whatever value that might be, will it outweigh the 20 years of other people's volunteer efforts to maintain the article, keeping it free of vandalism and ensuring that it doesn't surreptitiously turn into a promotion piece for some company or another? The answers might be "yes". But I don't see much evidence in this piece considered the questions. Lots of things that don't seem deserving have in-depth Wikipedia coverage. Many of those things probably really don't belong in an encyclopedia! But there are two sides to this problem: the merit of the topic, and the cost, in volunteer time, of including them. A marginal topic can be defensible if it's easy to reliably cover it. A seemingly important technical topic might not be if the only way to say anything interesting about it is to write original research directly into its article. Late edit A useful tip for getting your open source project covered in its own Wikipedia article: don't have the Chief Marketing Officer of the company that owns the project write the article.
- SkyBelow 7y ago
  If a bulldog clip is notable, why wouldn't an open source project that hundreds of companies use not be? https://en.wikipedia.org/wiki/Bulldog_clip https://en.wikipedia.org/wiki/Bulldog_clip s/bulldog clip/many other random office supply items/ Edit: Swapped to bulldog clip as a better example of a less notable office supply.
lmeyerov 7y ago
For context, some other companies contributing to it are in the GPU space, so orthogonal to CPU-centric Dremio: Nvidia, Blazing SQL, and Graphistry (us). Likewise, the pydata big guns intersect a bit here: conda, pandas, ... . This effort got a BOSSIE award for GPU dataframes this year and is taking off now that it is becoming usable for more than just framework devs. The reason we all really on it is because a standardized columnar IO streaming format is an awesome idea for compositional HPC. It does sounds like maybe Dremio's CMO wrote the original articles and it came off centered on them? (Did not have a chance to read.)
- tptacek 7y ago
  Yes: Dremio's CMO wrote the article, and the article was overtly promotional. Of course it got killed.
  - TallGuyShort 7y ago
    And the same guy submitted it here 3 times. Also overtly promotional. riboflavin, genuine suggestion: ask one of the PMC members or committers to rewrite the article from scratch from an engineer's perspective, source everything, demonstrate notability, and resubmit. If they still don't take it, move on with your life. ... but you might have generated a lot of ill-will with the Wikipedia elites here already.
jabvigWe 7y ago
Add it to the Free Software Directory! https://directory.fsf.org/wiki/Main_Page https://directory.fsf.org/wiki/Main_Page
swayvil 7y ago
Wikipedia is administrated by inbred autistics. If it doesn't smell like their own thrice-huffed fartgas they will remove it.
- blattimwind 7y ago
  Now imagine the German Wikipedia, were Exclusionists are 200 times stronger than the Inclusionists, relative to the English Wikipedia.
  - kstrauser 7y ago
    200x? I'm imagining a single page, titled "Nein."
  - yorwba 7y ago
    The German Wikipedia is the fourth largest by number of articles, with one article for 33 German speakers, whereas the English Wikipedia has one article for 84.4 speakers. https://meta.wikimedia.org/wiki/List_of_Wikipedias_by_speakers_per_article https://meta.wikimedia.org/wiki/List_of_Wikipedias_by_speake... Doesn't look like the Exclusionists are really stronger on the German Wikipedia.
jccalhoun 7y ago
>Arrow is designed to serve as a shared foundation for SQL execution engines, data analysis systems, storage systems, and more – think Pandas, Spark, Parquet, etc. Engineers across the community are working together to establish Arrow as a standard for columnar in-memory processing. I like to think I'm fairly techy for a non-programmer but I have no idea what that means. That might be part of their problem if that is the description in their wikipedia entry.
- ggggtez 7y ago
  I believe it's what the kids call "buzzword bingo".
michelpp 7y ago
We're having the same issue getting the GraphBLAS API article to be accepted: https://en.wikipedia.org/wiki/Draft:GraphBLAS https://en.wikipedia.org/wiki/Draft:GraphBLAS. At first it was summarily deleted overnight, now we're stuck in Draft for who know how long.
mikl 7y ago
Why do you care about having a Wikipedia page for Arrow? Why is it important enough to whinge about on HN? Wikipedia is much like Stack Overflow these days, the community has become hostile to newcomers who fail to meet their somewhat arbitrary but very exacting standards for what is allowed on their site. Fortunately, you can just publish your own web site. No need to be bothered about not being on WP.
thrower123 7y ago
The hard-line Wikipedia deletionists should be deleted themselves. The argument is always brought up, like StackOverflow, that they have to be ruthless or it turns into an Eternal September dumping ground of garbage, but the quality is already very uneven and gatekeeping like Cerberus doesn't help further that goal. There's already a toxic Dead Sea effect where the pedantry and politicking has chased out a lot of people that would contribute; who the hell wants to bother putting in some hours writing something up if it is just going to be summarily deleted? Bandwidth and hard drives are cheap. Just spitballing, but it'd be nice if Wikipedia worked a little more like Linux distro repositories. Keep the tightly curated articles in a "core", but leave room for "community" or "nonfree" collections if you want to turn them on.
- CharlesColeman 7y ago
  > Just spitballing, but it'd be nice if Wikipedia worked a little more like Linux distro repositories. Keep the tightly curated articles in a "core", but leave room for "community" or "nonfree" collections if you want to turn them on. I think that's a fantastic idea, especially if it would lead to a drastic reduction in the number of articles served from the main Wikipedia domain (to a number that can meet some reasonable quality and maintenance standard, maybe 10 times the size of the most comprehensive print encyclopedia, or a 1/6 of Wikipedia's current size) [1]. [1] https://newrepublic.com/article/101795/encyclopedia-britannica-publish-information https://newrepublic.com/article/101795/encyclopedia-britanni...: "The 2002 Britannica contained 65,000 articles and 44 million words. Wikipedia currently contains close to four million articles and over two billion words..."
- bjourne 7y ago
  Most communities seem to go that way. In the beginning, most people spend their time contributing first-order content. Then, as the community grows, it attracts more meta-users who are more interested in moderating the content creators. They create ever more rules and policies requiring content creators to jump through more and more hoops. Eventually the experience becomes so frustrating that people give up. Wikipedia seem to me to be in that situation. StackOverflow is on its way there. It has exactly the same kind of problem with "deletionists" that Wikipedia has. Perfectly good questions are often closed for very arbitrary reasons.
qwerty456127 7y ago
Once I witnessed awesome articles [others added and I used with delight] on open source frameworks as well as some minor facts [I added] on other subjects deleted for being "insignificant" I decided I'm not donating to Wikipedia until this bullshit ends. Wiki articles are not videos, they take humble disk space to host so I can't recognize any reason in dismissing "insignificant" information other than a stupid rule. IMHO whatever can be considered a piece of knowledge should be there. BTW nearly the same applies to StackOverflow - thanks to high reputation points I earnt during the early days I can see deleted questions and answers and I often see really interesting (having three-figure upvvote scores and dozens of stars) questions and very informative (also heavily upvoted) answers deleted.
aaron695 7y ago
View the original declined drafts here - https://en.wikipedia.org/w/index.php?title=Draft:Apache_Arrow&action=history https://en.wikipedia.org/w/index.php?title=Draft:Apache_Arro... Geez if you want to use Wikipedia as an ad, put a bit of effort in, when did marketing become so lazy and blame the platform. Although this meta ad is possibly a far better payoff.
est31 7y ago
Hmmm this reminds me of the battle to get a Wikipedia page approved for Minetest, the biggest FLOSS voxel engine out there: https://en.wikipedia.org/wiki/Draft:Minetest https://en.wikipedia.org/wiki/Draft:Minetest https://en.wikipedia.org/wiki/Wikipedia:Articles_for_deletion/Minetest https://en.wikipedia.org/wiki/Wikipedia:Articles_for_deletio...
scarejunba 7y ago
https://en.wikipedia.org/wiki/Draft:Apache_Arrow https://en.wikipedia.org/wiki/Draft:Apache_Arrow This reads like it was written by the guy who wrote it. It can do this. It efficiently does that. It’s all promotional content. Not useful.
aabbcc1241 7y ago
You're free to post to anywhere; And each site admin/helper/whatever-title are free to do their own censorship or moderation. It's the nature of the web.
cdeil 7y ago
I think the reason this is discussed now is because yesterday I tried to re-submit the Apache Arrow article. Here's what I wrote: https://en.wikipedia.org/w/index.php?title=Draft:Apache_Arrow&oldid=931353298 https://en.wikipedia.org/w/index.php?title=Draft:Apache_Arro... It was rejected / reverted 10 minutes later by a Wikipedia editor. The blog post from Justin was in July 2019 (https://www.dremio.com/why-apache-arrow-wikipedia/ https://www.dremio.com/why-apache-arrow-wikipedia/) There's many interesting and good points in the discussion here, thank you! To add my 2 cents: - Apache Arrow is notable, deserves a Wikipedia page. It might not have been when someone first tried to create a Wikipedia page for it in 2017 (see https://en.wikipedia.org/w/index.php?title=Draft:Apache_Arrow&action=history https://en.wikipedia.org/w/index.php?title=Draft:Apache_Arro...), but in the three years since it has become a major project, see e.g. https://blogs.apache.org/foundation/entry/the-apache-software-foundation-announces46 https://blogs.apache.org/foundation/entry/the-apache-softwar... Notability is clearly subjective, depends on what the author and reviewer find interesting. In the variant I submitted yesterday I tried to make it clear why it's notable - Apache arrow is a standard format that connects different languages, runtimes, data systems, communities, e.g. the Python and Java data communities. See e.g. https://wesmckinney.com/blog/apache-arrow-pandas-internals/ https://wesmckinney.com/blog/apache-arrow-pandas-internals/ - Apache Arrow is to my knowledge partly the brainchild of Wes McKinney, creator of pandas, it's his attempt (looking strongly like success) to resolve a major issue in data science. - I think it's a good point Justin made at https://www.dremio.com/why-apache-arrow-wikipedia/ https://www.dremio.com/why-apache-arrow-wikipedia/ that it's bad that Wikipedia editors reject articles on stuff they know nothing about - if you look at their profiles, they don't seem to have any knowledge or interest about technology or software. That's not a good system. - I haven't contributed to Wikipedia really before, and I don't understand the rules, I admit that. Probably what I did yesterday was just not following their process, and that's the reason my edit was reverted. I guess it's also true that Justin at first didn't do a great job at submitting an impartial, non-PR article. However, my understanding from looking at some drafts and the talk page is that he then took the editor comments into account, and the last variant of the page he tried to submit in July 2019 was OK. - So overall I think the answer to the question "Why isn't there a Wikipedia page on Apache arrow?" is that it's an unfortunate case of authors and editors not doing a great job. At least I'm pretty sure I didn't do a good job yesterday, I wanted to help, but only had an hour, not a day to learn how Wikipedia ticks and to do more research to find better references. I hope someone with more experience in Wikipedia and Arrow will try to re-write and re-submit the Wikipedia article in the future. - The rule to discourage (or forbid?) people involved with Apache Arrow from contributing to its Wikipedia page is unfortunate. I recently started to use it and learn about it, but I don't know much about it at this point. E.g. Wes McKinney has written at this point 8 high-quality blog posts about it (https://wesmckinney.com/archives.html https://wesmckinney.com/archives.html) - those don't count as references? Even if he or the Apache Arrow team wrote a paper about it, it wouldn't count because it's a primary source, and Wikipedia only wants secondary sources to establish notability? There are ~ 100 videos on YouTube, and many blog posts and a few podcasts (e.g. https://softwareengineeringdaily.com/2016/07/17/apache-arrow-with-uwe-korn/ https://softwareengineeringdaily.com/2016/07/17/apache-arrow...) that mention Apache Arrow. Naturally almost all of them are from Apache Arrow contributors, or from companies using Apache Arrow. - Apache Arrow has an interesting story, and it has evolved over the past years and will keep evolving, so I think exactly for that reason a Wikipedia page would be good to have, since the current project page and old blog posts don't capture that well.
ksec 7y ago
It was the same with 802.11ax aka WiFi 6. Someone decided all the technical information on the subject are irrelevant and deleted all Data Rate and Technical Improvement section. Another reason was because those details were not finalised. While it was a little frustrating that those useful information were gone as one could always found those in other source and media, but they also deleted the whole section on DensiFi [1], where all the major companies ( Apple, Broadcom, Cisco, Intel, Qualcomm, Huawei, Samsung and others ) behind the 802.11ax decided to do the work behind close door. TL;DR They were trying to push 802.11ax to the market earlier despite of all the un-resolved issues. So I decided to add only the DensiFi section, and it was constantly being deleted within 24 hours. After a few weeks of fun the page simply got back to the original, where Data Rate and Improvement are back but DensiFi section is totally gone. So it turns out it wasn't the technical section they were trying to get rid of. P.S We should be glad someone in the working group discovered this and called out on the action. The current WiFi 6 / 802.11ax situation and UX is much better than what we had when 802.11ac were shipped. Although this is at the expense of somewhat 2 years delay of the standard. [1] https://mlexmarketinsight.com/insights-center/editors-picks/antitrust/north-america/doj-probes-role-of-special-interest-group-in-new-wifi-standard https://mlexmarketinsight.com/insights-center/editors-picks/...
dredmorbius 7y ago
For those who think that edit wars, content disagreements, and innacuracies are any special realm of Wikipedia, they're not. One of the best examples I've encountered demonstrating this is a 19th century edit revision war between the British and American publishers of Chamber's Encyclopaedia, on the topics of Free Trade, Protection Duties, Slavery, and certain salacious particulars concerning His Royal Highness, the Prince of Wales. https://old.reddit.com/r/dredmorbius/comments/4xe2k1/chambers_encyclopaedia_editorial_statement/ https://old.reddit.com/r/dredmorbius/comments/4xe2k1/chamber... What's novel concerning Wikipedia is that these disputes (as with those of free software vs. proprietary software) tend to occur, or at least leave significant evidence, in the open public record.
julianlam 7y ago
The whole concept of "notability" in Wikipedia-land is subjective as hell. Whether your article makes it in is simply a matter of rolling the dice the first time you submit the article. I created an article for NodeBB, a piece of forum software used worldwide by companies small and large (including several triple A gaming companies). We got AfD'd, and now every time someone creates an article for NodeBB, the AfD is brought up and the entire discussion ends as soon as it has begun. We even created an article the _suggested_ way, by submitting a draft for review. It got reviewed alright... instant rejection because they felt it looked like an ad. We made changes, but nobody ever took a second look at the article. Of course, a number of defunct open-source (and some proprietary) forum softwares with zero sources are still allowed on Wikipedia, simply due to the fact that they made it through when nobody was looking :) One could argue that we shouldn't be writing our own articles (and they'd be right), so we just quietly accepted our judgement and market NodeBB based on the merits of the software, instead of whether it appears in some arbitrary ranking of forum software. That said, it'd still be nice if we were listed in the Wikipedia list of forum softwares.... _sigh_, a guy can dream.
- zamadatix 7y ago
  From your own description it doesn't sound subjective as much as understaffed.
nanoscopic 7y ago
This reminds me of my "war" to get an article for my parser XML::Bare. There was a time when there was a comparison page for XML parsers, and many parsers had articles. Still existing parsers on Wikipedia that should be removed; if they are to stay true to their war on having useful software info in Wikipedia: https://en.wikipedia.org/wiki/Category:XML_parsers https://en.wikipedia.org/wiki/Category:XML_parsers The original argument was that if you can find a citation in print you can have whatever it is on Wikipedia, but that ceased to be true years ago and it has become a popularity contest and power struggle with obnoxious Wikipedia editors.

111 points by riboflavin 7y ago

deleted 7y ago

[deleted]

ggggtez 7y ago

I've never heard of it. Add my vote to removing the article. Cry more, company I never heard of either.

mistrial9 7y ago

sadly, I can jump in on the "Wikipedia fails" train here, also. In about five attempts to really change an article (different ones) in about five years, every single change was rejected, as far as I know. The changes were different, one was writing style and order of facts on a public historical event in this century; one was adding a lot of detail to the description of a popular fantasy fiction series; one was removing a controversial and provocative one-liner at the top of a page about people at the edge of (western) society; and another .. hmm I forget now, because I just gave up ! My aging colleague tells me, just keep doing the changes, they cant stop everything. However, my direct (and limited) experience is.. they do stop everything (that I try). I was logged in twice and used anonymous three times, and added citation a bit, too. To the point of the article, FOSS projects in wikipedia ? hmm maybe there could be a clear category for that ? software projects are proliferating rapidly.. dunno

wffurr 7y ago
Did you read the reasons for rejection and try to modify subsequent submissions to better comply with Wikipedia's published guidelines?
- mistrial9 7y ago
  yes, I did, and I feel that this revert behavior was more hazing/article control than substantive in all cases but one, and that one I dont personally agree.
  - ErrantX 7y ago
    Have you got an example?
zozbot234 7y ago
The way around mindless reverts is to first detail the proposed change on the talk page, then wait for anyone interested to object. If no one does, you can make the change live, pointing out that no objections were raised. Even if someone does object, such objections should ultimately be made actionable, i.e. it should be made clear how to address them to the other party's satisfaction.

tetromino_ 7y ago

See https://en.wikipedia.org/wiki/Wikipedia:Notability https://en.wikipedia.org/wiki/Wikipedia:Notability - all you need to show is that Apache Arrow has received significant coverage in reliable sources that are independent of the subject. So: find conference papers/talks by people not affiliated with Apache or the Apache Arrow project and that discuss Apache Arrow. Figure out how to incorporate the tidbits about Arrow from those papers into the article text. Add sources in footnotes. Done.

dajohnson89 7y ago
And to be clear -- this is the job of the article author, not the editor who rejects the article without doing a little bit of homework first?
- tetromino_ 7y ago
  Of course. Similarly, when you are submitting a PR to an open source project or a manuscript to an academic journal, it's your job as the author to take note of contributor guidelines.
- skywhopper 7y ago
  Yes. The author is supposed to cite the sources.
- wolfgang42 7y ago
  Yes; Wikipedia's expectation is that authors have researched the topic about which they are writing, and therefore they are in the best position to provide the sources from which they got their information. The editors' job is to ensure that Wikipedia's standards are met, not to re-do the research that the author should already have performed. If the case is that no research was performed because the author is already an expert in the area, they are still expected to provide citations so that the same standard can be applied to all authors.
sb057 7y ago
A version which was rejected included the following (non-Apache-affiliated [afaik]) references: https://www.xenonstack.com/insights/what-is-apache-arrow/ https://www.xenonstack.com/insights/what-is-apache-arrow/ https://link.springer.com/chapter/10.1007%2F978-1-4842-1311-7_5 https://link.springer.com/chapter/10.1007%2F978-1-4842-1311-... https://www.biorxiv.org/content/biorxiv/early/2016/08/23/071092.full.pdf https://www.biorxiv.org/content/biorxiv/early/2016/08/23/071... http://delivery.acm.org/10.1145/3110000/3103003/p138-Maas.pdf http://delivery.acm.org/10.1145/3110000/3103003/p138-Maas.pd... https://www.theregister.co.uk/2016/02/17/apache_arrow_toplevel_project/ https://www.theregister.co.uk/2016/02/17/apache_arrow_toplev... https://www.cio.com/article/3034279/big-data-gets-a-new-open-source-project-apache-arrow.html https://www.cio.com/article/3034279/big-data-gets-a-new-open... https://www.infoworld.com/article/3033446/hadoop/apache-arrow-aims-to-speed-access-to-big-data.html https://www.infoworld.com/article/3033446/hadoop/apache-arro... https://sdtimes.com/apache/guest-view-first-release-apache-arrow/ https://sdtimes.com/apache/guest-view-first-release-apache-a... https://www.infoq.com/news/2016/12/le-dem-apache-arrow/ https://www.infoq.com/news/2016/12/le-dem-apache-arrow/ http://dbmsmusings.blogspot.com/2017/10/apache-arrow-vs-parquet-and-orc-do-we.html http://dbmsmusings.blogspot.com/2017/10/apache-arrow-vs-parq... https://dbmsmusings.blogspot.com/2017/10/apache-arrow-vs-parquet-and-orc-do-we.html https://dbmsmusings.blogspot.com/2017/10/apache-arrow-vs-par...

TallGuyShort 7y ago

Having worked on commercially-resold Apache projects, can't say I argue with Wikipedia a whole lot on this. It seems to me it should be let in, but it's a bit silly to go and call them out like this on a corporate blog, IMO. Dremio does benefit from Apache Arrow publicity and notoriety, even if they don't profit directly. Having a de-facto standard data format and open-source engines is a selling point for some. That's why Dremio explicitly calls it out on their own website. It also never hurts in the recruiting department. (edit: there's a reason the article was submitted by someone working in marketing & strategy) >> I’m wondering if Wikipedia can continue to be considered a reliable source of information for technical folks who want to learn more about the vast system of Apache open source software projects. Sign up for the Olympics, because that's a hell of a leap. You didn't get your page in, it's really not much of a reflection on the rest of Wikipedia. It's an open-source project. It should have it's own freely available documentation that fills much the same purpose anyway. If I want to learn about Apache X, I go straight to x.apache.org. They concede that it's not an end-user product anyway, so I'd think their key audience knows how to find an open-source project website. Lower the bar too far the other way, and there are plenty of semi-open-source project's marketing departments would be all over using Wikipedia to their own ends - I've seen my own former employer do this for their Apache projects.

duskwuff 7y ago
> I’m wondering if Wikipedia can continue to be considered a reliable source of information for technical folks who want to learn more about the vast system of Apache open source software projects. I'm confused why the writer thinks it should be! The Apache Foundation is a big tent. There's some clearly notable projects in there (like Apache httpd), but there's also a lot of really obscure crap that basically nobody outside ASF cares about (like Apache Creadur or Apache Pony Mail). Expecting Wikipedia to document every Apache project is ridiculous. Is this particular project notable enough for a Wikipedia article? I don't know a lot about it, so I can't say for sure. But the article drafts that I've seen don't convince me that it is.
- lallysingh 7y ago
  > Expecting Wikipedia to document every Apache project is ridiculous. Wikipedia has a lot of rather obscure entries. Long, long lists that I think easily under-rank the Apache project. I'm not saying you're wrong, but the bar for notability is kinda vague. Lists of every episode of a series, every kind of kim-chee, etc.

oefrha 7y ago

https://en.wikipedia.org/wiki/Draft:Apache_Arrow https://en.wikipedia.org/wiki/Draft:Apache_Arrow > REVIEWERS: Please note that the submitting editor is the chief marketing officer and vice president of strategy at this company. Yeah, sorry, big no no there. Disclosure: consider myself a Wikipedian to some extent, got a couple hundred edits on Wikipedia.

foota 7y ago
This is wrong though, Apache Arrow is an open source spec, the fact that they happen to work at a company working on it seems secondary.
- oefrha 7y ago
  1. Conflict of interest has nothing to do with whether it’s open source. If I submit a Wikipedia article about my completely uncommercial personal project it’s still a huge conflict of interest. Rule of thumb: don’t submit Wikipedia articles about yourself, your project, your employer or your employer’s project. If it’s notable enough eventually someone without a conflict of interest will do it. 2. Tons of companies use open source for marketing. This one is no different as far as I can tell. Even had the chief marketing officer submit a Wikipedia article for their project.
  - mxfh 7y ago
    The problem with these rules are, that they are so selectively enforced, it is a farce. They selectively assume bad faith, where, by all objective means is none, and brush over others, as long as it is backed up by random arguably non-neutral publications. It's near impossible to put article's up over prolific female journalist for example, because all this can be enforced since, the publications are all from the same source (the publisher they work for) or are interviews or some sort of talk or award, where it's almost never for men, which get away* by linking to a podcast. tldr: *WP:OTHERSTUFFEXISTS
  - foota 7y ago
    But it's not _their_ project, it's a project they contribute to. Google contributing an article about k8s would be a completely different thing from Google contributing an article on say Hadoop.

mxfh 7y ago

It was a pain to get gitlab in 5 years ago after a "controversial" deletion, so it wasn't available for simple undeletion. Domain specific knowledge has it notouriously hard with wikilawyers who, at large, seemingly stopped adding new things to their world view 15 years ago. Then it becomes a game of jumping through hoops and hoping you end up with a kind wiki-landlord or knowing a friendly wikipedia admin. Doing the latter by anouncing your concern on social media and hoping a sympathetic admin picks it up, might be the easiest on human time and resources, just let them copy your reasonably well sourced article draft from your personal space and see what happens.

sien 7y ago
At one point about 10 years ago the entry for Atlasssian was deleted for not being notable.
- Geimfari 7y ago
  It was deleted in 2005 for being a blatant advert, and again in 2010 for the same reason. I doubt it was actually a notable company in 2005. [1] https://en.wikipedia.org/wiki/Wikipedia:Articles_for_deletion/Atlassian_Software_Systems https://en.wikipedia.org/wiki/Wikipedia:Articles_for_deletio... [2] https://en.wikipedia.org/wiki/Special:Log?type=delete&page=Atlassian&wpdate=2010-01-19 https://en.wikipedia.org/wiki/Special:Log?type=delete&page=A...
  - yellowapple 7y ago
    2005 was 15 years ago, not 10.
  - mxfh 7y ago
    Even if it was a clumsy self-promotion or over-ambitious fans with no clue on wikipedia inner mechanics shouldn't set back a viable interest on information about a given company or other entitity by a multitude of years. After a deletion it's just magnitudes harder for anyone to get an article restored, compared to an entity which didn't have the "luck" to get added to wikipedia too early. Deletion history shouldn't have that much of a say on actively developing entities as it has now.

Ninjaneered 7y ago

Here's the link to the draft: https://en.wikipedia.org/wiki/Draft:Apache_Arrow https://en.wikipedia.org/wiki/Draft:Apache_Arrow And some possible additional sources: * https://www.forbes.com/sites/forbestechcouncil/2019/09/24/dremio-helps-reduce-proximity-to-your-data/amp/ https://www.forbes.com/sites/forbestechcouncil/2019/09/24/dr... * https://www.businesswire.com/news/home/20180906005114/en https://www.businesswire.com/news/home/20180906005114/en * https://thesiliconreview.com/2016/02/apache-arrow-is-the-new-open-source-project-for-big-data https://thesiliconreview.com/2016/02/apache-arrow-is-the-new...

tptacek 7y ago
The first article is a paid promotion piece, which WP won't accept as an RS. The second is a press release by Arrow's sponsoring company, which, obviously, WP won't accept as an RS. I have no idea what "The Silicon Review" is; this is the first time I've ever seen it. To the extent it's not a pay-to-play trade publication, it might qualify as a notability-establishing source. The fact that the "Review" does not itself have a WP page might make it harder to claim it's reliable, since it suggests nobody else knows what it is, either.
- Ninjaneered 7y ago
  Looks like my lateral reading was sub-par (actually I didn't even try, just a quick Google/post). The "Silicon Review" one looks like a pay-to-play as well after further review, it's used in citation on a few other Wikipedia articles, but as far as I can tell, and due to some anecdotal stories, it doesn't look good. * https://www.reddit.com/r/PublicRelations/comments/bha6hs/silicon_review_emailed_me_about_an_offer_to/ https://www.reddit.com/r/PublicRelations/comments/bha6hs/sil... * https://arpr.com/blog/4-pay-for-play-scams/ https://arpr.com/blog/4-pay-for-play-scams/ Good catch, thanks for spending the time to review my links. Reading your comments above, I largely agree. It's a high bar (mostly) to get an article on Wikipedia, and that's a good thing. It allows us to read the majority of content on Wikipedia without too much suspicion.
SquishyPanda23 7y ago
I read the draft. Maybe this is an unpopular opinion, but it's obvious advertising and has no place on Wikipedia. Maybe a Medium post would be more appropriate. Wikipedia already has a problem with bad software articles like this.

pradn 7y ago

It looks like there aren't enough independent, non-commercial articles to use as references. This is somewhat common for many newish technical projects. Add some academic papers, some usage numbers, some summary blog posts that aren't related to the project. Wiki editors are very suspicious of people from companies editing articles related to their work.

PeterCorless 7y ago
I'm not so sure it's falling afoul of 'not-notable' so much as WP:COI.

foota 7y ago

Can HN create a draft that would be accepted? :)

thebooktocome 7y ago
Nope, this violates Wikipedia's policy against "meat puppetry".
- ErrantX 7y ago
  Not at all! The main thrust of that policy is for discussions. Improving Wikipedia by conspiring to write a compliant article is explicitly allowed!
  - thebooktocome 7y ago
    Tell your deletionist editors that.

zeveb 7y ago

One could perhaps be forgiven for wishing that the deletionists would … delete themselves. Seriously, though, bytes are cheap, and an article sitting somewhere in Wikipedia doing nothing and bothering no-one is pretty damned cheap too.

SpicyLemonZest 7y ago
I dunno, this kind of thing seems like exactly the canonical argument for deletionism. Maybe there's no cost to a page sitting on Wikipedia describing, like, some guy's special attack from Naruto. There are reasonable arguments that allowing things like that would set a bad precedent and encourage behavior that doesn't help the project, but I admit it's pretty tenuous. There are obvious and important costs if Wikipedia articles start being perceived as promotional material rather than encyclopedia entries.
- CharlesColeman 7y ago
  > Maybe there's no cost to a page sitting on Wikipedia describing, like, some guy's special attack from Naruto. There is a cost, but it's measured in hours of maintenance labor not bytes of storage. If Wikipedia wants to maintain a semblance of accuracy [1] in the face of declining participation, it needs to concentrate its labor resources rather than spread them out. [1] which IMHO is vital given its unwise prestige as arbiter or truth
  - sgift 7y ago
    Since Wikipedias concentration of labor itself is a source of declining participation[1] it's doubtful that continuing this behavior will result in something else than a death spiral with even fewer people ready to do the work, more concentration, even fewer .. and so on. [1] Among other reasons, more here: https://en.wikipedia.org/wiki/Wikipedia:Why_is_Wikipedia_losing_contributors_-_Thinking_about_remedies https://en.wikipedia.org/wiki/Wikipedia:Why_is_Wikipedia_los...
    - CharlesColeman 7y ago

ForHackernews 7y ago

I've literally never heard of this piece of software, and it's fair to say I'm much more interested in FLOSS than the average person on the internet. Why should this thing have its own article and not just appear in a list of Apache foundation projects?

m_ke 7y ago
Because it's actually a pretty big deal for the (python) data science ecosystem.

xibalba 7y ago

As a strategy for getting Dremio on the front page of HN and thus on the radar of a large group of tech people (i.e. Dremio's prospects), this is article is very clever. As a critique of Wikipedia, not so much.

jessaustin 7y ago
This seems like a good reason to flag it.

tptacek 7y ago

Open source projects are particularly tricky for Wikipedia. There are tens of thousands of them. Their owners are often passionate. They compete with each other, so there's incentive to write hard-to-adjudicate competing claims. Many have commercial backing, which further warps incentives. The projects themselves are highly technical; many, like Arrow, are software development tools and components. There are few authoritative sources that reliably track open source projects. Keeping up involves directly following bug trackers and message boards and then synthesizing a narrative, which is the definition of "original research", forbidden in the encyclopedia. It's likely that Arrow does deserve a WP article. But Arrow's sponsors misunderstand more about Wikipedia than Wikipedia does about Arrow. Writing a defensible article about their project will require work; in particular, they're going to need to spend the time tracking down authoritative sources for why Arrow is notable, and those claims will probably need to be something more persuasive than "hundreds of companies use it"; hundreds of companies use all sorts of things that don't, and shouldn't, be featured in their own encyclopedia articles. I understand the impulse behind "this project is important; it should have a Wikipedia article". But when you take a step back and accept what Wikipedia actually is, rather than what you think it should be, you're left with the question: do we really need to feature this particular piece of software in its own encyclopedia article? 20 years from now, will people still be getting value from it? Whatever value that might be, will it outweigh the 20 years of other people's volunteer efforts to maintain the article, keeping it free of vandalism and ensuring that it doesn't surreptitiously turn into a promotion piece for some company or another? The answers might be "yes". But I don't see much evidence in this piece considered the questions. Lots of things that don't seem deserving have in-depth Wikipedia coverage. Many of those things probably really don't belong in an encyclopedia! But there are two sides to this problem: the merit of the topic, and the cost, in volunteer time, of including them. A marginal topic can be defensible if it's easy to reliably cover it. A seemingly important technical topic might not be if the only way to say anything interesting about it is to write original research directly into its article. Late edit A useful tip for getting your open source project covered in its own Wikipedia article: don't have the Chief Marketing Officer of the company that owns the project write the article.

SkyBelow 7y ago
If a bulldog clip is notable, why wouldn't an open source project that hundreds of companies use not be? https://en.wikipedia.org/wiki/Bulldog_clip https://en.wikipedia.org/wiki/Bulldog_clip s/bulldog clip/many other random office supply items/ Edit: Swapped to bulldog clip as a better example of a less notable office supply.

lmeyerov 7y ago

For context, some other companies contributing to it are in the GPU space, so orthogonal to CPU-centric Dremio: Nvidia, Blazing SQL, and Graphistry (us). Likewise, the pydata big guns intersect a bit here: conda, pandas, ... . This effort got a BOSSIE award for GPU dataframes this year and is taking off now that it is becoming usable for more than just framework devs. The reason we all really on it is because a standardized columnar IO streaming format is an awesome idea for compositional HPC. It does sounds like maybe Dremio's CMO wrote the original articles and it came off centered on them? (Did not have a chance to read.)

tptacek 7y ago
Yes: Dremio's CMO wrote the article, and the article was overtly promotional. Of course it got killed.
- TallGuyShort 7y ago
  And the same guy submitted it here 3 times. Also overtly promotional. riboflavin, genuine suggestion: ask one of the PMC members or committers to rewrite the article from scratch from an engineer's perspective, source everything, demonstrate notability, and resubmit. If they still don't take it, move on with your life. ... but you might have generated a lot of ill-will with the Wikipedia elites here already.

jabvigWe 7y ago

Add it to the Free Software Directory! https://directory.fsf.org/wiki/Main_Page https://directory.fsf.org/wiki/Main_Page

swayvil 7y ago

Wikipedia is administrated by inbred autistics. If it doesn't smell like their own thrice-huffed fartgas they will remove it.

blattimwind 7y ago
Now imagine the German Wikipedia, were Exclusionists are 200 times stronger than the Inclusionists, relative to the English Wikipedia.
- kstrauser 7y ago
  200x? I'm imagining a single page, titled "Nein."
- yorwba 7y ago
  The German Wikipedia is the fourth largest by number of articles, with one article for 33 German speakers, whereas the English Wikipedia has one article for 84.4 speakers. https://meta.wikimedia.org/wiki/List_of_Wikipedias_by_speakers_per_article https://meta.wikimedia.org/wiki/List_of_Wikipedias_by_speake... Doesn't look like the Exclusionists are really stronger on the German Wikipedia.

jccalhoun 7y ago

>Arrow is designed to serve as a shared foundation for SQL execution engines, data analysis systems, storage systems, and more – think Pandas, Spark, Parquet, etc. Engineers across the community are working together to establish Arrow as a standard for columnar in-memory processing. I like to think I'm fairly techy for a non-programmer but I have no idea what that means. That might be part of their problem if that is the description in their wikipedia entry.

ggggtez 7y ago
I believe it's what the kids call "buzzword bingo".

michelpp 7y ago

We're having the same issue getting the GraphBLAS API article to be accepted: https://en.wikipedia.org/wiki/Draft:GraphBLAS https://en.wikipedia.org/wiki/Draft:GraphBLAS. At first it was summarily deleted overnight, now we're stuck in Draft for who know how long.

mikl 7y ago

Why do you care about having a Wikipedia page for Arrow? Why is it important enough to whinge about on HN? Wikipedia is much like Stack Overflow these days, the community has become hostile to newcomers who fail to meet their somewhat arbitrary but very exacting standards for what is allowed on their site. Fortunately, you can just publish your own web site. No need to be bothered about not being on WP.

thrower123 7y ago

The hard-line Wikipedia deletionists should be deleted themselves. The argument is always brought up, like StackOverflow, that they have to be ruthless or it turns into an Eternal September dumping ground of garbage, but the quality is already very uneven and gatekeeping like Cerberus doesn't help further that goal. There's already a toxic Dead Sea effect where the pedantry and politicking has chased out a lot of people that would contribute; who the hell wants to bother putting in some hours writing something up if it is just going to be summarily deleted? Bandwidth and hard drives are cheap. Just spitballing, but it'd be nice if Wikipedia worked a little more like Linux distro repositories. Keep the tightly curated articles in a "core", but leave room for "community" or "nonfree" collections if you want to turn them on.

CharlesColeman 7y ago
> Just spitballing, but it'd be nice if Wikipedia worked a little more like Linux distro repositories. Keep the tightly curated articles in a "core", but leave room for "community" or "nonfree" collections if you want to turn them on. I think that's a fantastic idea, especially if it would lead to a drastic reduction in the number of articles served from the main Wikipedia domain (to a number that can meet some reasonable quality and maintenance standard, maybe 10 times the size of the most comprehensive print encyclopedia, or a 1/6 of Wikipedia's current size) [1]. [1] https://newrepublic.com/article/101795/encyclopedia-britannica-publish-information https://newrepublic.com/article/101795/encyclopedia-britanni...: "The 2002 Britannica contained 65,000 articles and 44 million words. Wikipedia currently contains close to four million articles and over two billion words..."
bjourne 7y ago
Most communities seem to go that way. In the beginning, most people spend their time contributing first-order content. Then, as the community grows, it attracts more meta-users who are more interested in moderating the content creators. They create ever more rules and policies requiring content creators to jump through more and more hoops. Eventually the experience becomes so frustrating that people give up. Wikipedia seem to me to be in that situation. StackOverflow is on its way there. It has exactly the same kind of problem with "deletionists" that Wikipedia has. Perfectly good questions are often closed for very arbitrary reasons.

qwerty456127 7y ago

Once I witnessed awesome articles [others added and I used with delight] on open source frameworks as well as some minor facts [I added] on other subjects deleted for being "insignificant" I decided I'm not donating to Wikipedia until this bullshit ends. Wiki articles are not videos, they take humble disk space to host so I can't recognize any reason in dismissing "insignificant" information other than a stupid rule. IMHO whatever can be considered a piece of knowledge should be there. BTW nearly the same applies to StackOverflow - thanks to high reputation points I earnt during the early days I can see deleted questions and answers and I often see really interesting (having three-figure upvvote scores and dozens of stars) questions and very informative (also heavily upvoted) answers deleted.

aaron695 7y ago

View the original declined drafts here - https://en.wikipedia.org/w/index.php?title=Draft:Apache_Arrow&action=history https://en.wikipedia.org/w/index.php?title=Draft:Apache_Arro... Geez if you want to use Wikipedia as an ad, put a bit of effort in, when did marketing become so lazy and blame the platform. Although this meta ad is possibly a far better payoff.

est31 7y ago

Hmmm this reminds me of the battle to get a Wikipedia page approved for Minetest, the biggest FLOSS voxel engine out there: https://en.wikipedia.org/wiki/Draft:Minetest https://en.wikipedia.org/wiki/Draft:Minetest https://en.wikipedia.org/wiki/Wikipedia:Articles_for_deletion/Minetest https://en.wikipedia.org/wiki/Wikipedia:Articles_for_deletio...

scarejunba 7y ago

https://en.wikipedia.org/wiki/Draft:Apache_Arrow https://en.wikipedia.org/wiki/Draft:Apache_Arrow This reads like it was written by the guy who wrote it. It can do this. It efficiently does that. It’s all promotional content. Not useful.

aabbcc1241 7y ago

You're free to post to anywhere; And each site admin/helper/whatever-title are free to do their own censorship or moderation. It's the nature of the web.

cdeil 7y ago

I think the reason this is discussed now is because yesterday I tried to re-submit the Apache Arrow article. Here's what I wrote: https://en.wikipedia.org/w/index.php?title=Draft:Apache_Arrow&oldid=931353298 https://en.wikipedia.org/w/index.php?title=Draft:Apache_Arro... It was rejected / reverted 10 minutes later by a Wikipedia editor. The blog post from Justin was in July 2019 (https://www.dremio.com/why-apache-arrow-wikipedia/ https://www.dremio.com/why-apache-arrow-wikipedia/) There's many interesting and good points in the discussion here, thank you! To add my 2 cents: - Apache Arrow is notable, deserves a Wikipedia page. It might not have been when someone first tried to create a Wikipedia page for it in 2017 (see https://en.wikipedia.org/w/index.php?title=Draft:Apache_Arrow&action=history https://en.wikipedia.org/w/index.php?title=Draft:Apache_Arro...), but in the three years since it has become a major project, see e.g. https://blogs.apache.org/foundation/entry/the-apache-software-foundation-announces46 https://blogs.apache.org/foundation/entry/the-apache-softwar... Notability is clearly subjective, depends on what the author and reviewer find interesting. In the variant I submitted yesterday I tried to make it clear why it's notable - Apache arrow is a standard format that connects different languages, runtimes, data systems, communities, e.g. the Python and Java data communities. See e.g. https://wesmckinney.com/blog/apache-arrow-pandas-internals/ https://wesmckinney.com/blog/apache-arrow-pandas-internals/ - Apache Arrow is to my knowledge partly the brainchild of Wes McKinney, creator of pandas, it's his attempt (looking strongly like success) to resolve a major issue in data science. - I think it's a good point Justin made at https://www.dremio.com/why-apache-arrow-wikipedia/ https://www.dremio.com/why-apache-arrow-wikipedia/ that it's bad that Wikipedia editors reject articles on stuff they know nothing about - if you look at their profiles, they don't seem to have any knowledge or interest about technology or software. That's not a good system. - I haven't contributed to Wikipedia really before, and I don't understand the rules, I admit that. Probably what I did yesterday was just not following their process, and that's the reason my edit was reverted. I guess it's also true that Justin at first didn't do a great job at submitting an impartial, non-PR article. However, my understanding from looking at some drafts and the talk page is that he then took the editor comments into account, and the last variant of the page he tried to submit in July 2019 was OK. - So overall I think the answer to the question "Why isn't there a Wikipedia page on Apache arrow?" is that it's an unfortunate case of authors and editors not doing a great job. At least I'm pretty sure I didn't do a good job yesterday, I wanted to help, but only had an hour, not a day to learn how Wikipedia ticks and to do more research to find better references. I hope someone with more experience in Wikipedia and Arrow will try to re-write and re-submit the Wikipedia article in the future. - The rule to discourage (or forbid?) people involved with Apache Arrow from contributing to its Wikipedia page is unfortunate. I recently started to use it and learn about it, but I don't know much about it at this point. E.g. Wes McKinney has written at this point 8 high-quality blog posts about it (https://wesmckinney.com/archives.html https://wesmckinney.com/archives.html) - those don't count as references? Even if he or the Apache Arrow team wrote a paper about it, it wouldn't count because it's a primary source, and Wikipedia only wants secondary sources to establish notability? There are ~ 100 videos on YouTube, and many blog posts and a few podcasts (e.g. https://softwareengineeringdaily.com/2016/07/17/apache-arrow-with-uwe-korn/ https://softwareengineeringdaily.com/2016/07/17/apache-arrow...) that mention Apache Arrow. Naturally almost all of them are from Apache Arrow contributors, or from companies using Apache Arrow. - Apache Arrow has an interesting story, and it has evolved over the past years and will keep evolving, so I think exactly for that reason a Wikipedia page would be good to have, since the current project page and old blog posts don't capture that well.

ksec 7y ago

It was the same with 802.11ax aka WiFi 6. Someone decided all the technical information on the subject are irrelevant and deleted all Data Rate and Technical Improvement section. Another reason was because those details were not finalised. While it was a little frustrating that those useful information were gone as one could always found those in other source and media, but they also deleted the whole section on DensiFi [1], where all the major companies ( Apple, Broadcom, Cisco, Intel, Qualcomm, Huawei, Samsung and others ) behind the 802.11ax decided to do the work behind close door. TL;DR They were trying to push 802.11ax to the market earlier despite of all the un-resolved issues. So I decided to add only the DensiFi section, and it was constantly being deleted within 24 hours. After a few weeks of fun the page simply got back to the original, where Data Rate and Improvement are back but DensiFi section is totally gone. So it turns out it wasn't the technical section they were trying to get rid of. P.S We should be glad someone in the working group discovered this and called out on the action. The current WiFi 6 / 802.11ax situation and UX is much better than what we had when 802.11ac were shipped. Although this is at the expense of somewhat 2 years delay of the standard. [1] https://mlexmarketinsight.com/insights-center/editors-picks/antitrust/north-america/doj-probes-role-of-special-interest-group-in-new-wifi-standard https://mlexmarketinsight.com/insights-center/editors-picks/...

dredmorbius 7y ago

For those who think that edit wars, content disagreements, and innacuracies are any special realm of Wikipedia, they're not. One of the best examples I've encountered demonstrating this is a 19th century edit revision war between the British and American publishers of Chamber's Encyclopaedia, on the topics of Free Trade, Protection Duties, Slavery, and certain salacious particulars concerning His Royal Highness, the Prince of Wales. https://old.reddit.com/r/dredmorbius/comments/4xe2k1/chambers_encyclopaedia_editorial_statement/ https://old.reddit.com/r/dredmorbius/comments/4xe2k1/chamber... What's novel concerning Wikipedia is that these disputes (as with those of free software vs. proprietary software) tend to occur, or at least leave significant evidence, in the open public record.

julianlam 7y ago

The whole concept of "notability" in Wikipedia-land is subjective as hell. Whether your article makes it in is simply a matter of rolling the dice the first time you submit the article. I created an article for NodeBB, a piece of forum software used worldwide by companies small and large (including several triple A gaming companies). We got AfD'd, and now every time someone creates an article for NodeBB, the AfD is brought up and the entire discussion ends as soon as it has begun. We even created an article the _suggested_ way, by submitting a draft for review. It got reviewed alright... instant rejection because they felt it looked like an ad. We made changes, but nobody ever took a second look at the article. Of course, a number of defunct open-source (and some proprietary) forum softwares with zero sources are still allowed on Wikipedia, simply due to the fact that they made it through when nobody was looking :) One could argue that we shouldn't be writing our own articles (and they'd be right), so we just quietly accepted our judgement and market NodeBB based on the merits of the software, instead of whether it appears in some arbitrary ranking of forum software. That said, it'd still be nice if we were listed in the Wikipedia list of forum softwares.... _sigh_, a guy can dream.

zamadatix 7y ago
From your own description it doesn't sound subjective as much as understaffed.

nanoscopic 7y ago

This reminds me of my "war" to get an article for my parser XML::Bare. There was a time when there was a comparison page for XML parsers, and many parsers had articles. Still existing parsers on Wikipedia that should be removed; if they are to stay true to their war on having useful software info in Wikipedia: https://en.wikipedia.org/wiki/Category:XML_parsers https://en.wikipedia.org/wiki/Category:XML_parsers The original argument was that if you can find a citation in print you can have whatever it is on Wikipedia, but that ceased to be true years ago and it has become a popularity contest and power struggle with obnoxious Wikipedia editors.