12 ms·
AOL Moloch: open-source, large scale, packet-capturing, indexing database system
- sparrish 6y agoWhat a horrible name for a project. Why would you want it associated with child sacrifice? It's disgusting and distracts from any merits of the tech.
- mumphster 6y agocounterpoint: I like it and owls are pretty rad
- dudeinjapan 6y agoAnd also Moloch isn't a good name either.
- ignoramous 6y agoNot the greatest choice of name: Moloch[a] (also Molech, Mollok, Milcom, or Malcam) is the biblical name of a Canaanite god associated with child sacrifice, through fire or war. From: https://en.wikipedia.org/wiki/Moloch https://en.wikipedia.org/wiki/Moloch
- totetsu 6y agoSheesh, I hope no-one looks up my user name on wikipedia..
- pnathan 6y agoThat was my immediate thought as well. Profoundly and deeply unsavory.
- simlevesque 6y agoYou are offended by the name of an open source project ? Your life must be easy.
- pgsbathhouse2 6y agoAre you going to be that daft?
- simlevesque 6y agoI'm really sorry you don't approve the name.
- nicehat 6y agoNo, that guy disapproved of you being rather daft. See, names matter.
- pgsbathhouse2 6y agoI don't have a stake in this argument. But nice signaling attempt.
- pnathan 6y agoNames have symbolic value beyond the intended referent. Typically, wisdom dictates choosing names that aren't immediately offensive out of the gate. I won't speculate on whatever led you to such fatuous conclusions, because ad hominem is a bad reasoning.
- striking 6y agoPerhaps they were going for an association with Allen Ginsburg's Howl (https://www.poetryfoundation.org/poems/49303/howl https://www.poetryfoundation.org/poems/49303/howl) instead?
- lsb 6y agoReminds me of when someone saw that the NSA's public key (http://www.cypherspace.org/adam/hacks/lotus-nsa-key.html http://www.cypherspace.org/adam/hacks/lotus-nsa-key.html) to be included in Lotus Notes had an organizational name of "MiniTruth", and a common name of "Big Brother". Moloch isn't the good guy in Howl. Moloch! Solitude! Filth! Ugliness! Ashcans and unobtainable dollars! Children screaming under the stairways! Boys sobbing in armies! Old men weeping in the parks! Moloch! Moloch! Nightmare of Moloch! Moloch the loveless! Mental Moloch! Moloch the heavy judger of men! Moloch the incomprehensible prison! Moloch the crossbone soulless jailhouse and Congress of sorrows! Moloch whose buildings are judgment! Moloch the vast stone of war! Moloch the stunned governments!
- peterwwillis 6y agoMoloch the dude in the cubicle across from mine that always eats his lunch at his desk and chews with his mouth open! Moloch the meddling micromanager from Maine! Moloch the Junior Architect! Moloch the 3-space indenter!
- hprotagonist 6y agowhich is the same referent, and begs the same question: "What sphinx of cement and aluminum bashed open their skulls and ate up their brains and imagination?"
- theprotocol 6y agoWasn't Allen Ginsburg also in favor of child abuse?... He was a member of NAMBLA... At least Moloch isn't real.
- pas 6y agohttps://slatestarcodex.com/2014/07/30/meditations-on-moloch/ https://slatestarcodex.com/2014/07/30/meditations-on-moloch/
- _jal 6y agoKind of fitting for AOL though. I can name three startups that went to die there off the top of my head.
- chews 6y agoIt's an inside joke to the Deep State that actively needs captures of network data, AOL... is a Verizon company after all. In these discordian times, even Eris would blush.
- mensetmanusman 6y agoFunny you mentioned the deep state. I remember being perturbed that the email leak from the 2016 race had references to child-sacrifice moloch. Strange times
- scandox 6y agoI think this is a bit distracting. It's an ancient name. Probably a better paragraph to fixate on would be: > Moloch has been used figuratively in English literature from John Milton's Paradise Lost (1667) to Allen Ginsberg's "Howl" (1955), to refer to a person or thing demanding or requiring a very costly sacrifice. I mean a project called Zeus would hardly founder on it's namesakes fetish for turning into a bull and assaulting human women.
- HideousKojima 6y agoZeus is known a bit better and for more things than Moloch
- sildur 6y agoBut today everyone knows about Zeus huge sex drive.
- derptron 6y agoAm I the only one who thinks of Moloch aka Edgar Jacobi, the old villain in Watchmen dying of cancer?
- smoe 6y agoInteresting, newer knew the word is of biblical origin. In German the word Moloch is in somewhat common use to refer to an abstract merciless, all-consuming or all devouring power. E.g big cities might get referred to as molochs the way they eat into nature around it or suck people into its anonymity
- woodpanel 6y agoI was thinking the same. In German Moloch is mostly used for groteskly big/growing cities. But thinking about it, the "eating into nature"-theme aligns well with the biblical "eating into kids"-theme. For me at least, because of the city-context, Moloch matches: The moloch could stand for the amount of activity in a system that is humanly insurmountable to get a better-than-superficial understanding of.
- smoe 6y agoHa, so I learned two things about Moloch today. I didn't have that strong of an association of Moloch with big cities as it felt like when I looked it up for the above answer, so I googled a bit of its usage on news sites and it looks like in Germany it is almost exclusively used for big cities, whereas in Switzerland, cities is the most common usage but it is applied way more liberally. Within a minute or so I saw it used to refer to: The EU, a government bicycle program, a 2km long tunnel in Zürich, iTunes, the Tour de France, Goldman Sachs and fashion.
- finnthehuman 6y agoWhy not? That's a pretty metal name. Don't tell me anyone still has appetite for yet another saccharine cutesy-wutesy "yay software squee" name.
- Hackbraten 6y agoThere’s always room for more cute in software.
- kfrzcode 6y agoHard disagree. Software should be cold, calculated and cannibalistic. Leave cuteness to the experts: puppies and such.
- runawaybottle 6y agoHow about AOL PackagieWackigie?
- cryptoquick 6y agoMore like UWU Mowoch x3
- Thaxll 6y agoAnyone who works on Linux probably knows that sometime you need to sacrifice a child...
- gyehuda 6y agoNaming open source projects is a challenging task. The team hears this all the time, but hey, open source is about code you can use, a community you can join to make the code better, and the pride that people around the world want to use your code. If you are on a blue-team, you'll want to look at this project. If you are making lists of names that distract attention this goes on it.
- ignoramous 6y agoIf someone names their CLI JDahmer which then requires me to use it jdahmer <command> <options>, it is going to cringe me out. That said, I totally respect the project's decision to name it so. I'm sorry if it comes off as an outrage or an unnecessary rant. I understand it takes away the energy which is otherwise better spent on a constructive discussion of this excellent project, which is also surprisingly well-maintained. But, I hope you realise why the name is controversial, especially since everyone that has to use the project has to get used to seeing/reading/using/typing Moloch everywhere from the docs, to the CLI, to the FAQ pages, and what-not. May be the project can explain their rationale in a separate webpage (if they want to) and that might help?
- xenihn 6y agoHonestly the first thing that came to my mind was Watchmen.
- travbrack 6y agoAt least it's not trafficly
- phs318u 6y agohttps://slatestarcodex.com/2014/07/30/meditations-on-moloch/ https://slatestarcodex.com/2014/07/30/meditations-on-moloch/ Makes more sense.
- yingw787 6y agoThis is cool, thanks for sharing! I'll be honest, I didn't think AOL did anything...interesting...but this is interesting to me! What are some open-source alternatives / analogues to this product?
- gyehuda 6y agoSorta like https://www.wireshark.org/ https://www.wireshark.org/. But Moloch is a very active project, used by many, and used internally at Verizon Media. Aol is part of Verizon Media (which brought AOL and Yahoo together). Open source is very active here. ;-) Compare also to https://enterprise.verizon.com/products/security/advanced-threat-analytics-and-detection/network-detection-response/ https://enterprise.verizon.com/products/security/advanced-th... (formerly known as Protectwise), which is not open source.
- solarkraft 6y agoIgnorant question for you: What does AOL do these days (I can't view the website for the same reason I don't read TechCrunch articles)? Is it fully about online publications now?
- gyehuda 6y agoYou can see https://www.verizonmedia.com/our-brands https://www.verizonmedia.com/our-brands to see the collection of online brands in the family. You might use lots of these brands today without really noticing. There's a lot of internet content you get via https://www.verizondigitalmedia.com/ https://www.verizondigitalmedia.com/ which is also part of the same company. Aol is still a thing, people do use it. Many people use lots of these brands as part of their internet experience. If you are here, you probably care about internet security. Verizon Media runs a pretty significant Bug Bounty program https://hackerone.com/verizonmedia https://hackerone.com/verizonmedia which you can read about here https://www.protocol.com/hackerone-bug-bounty-virtual-verizon https://www.protocol.com/hackerone-bug-bounty-virtual-verizo... and their blog here https://www.verizonmedia.com/technology/security#/bug-bounty https://www.verizonmedia.com/technology/security#/bug-bounty You might even be looking for a job as an information security professional. You can join "The Paranoids" team (now that's a good name, don't you think!) by checking out some of their jobs. https://www.verizonmedia.com/careers/search.html?q=paranoids https://www.verizonmedia.com/careers/search.html?q=paranoids
- seemslegit 6y agoThis should work nicely with Google Behemoth and Facebook Beelzebub
- burpsnard 6y agoSee also : SATAN https://en.m.wikipedia.org/wiki/Security_Administrator_Tool_for_Analyzing_Networks https://en.m.wikipedia.org/wiki/Security_Administrator_Tool_...
- awsanswers 6y agoAOL github account's location is Dulles, VA. Thinking of what this does, the name's meaning and its location in 'the corridor' is giving me the creeps
- gyehuda 6y agoI fixed that for you https://github.com/aol https://github.com/aol
- awsanswers 6y agoYeah much less creepy
- nerdbaggy 6y agoIt’s been there since before the Mapquest days.
- cfeduke 6y agoAOL's location has been there for a very long time.
- deleted 6y ago[deleted]
- dguido 6y agoFun fact, Moloch was initially created with funding from DARPA's famous Cyber Fast Track program! It's great to see that Moloch is still going strong since ~2013. There is some previous discussion of Moloch when it was released in this older thread: https://news.ycombinator.com/item?id=20586005 https://news.ycombinator.com/item?id=20586005
- awick 6y agoNot sure if you are joking, but Moloch was never part of DARPA's short lived Cyber Fast Track program. :) We do welcome contributions from everyone, and lots of different folks use Moloch. If interested join us over at https://molo.ch https://molo.ch
- dguido 6y agoThat's odd. Are you one of the original authors? The CFT project list had Moloch on it. I'll try and dig it up, it's probably floating around my Google Drive. You may want to speak with Eoin Miller, as I believe he was the point of contact for the project in the document I'm thinking of.
- dguido 6y agoFuckkkkk I think I found the source of my confusion. I am wrong, you are right. I DID find documents about Moloch floating around my Google Drive from ~2013-ish. I believe I invited your co-author Eion to present at a conference I was running, THREADS, in 2014 and that he was not able to make it. The focus the _year prior_ was exclusively on DARPA CFT. I combined those two events in my head and thought your project got some seed funding from DARPA too. I'm sorry! Here is the conference: THREADS 2014 when you were invited: https://github.com/trailofbits/threads/tree/master/2014 https://github.com/trailofbits/threads/tree/master/2014 THREADS 2013 was a retrospective on DARPA CFT: https://github.com/trailofbits/threads/tree/master/2013 https://github.com/trailofbits/threads/tree/master/2013
- whyhead 6y agoThanks, John. I tried to do something a while back with wireshark but I couldn't get the thing to scale. Gonna give this one a hard look.
- dang 6y agoA thread from last year: https://news.ycombinator.com/item?id=20586005 https://news.ycombinator.com/item?id=20586005
- dang 6y agoUrl changed from https://github.com/aol/moloch https://github.com/aol/moloch, which points to this.
- jstewartmobile 6y agoAs the n95 masks are going on, other masks are coming off.
- hprotagonist 6y agoThis gets posted once in a while and I always, without fail, think of Ginsburg. Moloch whose mind is pure machinery! Moloch whose blood is running money! Moloch whose fingers are ten armies! Moloch whose breast is a cannibal dynamo! Moloch whose ear is a smoking tomb!
- caconym_ 6y agoYeah, I literally heard that line in my head immediately after reading the title of this post. What else are we supposed to think of?
- burpsnard 6y agoThe game
- bigiain 6y agoAnd whenever anything here reminds me of Ginsberg, I always, without fail, remember: “The best minds of my generation are thinking about how to make people click ads.” –Jeff Hammerbacher
- mdonahoe 6y agoFor the uninitiated (like myself) Allen Ginsberg's poem "Howl" starts with the line: "I saw the best minds of my generation destroyed by madness"
- sergiotapia 6y agoSeriously "Moloch", was Lucifer or Satan taken already? Edit: And the logo is an owl lmao hard pass - what can they gain from this?
- simlevesque 6y agowhat is your problem with owls ?
- sergiotapia 6y agoHN ain't the place.
- simlevesque 6y agofor owls ?
- kfrzcode 6y agoFor discourse, it seems.
- cryptoquick 6y agoHaha, tell me about it. The downvoters are out in droves. Cowards.
- sibane 6y agoI believe it refers to the symbolic use of the owl of Athena/Minerva in Enlightenment-era secret societies and elite clubs like the Bohemian Grove. Alex Jones tends to call the owl Moloch, an old Semitic deity linked to child sacrifice (and actually more commonly depicted as a horned bovine kind of creature), because he believes those societies are at their core satanic. That's pretty heavy symbolism for an open source project.
- rsynnott 6y agoWell, there's already a database called Voldemort...
- gyehuda 6y agoAccording to my sources, DARPA wasn't involved in any of the child sacrifices, but they might use Moloch to help secure their networks.
- dguido 6y agoYep! As far as I'm aware, a number of military services (I think the USAF) have been public about using Moloch on their networks. It was money well spent by DARPA.
- gyehuda 6y agoMoloch is free. :-)
- cryptoquick 6y agoWell, Moloch is free now. DARPA had to make a lot of sacrifices to make that happen, though.
- gyehuda 6y agoThey do an amazing job keeping it secret too. Can't even find them on the contributor list https://github.com/aol/moloch/graphs/contributors https://github.com/aol/moloch/graphs/contributors
- fireattack 6y agoOff-topic question: does this graph mean that top two people write 99% of the code? Or they're just sharing the account(s)?
- cryptoquick 6y agoThose are those who contributed code. Unfortunately, defense contract money doesn't have a tab on GitHub. Trust me, I've used palantir/tslint
- omarforgotpwd 6y agoI’m sorry. AOL?
- kfrzcode 6y agoThe amount of professional-grade hand-wringing virtue-signaling in this comment section makes me feel ill. Why get bent out of shape over the name of a software project? It's virtually a meaningless factor in day-to-day life. What about hearing phrases "Sacrificing a Chicken to Moloch," the "spirit cooking" culture and related symbolism rampant in elite political circles? Shouldn't we be more interested in that? Lots of low-hanging fruit to pick, I guess. Someday I might unlock the "downvote" ability on this platform. Until then my opinions don't carry weight here. Also, uhh who decides the "threshold" for downvoting? Hint: nobody knows. [0] This platform has become more of an echo chamber than a host of rational discussion based on merit. I suppose that's a problem with growth. There's a lot of conversation around "Moloch" as a name and the subsequent emotional responses... but not a lot of discussion about the tech at-hand. And it's a repost. Where's the value? [0]: https://news.ycombinator.com/newsfaq.html https://news.ycombinator.com/newsfaq.html Edit: while this is high-ranking comment, I'd point out that if I had the ability I would have just downvoted the comments I didn't like. Take that for whatever it's worth.
- gyehuda 6y agoPreach Brother! Moloch is a serious open source project, run by serious people who care about network security. They published their code under an open source license, showing off just how confident they are that it is solid. You can use this project to inspect packets on your network, you can learn how they built it and become a valuable network security engineer with a job somewhere finding people trying to hack in to your site. You can propose modifications to make this even better (and if your code is good enough, it will get accepted and used by security teams around the world). Or you could focus on the name they chose and the name of the company at the time this was published. I'd seize the opportunity to talk tech and focus on network security. Infosec jobs pay better than brand marketing jobs.
- kfrzcode 6y agoThe fact we have access to tools like this (and many, many others) is so damn motivating. I am working through some structured course material in preparation for net-sec baseline `$vendor-$certification` exams, and there are so many complexities and rabbit holes that it's easy to be overwhelmed. Some nights are harder than others to see forest for the trees when it feels I'm trying to understand how to pick up a single pine cone to study. Then I see projects like this, or Jaeger/OpenZipkin, or Chaos Monkey, and I'm simply inspired. Much like listening to a killer record. I don't spend time feeling sorry for myself because an ancient, somewhat esoteric proper noun was used to describe the project.
- frequentnapper 6y agointeresting. didn't realize till now that .ch was a TLD for switzerland!
- simlevesque 6y agoIt means Confœderatio Helvetica
- voz_ 6y agoPardon my ignorance - Is this the same AOL as America Online?
- kfrzcode 6y agoIt certainly is!
- gyehuda 6y agoWhen this project was first published, the name of the company that published it was AOL. Before that, the company was called America Online. Subsequently it was named Oath, and now it's Verizon Media. Names change. The common theme: names are not easy, sometimes they are beloved brands, sometimes they fall out of favor. Sometimes they were just bad ideas from the start, but happened anyway. This gives rise to an interesting challenge for open source projects when you have an open source project in a github org, and the name of your company changes (or your company gets acquired), should you move the project? The problem is real since you don't want to lose your community, but you don't want to be stuck in the past.
- deleted 6y ago[deleted]
- narrator 6y agoYou'd think with a name like that that this would be a daemon that would take as input the cores of killed child processes.
- pgsbathhouse2 6y agoNaming things is hard in software. But some people won't even bother trying.
- X6S1x6Okd1st 6y agoIs there anyone that uses something like this in a home/personal setting?
- jlgaddis 6y agoThe Filesystem Hierarchy Standard [0] has been around for ~25 years but it still took quite a long time before most Linux distros decided to adhere to it (for the most part; some still do "non-standard" things at times). Now that we're to that point, please stop screwing it up and coming up wih your own locations for application binaries, data, etc. To be clear, the "/data" directory -- under which Moloch's pre-built packages apparently install to, according to the README [1] -- is not part of the FHS. --- [0]: https://en.wikipedia.org/wiki/Filesystem_Hierarchy_Standard https://en.wikipedia.org/wiki/Filesystem_Hierarchy_Standard [1]: https://raw.githubusercontent.com/aol/moloch/master/release/README.txt https://raw.githubusercontent.com/aol/moloch/master/release/...
- traden210 6y agoThis is probably intended to be run in a container, but as that's not specified, it's certainly extremely weird.
- deathanatos 6y agoEven inside a container, you want stuff in the standard places, for minimal surprises.
- peterwwillis 6y agoIf you look at the source code of most open source software, the devs typically install by default to arbitrary directories. They either don't know about, or care about, operating system standards. When Linux distributions package that software, they choose to change how that software works to align with standards. Usually their own standards, but those can often align with distro-independent standards too. But as a packager, you expect that you'll be modifying paths and making patches. So really it doesn't matter where a random dev decides to install their software to by default, since either A) it's proprietary and it's non-standard anyway, or B) it's up to us to package it the way we want it.
- brigandish 6y agoNot everyone is using a package manager for every install, and even then it's not a magic bullet. I'm a Mac user and I've no idea why devs insist on installing to places like /usr/local or putting configuration in a dot file in my home directory when there's a standard place to put those (~/Library/Application Support), among other well thought out directory structuring[1]. Further to this, the moment you want to install somewhere else or have a dependency elsewhere (e.g. /opt/anything) builds break because of hard coded paths and poor assumptions. To top it all, I've no idea why a package manager like Homebrew then contributes to all of this by acting like no of this matters because a Mac is just like Linux, apparently. If they don't like Apple's way then why not XDG[2]? Reinventing the wheel badly is more than just hubris, it breaks things and it's annoying. [1] https://developer.apple.com/library/archive/documentation/MacOSX/Conceptual/BPFrameworks/Concepts/FrameworkAnatomy.html https://developer.apple.com/library/archive/documentation/Ma... [2] https://specifications.freedesktop.org/basedir-spec/latest/ https://specifications.freedesktop.org/basedir-spec/latest/
- jzer0cool 6y agoI read the instructions but was not clear. Is this just a pcap viewer or something more? The way I read it, it appears to be a running daemon which listens on all port which saves the pcap file, which then exposes API's for accessing such data. If you had an application http server running, is traffic sent to Moloch first, and forwarded to the http server like a proxy?
- jlgaddis 6y agoWhen running something like this on a large scale to capture all traffic going across a network, you'd typically use a "network packet broker" (cf. Google) that sends a copy of all traffic to the machine(s) running this software. Your hypothetical application server would not even be aware that this was taking place.
- yters 6y agoThere should be a competition for most useful software with most offensive name. Then we can make all the software stacks of top companies likely to result in jail time when discussed in public. That would be funny.
- deleted 6y ago[deleted]
- nodaemon 6y agoSeriously, AOL named this after the god of child sacrifice, you might want to put a new marketeer on this project, I won't be touching anything with a name like that.