5 ms·
I wonder what their stack is. I can't readily determine that, which might just be a function of working too hard today. I don't have time to search for the info
by Mz 11y ago
I wonder what their stack is. I can't readily determine that, which might just be a function of working too hard today. I don't have time to search for the info. (I was specifically wondering if they use Hadoop.)
- chrislloyd 11y agoThey're big fans of Clojure (and are sponsoring clojure/conj).
- whalesalad 11y agoMany of us will be there this year and will probably be in our borderline-offensive green shirts so please do say hello!
- rev_bird 11y agoI can't remember seeing them looking for anything other than Postgres, which seems less like "big data" than... regular data. Maybe they don't put their heavy duty stuff in job listings?
- Mz 11y agoThanks. I did see a job listing that noted "familiarity with stuff like Hadoop" (paraphrasing). So it seemed to cast doubt on them using Hadoop, but it neither served to confirm nor deny.
- terravion 11y agoFarming is often in the raster world as opposed to the line/row/text/vector world. Some of things in Postgres might be huge. Farm data sets could easily be only adding a few thousand rows a day, but the objects associated with each row could be several gigabytes. Meaning that the size of the data is bigger than so-called "big data" but the row analysis tool set looks more like your "regular data." However, there's a lot that goes into the raster analysis that's a whole different beast.
- rev_bird 11y agoHuh. Thanks for taking the time to outline this, I don't know why it never occurred to me. I have to admit, I'm unfamiliar with "rasters" in the way you seem to be referencing them. It sounds, though, like the relational bits of the DB are really being used more as a file system than a database, if there are even really distinctions in the first place. If "a couple thousand rows" are basically being used as a metadata store for the rasters, is that an unusual use of the database, or is everybody doing this and I just never had enough data to care?
- whalesalad 11y agoYou'd be surprised at what you can do with PostgreSQL. Also, there are also some really exciting new ways to handle data pipelines using Docker and tiny applications (see Pachyderm) as opposed to classic approaches like HDFS and Hadoop. Not everything needs to be in an enterprise grade multi-node C* cluster to be big data!
- llambda 11y agoThey use Clojure and I believe ClojureScript on the frontend. (I'm not sure about Hadoop, when I had spoken with their devs before this didn't come up--but since they're already on the JVM such tools are a natural fit.) They're one of the many success stories in the community of startups who have chosen Clojure as their primary weapon. (See also: Climate Corp, now owned by Monsanto.) Lisp is having its victories, here and there, but notably in meaningful and economically real ways. Edit: They DO NOT use ClojureScript, sadly. Unlike say, CircleCI or Prismatic, who both have very compelling stories with regard to using Clojure from the front to the backend.
- vcarl 11y agoFrontend is currently Backbone + React, moving towards more React, written in CoffeeScript.
- llambda 11y agoA bit of a shame it's not ClojureScript, to be honest. What's the rationale CoffeeScript?
- vcarl 11y agoI'm relatively new so I can't speak to the original rationale, but right now it fits our needs. I'd love to do some ClojureScript (or just ES2015 through Babel), but right now we're focusing on building stuff instead of playing with languages :)
- llambda 11y ago> ...but right now we're focusing on building stuff instead of playing with languages Which is exactly what's so compelling with regard to ClojureScript and say Om or any one of the React wrappers out there: you guys already use Clojure and ClojureScript and its libraries are incredibly pragmatic and designed for "real work". CoffeeScript would seem more like language play than ClojureScript, but of course I'm speaking as an outsider. :) Also if you haven't already, take a look at David Nolen's recent talk on Om Next[1]. Personally if I were using Clojure on the backend, I'd be pushing hard for it on the frontend too. [1] https://www.youtube.com/watch?v=ByNs9TG30E8 https://www.youtube.com/watch?v=ByNs9TG30E8
- btmills 11y agoThey did a StackShare blog post [0] and podcast in February talking about their stack, and it makes for a pretty good read. About Hadoop specifically, it doesn't show up on their StackShare page [1] anywhere. [0] http://stackshare.io/posts/how-farmlogs-is-building-software-to-power-the-future-of-farming http://stackshare.io/posts/how-farmlogs-is-building-software... [1] http://stackshare.io/farmlogs http://stackshare.io/farmlogs