5 ms·
As one of the main drivers of Python at Netflix, this one is near and dear to my heart. Let me know if you have any questions, or come visit our booth!
by jedberg 14y ago
As one of the main drivers of Python at Netflix, this one is near and dear to my heart. Let me know if you have any questions, or come visit our booth!
- arpineh 14y agoHi. There was many interesting things listed at the blog post. I have been planning a system similar in idea to Sting, your OLAP server (?). I'm planning a light RPC style system on top of Pandas dataframe objects, transferring HTTP requests to Pandas method calls. Memory usage of such a system with our data is too expensive if all data is in use. So, I'm investigating possible caching and database solutions. Could you perhaps elaborate on the implementation of Sting's in memory database? [posted from yet another HN account]
- jedberg 14y agoI just posted this above: "I don't have any info about Sting, but we're working on some blog posts that go deeper into each of these topics. I'll make sure that the interest in Sting is accounted for in the prioritization." So now that's +2. :)
- daemon13 14y ago+3 please :-) The idea of marrying "slow" Python and OLAP is disturbing my sleep!
- pwang 14y agoHave you looked at the Cubes project? http://databrewery.org/cubes.html http://databrewery.org/cubes.html
- yellow 14y agoI'm pushing for more python use at my company as well. It's tough as everyone is drinking the ruby kool-aid. P.S. I really enjoyed your talk at QCon SF last year!
- irahul 14y ago> It's tough as everyone is drinking the ruby kool-aid. Ruby and Python are similar in terms of execution speed and developer productivity. If your work requires something that Python has and Ruby doesn't(say numpy), then it might make sense; otherwise what would you accomplish by switching from Ruby to Python?
- jonascopenhagen 14y agowhat would you accomplish by switching from Ruby to Python? Simpler syntax?
- mixmastamyk 14y agoA larger and more varied developer community, resulting in applicability to a wider range of problems.
- deleted 14y ago[deleted]
- e1ven 14y agoKeep it up! Posts like these are great on a tech level, but they're also good recruitment - Seeing Python, along with hackers that I respect (like you) at Netflix make me at least the littlest bit tempted to go there ;)
- jedberg 14y agoYou've caught on to our ploy! But in all seriousness, we like to share what we do because sharing is cool and it attracts other people who like to share cool technology, which are the kind of people we like to work with.
- rbanffy 14y agoI strongly suspect preventing me from visiting PyCon is one of the reasons my CTO scheduled so many product launches for the next two weeks...
- pwang 14y agoYou should also come by our booth (Continuum Analytics) and we can talk shop!
- DoggettCK 14y agoWe were actually talking about your company in our sprint planning meeting a few minutes ago, since we're doing in-house Python analytics. You might be getting a call.
- pwang 14y agoAwesomeness. We are making great headway on Blaze and the new persistence format to supplant HDF5/PyTables (Although, Blaze will always work great with carray, NumPy, and Pytables), and the Numba compiler project is working out great. Wakari (http://wakari.io http://wakari.io) is nearing 1.0 and we're about to turn on paid monthly plans for beefier nodes and GPU nodes, and we have also been in talks with several companies about in-house private deployments. Give us a shout or shoot me an email at pwang (at) continuum.io! :-)
- bsimpson 14y agoYou know how Google claims that by saving fractions of a second for every search that billions of people do, they've saved many man-lives of productivity? As a leading engineer at both Reddit and Netflix, you've undone all of that. Well done.
- qxf2 14y agoUnfair tone. I hope it was in jest. My life is not all about productivity. There is a leisure aspect as well: http://en.chessbase.com/home/TabId/211/PostId/4008728 http://en.chessbase.com/home/TabId/211/PostId/4008728
- jedberg 14y agoI always tell people that I've spent the last six years of my life as a digital crack dealer.
- BrianEatWorld 14y agoWhere can I find more info about Sting? I didn't see a link in the article and some quick Google searches haven't proved fruitful.
- jedberg 14y agoI don't have any info about Sting, but we're working on some blog posts that go deeper into each of these topics. I'll make sure that the interest in Sting is accounted for in the prioritization.
- pwang 14y agoKurt Brown had some slides about data & Python at the NYC Python meetup last October. He did a Sting demo there and some of the slides talk about it (I think... it might be a different thing, though. October was a long time ago..) http://www.meetup.com/nycpython/files/ http://www.meetup.com/nycpython/files/
- tootie 14y agoI have to say that after doing a python project I totally don't get why people love it so much. The syntax is slightly cleaner, but the lack of threading, first-class functions and dependency management tools make it waste at least as much time as it saves. I honestly don't believe in the concept of a programming language being able to make code more or less readable. A Java project with a POM file can be up and running on any OS in about 5 minutes.
- brown9-2 14y agoPython doesn't have first class functions?
- maxerickson 14y agoI often get the idea that people confuse anonymous and first class functions (apparently because they discover the latter expressed in terms of the former).
- nightpool 14y agoa) python-pip combined with virtualenv gives very good requirement support. b) python does have first class functions? c) threading still happens, you just can't take advantage of multiple cores. This isn't as big of a problem as you'd think though, you can still write very fast code.
- frankwiles 14y agoUmmm, not sure where you're getting your information but Python has threads (http://docs.python.org/2/library/threading.html http://docs.python.org/2/library/threading.html), first-class functions (http://en.wikipedia.org/wiki/First-class_function http://en.wikipedia.org/wiki/First-class_function), and dependency management tools (google for pip, easy_install, etc.)
- herge 14y agoI think he was talking about threads that run on multiple processors. Also, packaging is a whole different can of worms (but hopefully will be sorted by distutils2!).
- bryanwb 14y agowould you be interested to talk about these tools on the http://foodfightshow.org http://foodfightshow.org devops podcast? we would love to discuss them with you!
- jedberg 14y agoI would, but I could probably find you people who know more about the tools, like the folks who wrote them. :) Hit me up on email (it's in my profile).
- deleted 14y ago[deleted]
- cpenner461 14y agoHey jedberg - any plans/timeline for open sourcing aminator? It sounds pretty cool...