windlep

19 ms·

91.
by windlep 11y ago
Yes, exactly. Which is why we switched from Go to Python (using twisted), and dropped our memory use substantially for holding open hundreds of thousands (and soon millions) of websocket connections. M:N schedulers and the primitives they u
92.
by windlep 11y ago
Any plans to open-source the futures-based Rust framework? :)
93.
by windlep 11y ago
This ignores the problem of not using a framework, that is, for a sufficiently complex project you will eventually end up with your own ad-hoc framework if you don't use an existing one. Maybe it's better, maybe worse, but one wil
94.
by windlep 11y ago
It will be interesting to see if this one takes off more than a similar one that launched about a decade ago now... http://techcrunch.com/2007/11/15/social-networking-platform-... CrowdVine also let people se
95.
by windlep 11y ago
If we ignore waste for a minute, this article highlights a key issue for why nuclear in the USA will never be as cheap as France or South Korea. Those countries can impose a single builder, single supplier requirement and enforce it the w
96.
by windlep 11y ago
Yes, that was a bit of snark, sorry. Was hoping that would emphasize how huge of news it is that Python moved past PHP even on such an arbitrary language benchmark.
97.
by windlep 11y ago
Amongst what bubble of people you're entangled with? I consider this a troll question because frequently I see HN questions like this completely disconnected from programming communities at large (which do not participate in HN), extra
98.
by windlep 11y ago
Yes, Python is dying, that's why it just moved up in the Tiobe rankings to #5, displacing PHP (which obviously no one ever uses): http://www.tiobe.com/index.php/content/paperinfo/tpci/index.... It&#
99.
by windlep 11y ago
A great example was given at the Chrome dev summit, which I think highlights the utility. First, remember that you only need to click 'Yes' when your browser asks you for Push permission. You have revealed nothing new to the websi
100.
by windlep 11y ago
Unfortunately you can click on the notification in Firefox and immediately kill all further notifications from the spammers. Hopefully other implementations will be more friendly to spammers and hide the 'make it stop' button deep
101.
by windlep 11y ago
Firefox implements a quota system that restricts how many notifications may be sent "without user interaction". This is to reduce the likelihood that one could use this effectively for background tracking and to ensure you won&#x2
102.
by windlep 11y ago
This is cool, but not really the problem I have, nor many I've talked to have when actually doing something 'real' in Haskell. (Granted we're a rare breed, actually trying to use Haskell for real-world use outside acad
103.
by windlep 11y ago
In the past 3 years on usenet, something has changed though. Perhaps in response to this article and/or the others talking about it. Some studios are getting drastically faster at getting their content pulled within minutes of nzb&#x27
104.
by windlep 11y ago
There is a massive cultural pressure on women in America (and in many countries) to have children. Its in the movies, magazines, and comes up amazingly often in casual conversations. Regardless of what people say, it also comes with a socia
105.
by windlep 11y ago
It seems people aren't thrilled with the prospect of debris landing on people underneath. More likely we'll end up with bigger drones that grab smaller drones with a net so they won't fall at this rate, or some sort of net-gu
106.
by windlep 11y ago
I have two samples of a websocket echo program in Haskell, and was unfortunately unable to make it not leak memory merely sending the same data back over the connections repeatedly. I will admit I haven't tested this lately, and I didn
107.
by windlep 11y ago
We're using twisted, but I believe Python 3's asyncio has a similar feature with use of non-blocking sockets, which is that you can add a hook to be triggered when too much data accumulates in user-space (can't be flushed to
108.
by windlep 11y ago
Nope, so, the goal here is to reduce how many machines (each with their own RAM limits) are used. The task is holding open bidirectional SSL wrapped long-lived websocket connections. They're held open for hours at a time, since we need
109.
by windlep 11y ago
Yep, this is what I meant by 'goroutine pools'. The select statements were on the sending side to ensure if the feed channel was full we wouldn't retain too much additional state. It works, but at that point its starting to l
110.
by windlep 11y ago
We used Python 2.7 for pypy compatibility. I really dig the new async/await primitives added to Python 3.5 and think the asyncio path is the best path forward. I hope the work to get all the latest asyncio stuff working on PyPy picks u
111.
by windlep 11y ago
Regardless of how good you are, its easier to write code that has a predictable runtime memory footprint if you can actually predict how things will behave. Predicting an M:N scheduler is.... not easy. This is part of why Rust removed the M
112.
by windlep 11y ago
Why would it? Using twisted (or py3's asyncio), the event loop is spinning, there is no thread for Python to be switching to. Except that we're using a thread-pool of course (to make external blocking network calls), and those thr
113.
by windlep 11y ago
I have to note this, because I think it deserves quite a bit more attention. SSL is extremely expensive, on RAM (perhaps the implementations have optimized for throughput over RAM). I have yet to benchmark any SSL implementation in any lang
114.
by windlep 11y ago
So do I! Eventually you 'top-out' in a language/framework though... and then its all tears.
115.
by windlep 11y ago
I should note, we don't care about throughput for the most part. Our constraint is purely the memory use of holding open the connections. The aim is to hold as many connections as possible within 10-20% of the machines RAM, and not exc
116.
by windlep 11y ago
This was before Go 1.4, which dropped memory use per goroutine from 8kb -> 2kb. They're fairly close now (minus my leaking goroutines). ;)
117.
by windlep 11y ago
It could've been. Though now that so much of heka is involved in moving data between the Lua layer, turning it into plain C is a better option. trink, one of the heka authors has a project called hindsight that does just that.
118.
by windlep 11y ago
I mentioned during the talk, that the reason for giving it a spin in Python in the first place was that we had to fill in gaps in a Go library. While there's lots of Go libraries, many of them are immature, and have chunks of functiona
119.
by windlep 11y ago
I meant that partly as a joke, I do happen to like Rust though. And in the context of this technical problem, its memory control would make it an attractive possibility (as would C/C++/etc). Lots of context missing as these are me
120.
by windlep 11y ago
For heka, one of the things we wanted besides performance, was a small, easy-to-distribute binary you could drop on a system and 'just run'. Go is fantastic for distribution thanks to the small staticly linked binaries you can get

More ›