6 ms·

This comes up in my work modestly frequently, generally with a slightly different scenario for the tradeoff between "complicated and considered" versus "cheap a

by patio11 13y ago

This comes up in my work modestly frequently, generally with a slightly different scenario for the tradeoff between "complicated and considered" versus "cheap and dirty but gets the job done." e.g. We could spend 3 weeks using feature vectors and backtesting against prior data to figure out what signals accounts which are likely to churn send (for the purpose of proactively identifying them and attempting to derisk them, like by having the customer success team help them out directly)... or we could write a two-line if statement based on peoples' intuitive understandings of what unsuccessful accounts look like. (No login in 30 days and doesn't have the $STICKY_FEATURE enabled? Churn risk!) The chief benefit of the second approach is that it actually ships, 100% of the time, whereas priorities change and an emergency develops and suddenly 75% complete analysis gets thrown in a git repo and forgotten about. Actually shipping is a very useful feature to have.

lifeisstillgood 13y ago
I get the feeling this is more "era-defining" than that... What hit me was the "Processors are so fast now we can Brute force grep over 100GB in a second". We are entering a world where 20TB on a magnetic disk is viable, but randomly accessing that data could take months to extract. So how we store data on disks will become vitally important to how we use the data - not unlike tape drives of pre-1980s era where rewinding to the front of the tape cost you several minutes of (expensive) waittime. Imagine a scenario where these guys design how to stream logs to the disk to maximise streaming reads, then optimise reading that out to SSD thence to RAM and L2 and so forth. All designed to drive text past a regex running at bazillions of times a second. Lets call it the New Brute Force, where its just as much effort to get out the door as elegant algorithms, but it is much much much simpler. And of course, sells more hardware. Expect to see Intel Inside wrapped around the New Brute Force any time soon :-) Edit: And they used Java ! I was expecting low-level C optimisations all over the place
- snewman 13y ago
  Indeed, in the not too distant future we expect to move to spinning disks, with the data arranged for streaming reads. Streaming off disk can be pretty fast. LZ4 compression is fast enough to give us another big boost without turning CPU into a bottleneck. But the big enabler will be spreading data across disks, so that in principle we can use every spindle we own in service of every nontrivial query. The fact that server logs have a very high write-to-read ratio really helps us out here. Today, we're keeping everything on SSD -- twice, in fact, to maintain a hot spare for each server. SSD prices have fallen to the point where, even renting from Amazon, we can make the cost model work; and SSD is a great crutch as we tune our storage layer. But spinning disk will be a lot cheaper. As for Java: we've been pretty surprised ourselves. Going in, we expected to need a lot more "native" (C) code. As it turned out, currently the only non-Java code we're using is the LZ4 library, and even there we only recently moved to the non-Java implementation. We do soon expect to move a few more bits of code to C, such as the core text-search loop. But based on the performance we've been getting in pure Java, we don't anticipate any need to move more than a fraction of 1% of our overall code base. We do have some code that lives in a sort of middle ground -- Java code written like it was in C. Our database buffers live in a set of 2GB byte arrays, soon to be replaced by native memory blocks. We do our own (very simple) memory allocation within that, and use a sprinkling of Unsafe operations to access it. This gets us closer to C performance (not all the way there), without any of the hassles of bridging across languages. This part is still well under 1% of our overall codebase.
  - lifeisstillgood 13y ago
    Amazing how much work 'simple' is :-) I am reminded of John Carmack's comment on Oculus - he was amazed that the hardware had the power but the headsets performed badly - then he looked at the code and saw seas of abstractions on abstractions. Good luck stripping out the layers guys !
    - innguest 13y ago
      Interesting. That must be why C/C++ is always the go-to solution. Instead of thinking "too many abstractions are making this code slow, therefore let's get rid of the abstractions" I usually had rather pick a better language where abstractions have little or no penalty (Haskell, Scheme, etc).
      - TheSOB888 13y ago
        Yeah, but the abstractions have a penalty on your mind.
        mercurial 13y ago
        They do? In my experience, lack of abstraction (often due to not analyzing the issue at hand thoroughly) results in non-abstract, verbose, hard to understand and refactor code. It's the difference between, say, building an SQL query by appending strings to a buffer (move one line and everything blows up) and building a model of your query (projection, etc...). Sure, the abstraction means more code, but it's much more easier to manipulate and considerably less risk-prone. It won't be faster than doing it the other way, but it won't be necessarily measurably slower.
      - lifeisstillgood 13y ago
        Maybe we are talking about different abstractions. Perhaps libraries? I reimplemented (probably badly) session handling code in wsgi (python) - because the Django code was over 2000 lines long and used other libraries and I did not understand it - especially when all I wanted was to store a 128bit number in a client cookie and then look it up when I saw it again. So the idea is simple (cookie sessions) but the different ways of implementing it can hold complexity, errors and abstractions. There is nothing stopping the same happening with Haskell - I can I am sure write terrible code even in the best languages (see my entire output for proof :-)
- npsimons 13y ago
  What hit me was the "Processors are so fast now we can Brute force grep over 100GB in a second". Ironically, this may only be because grep uses very well tuned, not immediately straightforward algorithms. See in particular http://lists.freebsd.org/pipermail/freebsd-current/2010-August/019310.html http://lists.freebsd.org/pipermail/freebsd-current/2010-Augu...
  - deleted 13y ago
    [deleted]
  - up_and_up 13y ago
    > The key to making programs fast is to make them do practically nothing. Great quote from that referenced post.
  - jug6ernaut 13y ago
    Awesome read, thanks for sharing! Almost worth a HN post in itself if you ask me.
    - npsimons 13y ago
      I think it was posted a while back; it was an interesting enough article that it stuck in my head and I was able to find it pretty easily by googling "why is grep so fast?". Ah, here it is: https://news.ycombinator.com/item?id=2393587 https://news.ycombinator.com/item?id=2393587 And again! https://news.ycombinator.com/item?id=6813937 https://news.ycombinator.com/item?id=6813937 And duplicate submissions are listed in those postings.
- hvidgaard 13y ago
  When analyzing big data algorithms, the amount of computation by the CPU is often irrelevant. It's ignored. What matters is disk reads and disk access patterns. If you can make a scanning algorithm that start at byte 0, and reads everything once in a sequential manner, you can make O(n^4) computation with it. The CPU will be long finished with the inefficient computation by the time the next set of data is retrieved from the (spinning) disk. And on using Java. It's actually pretty fast, and certainly fast enough for a prototype. Once you have the stack build, you can benchmark and optimize the bits that needs to be optimized, usually this means C or handcrafted ASM.
  - dekhn 13y ago
    I don't think I've ever heard anybody claim that streaming algorithms permit O(n4) computation. That seems implausible. Most streaming algorithms are linear, or sublinear. I'm not sure what you meant by O or n or 4, however.
    - raphaelj 13y ago
      Indeed, there will always be an instant, given that "n" is increasing toward infinity, when the computation time will take over the streaming time. Systems complexity are always bounded by their algorithm having the greatest complexity. As streaming is linear, there will always be a value of "n" after which the computation will take over, even with a very quasi-linear complexity (even an O(n^1.00000001) computation will be slower than the streaming for very very large values of n).
      - hvidgaard 13y ago
        I should have mentioned that it implies that you can work on sufficiently small values of n, usually measured in how many cache lines it takes up. Saying that "there will always be a value of "n" after which the computation will take over" is, while true, completely theoretical. Not that it have its uses, but I have yet to see a IO heavy process being limited by the CPU. If it's streaming, it's usually also quite simple to process, and if it's random access the CPU is doing something else most of the time.
nicholassmith 13y ago
I will always pick the cheap and dirty option until the project is at a stage where you have a lot of breathing room. Simple code trumps fancy code in reality where you don't have a large team, and you don't have a lot of time and you need something that works even if it's only half as efficient a solution.
- lifeisstillgood 13y ago
  this is the point where I say you won't get breathing room if your code is all quick and dirty. There is a balance between quick and dirty and well-formed and not needing tending every few weeks. Just do high quality code that will remain maintainable into the future. If the business feels that has too much of a financial or time penalty the business needs to find money to pay for more high quality (slower) developers, or to find a different you. Don't take someone else's problem / role on as your own to solve - it won't help you or them.
neilk 13y ago
I see what you're saying, but the article was not about "cheap and dirty" at all. They describe a precisely tuned system, riding on the controlled application of brute force. Like a rocket to Mars. It reminds me a lot of the paper that Poul-Henning Kamp wrote about "1975 programming". Few of us really appreciate how to use modern hardware. https://www.varnish-cache.org/trac/wiki/ArchitectNotes https://www.varnish-cache.org/trac/wiki/ArchitectNotes
- eternalban 13y ago
  This approach is precisely what PHK was arguing against. That position paper basically says "Go ahead and do the logically correct thing and let the OS and VM (memory) take care of making it efficient".
agibsonccc 13y ago
I think it depends on the context. Machine Learning and statistics isn't magic, and you definitely don't want to use it for the wrong reasons if you can figure it out with domain knowledge. I would just like to throw in here if you are going to be featurizing users to do churn prediction, I would highly recommend doing a whole data pipeline and do profit curves. A data driven approach is only as good as the algorithms used. If you used profit curves to augment churn prediction (churn prediction is a pre req), you can figure out how much a user is actually worth bringing back. This also allows you to figure out an optimal budget to profit ratio on a cost of a user. That being said, use it only if you have enough data to be accurate. And also only use it for the right reasons. Done routinely, I believe it can be a great investment.

by patio11 13y ago

lifeisstillgood 13y ago

I get the feeling this is more "era-defining" than that... What hit me was the "Processors are so fast now we can Brute force grep over 100GB in a second". We are entering a world where 20TB on a magnetic disk is viable, but randomly accessing that data could take months to extract. So how we store data on disks will become vitally important to how we use the data - not unlike tape drives of pre-1980s era where rewinding to the front of the tape cost you several minutes of (expensive) waittime. Imagine a scenario where these guys design how to stream logs to the disk to maximise streaming reads, then optimise reading that out to SSD thence to RAM and L2 and so forth. All designed to drive text past a regex running at bazillions of times a second. Lets call it the New Brute Force, where its just as much effort to get out the door as elegant algorithms, but it is much much much simpler. And of course, sells more hardware. Expect to see Intel Inside wrapped around the New Brute Force any time soon :-) Edit: And they used Java ! I was expecting low-level C optimisations all over the place

snewman 13y ago
Indeed, in the not too distant future we expect to move to spinning disks, with the data arranged for streaming reads. Streaming off disk can be pretty fast. LZ4 compression is fast enough to give us another big boost without turning CPU into a bottleneck. But the big enabler will be spreading data across disks, so that in principle we can use every spindle we own in service of every nontrivial query. The fact that server logs have a very high write-to-read ratio really helps us out here. Today, we're keeping everything on SSD -- twice, in fact, to maintain a hot spare for each server. SSD prices have fallen to the point where, even renting from Amazon, we can make the cost model work; and SSD is a great crutch as we tune our storage layer. But spinning disk will be a lot cheaper. As for Java: we've been pretty surprised ourselves. Going in, we expected to need a lot more "native" (C) code. As it turned out, currently the only non-Java code we're using is the LZ4 library, and even there we only recently moved to the non-Java implementation. We do soon expect to move a few more bits of code to C, such as the core text-search loop. But based on the performance we've been getting in pure Java, we don't anticipate any need to move more than a fraction of 1% of our overall code base. We do have some code that lives in a sort of middle ground -- Java code written like it was in C. Our database buffers live in a set of 2GB byte arrays, soon to be replaced by native memory blocks. We do our own (very simple) memory allocation within that, and use a sprinkling of Unsafe operations to access it. This gets us closer to C performance (not all the way there), without any of the hassles of bridging across languages. This part is still well under 1% of our overall codebase.
- lifeisstillgood 13y ago
  Amazing how much work 'simple' is :-) I am reminded of John Carmack's comment on Oculus - he was amazed that the hardware had the power but the headsets performed badly - then he looked at the code and saw seas of abstractions on abstractions. Good luck stripping out the layers guys !
  - innguest 13y ago
    Interesting. That must be why C/C++ is always the go-to solution. Instead of thinking "too many abstractions are making this code slow, therefore let's get rid of the abstractions" I usually had rather pick a better language where abstractions have little or no penalty (Haskell, Scheme, etc).
    - TheSOB888 13y ago
      Yeah, but the abstractions have a penalty on your mind.
      - mercurial 13y ago
        They do? In my experience, lack of abstraction (often due to not analyzing the issue at hand thoroughly) results in non-abstract, verbose, hard to understand and refactor code. It's the difference between, say, building an SQL query by appending strings to a buffer (move one line and everything blows up) and building a model of your query (projection, etc...). Sure, the abstraction means more code, but it's much more easier to manipulate and considerably less risk-prone. It won't be faster than doing it the other way, but it won't be necessarily measurably slower.
    - lifeisstillgood 13y ago
      Maybe we are talking about different abstractions. Perhaps libraries? I reimplemented (probably badly) session handling code in wsgi (python) - because the Django code was over 2000 lines long and used other libraries and I did not understand it - especially when all I wanted was to store a 128bit number in a client cookie and then look it up when I saw it again. So the idea is simple (cookie sessions) but the different ways of implementing it can hold complexity, errors and abstractions. There is nothing stopping the same happening with Haskell - I can I am sure write terrible code even in the best languages (see my entire output for proof :-)
npsimons 13y ago
What hit me was the "Processors are so fast now we can Brute force grep over 100GB in a second". Ironically, this may only be because grep uses very well tuned, not immediately straightforward algorithms. See in particular http://lists.freebsd.org/pipermail/freebsd-current/2010-August/019310.html http://lists.freebsd.org/pipermail/freebsd-current/2010-Augu...
- deleted 13y ago
  [deleted]
- up_and_up 13y ago
  > The key to making programs fast is to make them do practically nothing. Great quote from that referenced post.
- jug6ernaut 13y ago
  Awesome read, thanks for sharing! Almost worth a HN post in itself if you ask me.
  - npsimons 13y ago
    I think it was posted a while back; it was an interesting enough article that it stuck in my head and I was able to find it pretty easily by googling "why is grep so fast?". Ah, here it is: https://news.ycombinator.com/item?id=2393587 https://news.ycombinator.com/item?id=2393587 And again! https://news.ycombinator.com/item?id=6813937 https://news.ycombinator.com/item?id=6813937 And duplicate submissions are listed in those postings.
hvidgaard 13y ago
When analyzing big data algorithms, the amount of computation by the CPU is often irrelevant. It's ignored. What matters is disk reads and disk access patterns. If you can make a scanning algorithm that start at byte 0, and reads everything once in a sequential manner, you can make O(n^4) computation with it. The CPU will be long finished with the inefficient computation by the time the next set of data is retrieved from the (spinning) disk. And on using Java. It's actually pretty fast, and certainly fast enough for a prototype. Once you have the stack build, you can benchmark and optimize the bits that needs to be optimized, usually this means C or handcrafted ASM.
- dekhn 13y ago
  I don't think I've ever heard anybody claim that streaming algorithms permit O(n4) computation. That seems implausible. Most streaming algorithms are linear, or sublinear. I'm not sure what you meant by O or n or 4, however.
  - raphaelj 13y ago
    Indeed, there will always be an instant, given that "n" is increasing toward infinity, when the computation time will take over the streaming time. Systems complexity are always bounded by their algorithm having the greatest complexity. As streaming is linear, there will always be a value of "n" after which the computation will take over, even with a very quasi-linear complexity (even an O(n^1.00000001) computation will be slower than the streaming for very very large values of n).
    - hvidgaard 13y ago
      I should have mentioned that it implies that you can work on sufficiently small values of n, usually measured in how many cache lines it takes up. Saying that "there will always be a value of "n" after which the computation will take over" is, while true, completely theoretical. Not that it have its uses, but I have yet to see a IO heavy process being limited by the CPU. If it's streaming, it's usually also quite simple to process, and if it's random access the CPU is doing something else most of the time.

nicholassmith 13y ago

I will always pick the cheap and dirty option until the project is at a stage where you have a lot of breathing room. Simple code trumps fancy code in reality where you don't have a large team, and you don't have a lot of time and you need something that works even if it's only half as efficient a solution.

lifeisstillgood 13y ago
this is the point where I say you won't get breathing room if your code is all quick and dirty. There is a balance between quick and dirty and well-formed and not needing tending every few weeks. Just do high quality code that will remain maintainable into the future. If the business feels that has too much of a financial or time penalty the business needs to find money to pay for more high quality (slower) developers, or to find a different you. Don't take someone else's problem / role on as your own to solve - it won't help you or them.

neilk 13y ago

I see what you're saying, but the article was not about "cheap and dirty" at all. They describe a precisely tuned system, riding on the controlled application of brute force. Like a rocket to Mars. It reminds me a lot of the paper that Poul-Henning Kamp wrote about "1975 programming". Few of us really appreciate how to use modern hardware. https://www.varnish-cache.org/trac/wiki/ArchitectNotes https://www.varnish-cache.org/trac/wiki/ArchitectNotes

eternalban 13y ago
This approach is precisely what PHK was arguing against. That position paper basically says "Go ahead and do the logically correct thing and let the OS and VM (memory) take care of making it efficient".

agibsonccc 13y ago

I think it depends on the context. Machine Learning and statistics isn't magic, and you definitely don't want to use it for the wrong reasons if you can figure it out with domain knowledge. I would just like to throw in here if you are going to be featurizing users to do churn prediction, I would highly recommend doing a whole data pipeline and do profit curves. A data driven approach is only as good as the algorithms used. If you used profit curves to augment churn prediction (churn prediction is a pre req), you can figure out how much a user is actually worth bringing back. This also allows you to figure out an optimal budget to profit ratio on a cost of a user. That being said, use it only if you have enough data to be accurate. And also only use it for the right reasons. Done routinely, I believe it can be a great investment.