5 ms·

Awright, so I didn't write the linked piece, but Chris isn't talking about scaling our web traffic by an order of magnitude. We're a small-to-mid-size web stor

by brennen 13y ago

Awright, so I didn't write the linked piece, but Chris isn't talking about scaling our web traffic by an order of magnitude. We're a small-to-mid-size web store & light manufacturer and we do decent business and very occasionally get a bunch of traffic as far as mid-sized web stores go. We're not a big service provider, and we're never going to be one in the sense where we have to worry about scale the way that people with Real Problems do. We do plan to store a bunch more data than we deal with now, but mostly we migrated because we're looking to build a better ball of mud, to borrow a perfectly legitimate analogy from elsewhere in the thread. What we'd like to scale somewhat cleanly is a business which benefits from us building less broken applications, and PostgreSQL is obviously a less broken foundation for the kind of software where it matters if you are correct about things. If anyone would like a reason to let us know at length what incomparable buffoons we must be for any of writing PHP, rolling our own application framework, still owning physical hardware, or getting a little overexcited when the code doesn't break too badly, I will cop to all of those and more in public, but please don't read the linked post as a strong technical claim about Hard Problems in the Land of Webscalonia which demands & merits a detailed refutation. Posts on sparkfun.com don't tend to be written for a web nerd audience (we sell blinkenlights) and most of us survey a very different conceptual landscape on a day-to-day basis from the average HN reader. (For clarity here, I write web stuff at SparkFun, and most of my time lately has been on this project. Author of linked piece is my boss.) Edit: And my apologies if this reply to a rather simple question comes off a bit reactive; consider this my reply to the thread as a whole?

strlen 13y ago
Not reactive at all, well put response to mostly confused posts (including my own). I think there was mis-understanding on what was meant by scaling -- web community took it to mean something very different from what you meant. Honestly this befuddled me as well me as well, so I suspected as much -- I didn't think Sparkfun would need to go to horizontal scaling which I am quite confident you'll likely never need (unless you're doing something very wrong). One way someone else put it, MySQL is great if you're using it as a datastore to build an API layer on top (with relational features being a huge plus as opposed to a K/V store). Postgres, on the other hand, effectively does what an RDBMS was originally meant to do -- it itself acts as that data access API. Also SparkFun is an awesome company -- the SparkFun Arduino kits on my desk do a lot more than blink lights (though they do that too). Please continue to use run-times may be less trendy but which return responses quickly and using physical hardware too.
falcolas 13y ago
Wait... > PostgreSQL is obviously a less broken foundation for the kind of software where it matters if you are correct about things. and > writing PHP, rolling our own application framework In the same explanation? Seems like you still have some points where you can make serious gains in your technology stack that would have bought you more tangible benefits albeit not as much positive publicity.
- brennen 13y ago
  Thanks. I never know I'm actually in a conversation about how I do my job until this comment shows up.
  - falcolas 13y ago
    In my honest opinion, if you choose to change to PostgreSQL because it's "less broken" but still use PHP as your front end, I feel completely justified in asking WTF. Sure, you're securing the barn doors nicely, but the rest of the barn might as well be made of tissue paper. It is your job, and it's not my place to comment about how to do your job; but when making such a public announcement, 'less broken' when wrapped with PHP and custom frameworks isn't a very compelling message.
    - brennen 13y ago
      You can always ask WTF. What else is the internet for? I'll try to give you a face-value answer. If we could just sort of rewrite all of our code in a nicer language than PHP (taking it as read that most of us can think of one or two of those) with a commensurate effort, and the idea somehow just hadn't occurred to me until I read your comment, I probably would have started the project over my morning coffee. In the real world, we have a bunch of existing software, a business to run on it, and a well-developed understanding of the pitfalls of the language and environment. There's no conceivable justification for the amount of effort it would take to rebuild our stuff from scratch in one fell swoop. PHP is a pain in my ass, I yield to almost no one in my visceral contempt for PHP, but PHP is not my problem, and it hasn't been for a long time. What we can do, with a reasonable investment of time and energy, is move more of our business concerns into a more rigorous data store, and explore building new features against the data we model well using other languages and tools. Right now we've got various bits of tooling written in (at least) Ruby, Go, C, Perl, R, Python, and Node.js. Maybe someday we'll really zone in on something and figure out that it's a good time to start slicing off chunks of application and rebuilding them in [better language x] until the PHP has evaporated. Maybe we'll decide that everything should live in [third party system x]. Maybe we won't bother with any of this, because PHP gets a little less heinous with each passing release, we already know what we're doing with it, and there are bigger fish to fry. Reasoning by analogy is frequently suspect anyway, but in your picture, I'm not sure how databases are barn doors. They tend, at least for the kinds of things we're doing, to be much more fundamental to the framing of the barn. Or maybe they're the stuff stashed in the hayloft. I dunno. At any rate, I don't think it's super coherent to argue that all the things must be fixed at once in order to justify improving one of the things.
rpedela 13y ago
Thanks! You answered my question, and it makes sense. There have been times where I have changed underlying technologies because it provided a better foundation for the future even though there was nothing particularly wrong with the existing solution. No worries about the tone or replying to the entire thread instead of just my question. I would also be frustrated with some of the comments in this thread.

by brennen 13y ago

strlen 13y ago

Not reactive at all, well put response to mostly confused posts (including my own). I think there was mis-understanding on what was meant by scaling -- web community took it to mean something very different from what you meant. Honestly this befuddled me as well me as well, so I suspected as much -- I didn't think Sparkfun would need to go to horizontal scaling which I am quite confident you'll likely never need (unless you're doing something very wrong). One way someone else put it, MySQL is great if you're using it as a datastore to build an API layer on top (with relational features being a huge plus as opposed to a K/V store). Postgres, on the other hand, effectively does what an RDBMS was originally meant to do -- it itself acts as that data access API. Also SparkFun is an awesome company -- the SparkFun Arduino kits on my desk do a lot more than blink lights (though they do that too). Please continue to use run-times may be less trendy but which return responses quickly and using physical hardware too.

falcolas 13y ago

Wait... > PostgreSQL is obviously a less broken foundation for the kind of software where it matters if you are correct about things. and > writing PHP, rolling our own application framework In the same explanation? Seems like you still have some points where you can make serious gains in your technology stack that would have bought you more tangible benefits albeit not as much positive publicity.

brennen 13y ago
Thanks. I never know I'm actually in a conversation about how I do my job until this comment shows up.
- falcolas 13y ago
  In my honest opinion, if you choose to change to PostgreSQL because it's "less broken" but still use PHP as your front end, I feel completely justified in asking WTF. Sure, you're securing the barn doors nicely, but the rest of the barn might as well be made of tissue paper. It is your job, and it's not my place to comment about how to do your job; but when making such a public announcement, 'less broken' when wrapped with PHP and custom frameworks isn't a very compelling message.
  - brennen 13y ago
    You can always ask WTF. What else is the internet for? I'll try to give you a face-value answer. If we could just sort of rewrite all of our code in a nicer language than PHP (taking it as read that most of us can think of one or two of those) with a commensurate effort, and the idea somehow just hadn't occurred to me until I read your comment, I probably would have started the project over my morning coffee. In the real world, we have a bunch of existing software, a business to run on it, and a well-developed understanding of the pitfalls of the language and environment. There's no conceivable justification for the amount of effort it would take to rebuild our stuff from scratch in one fell swoop. PHP is a pain in my ass, I yield to almost no one in my visceral contempt for PHP, but PHP is not my problem, and it hasn't been for a long time. What we can do, with a reasonable investment of time and energy, is move more of our business concerns into a more rigorous data store, and explore building new features against the data we model well using other languages and tools. Right now we've got various bits of tooling written in (at least) Ruby, Go, C, Perl, R, Python, and Node.js. Maybe someday we'll really zone in on something and figure out that it's a good time to start slicing off chunks of application and rebuilding them in [better language x] until the PHP has evaporated. Maybe we'll decide that everything should live in [third party system x]. Maybe we won't bother with any of this, because PHP gets a little less heinous with each passing release, we already know what we're doing with it, and there are bigger fish to fry. Reasoning by analogy is frequently suspect anyway, but in your picture, I'm not sure how databases are barn doors. They tend, at least for the kinds of things we're doing, to be much more fundamental to the framing of the barn. Or maybe they're the stuff stashed in the hayloft. I dunno. At any rate, I don't think it's super coherent to argue that all the things must be fixed at once in order to justify improving one of the things.

rpedela 13y ago

Thanks! You answered my question, and it makes sense. There have been times where I have changed underlying technologies because it provided a better foundation for the future even though there was nothing particularly wrong with the existing solution. No worries about the tone or replying to the entire thread instead of just my question. I would also be frustrated with some of the comments in this thread.