3 ms·

> [...] there is no freaking way I can remember all the details of linux, python, bash, regular expressions, flask, elasticsearch, redis, haproxy, postgresql, m

by dozzie 10y ago

> [...] there is no freaking way I can remember all the details of linux, python, bash, regular expressions, flask, elasticsearch, redis, haproxy, postgresql, mongodb, Google Cloud Platform, kubernetes, logstash, kibana, Java... that's a real list from my last project. Sounds like badly written thing (probably not designed at all). And you do know that there are archaic things like documentation and own notes, right?

d23 10y ago
Why does that sound like a badly written thing, you having no knowledge of the specific domain he's referencing? Perhaps it's a web frontend in flask, load balanced by haproxy, uses redis heavily for caching, postgres for structured data, and mongo for analytics (yeah yeah, we can gripe about how bad mongo is). These applications run via kubernetes on GCE, the logs of which are forwarded to logstash, stored in elasticsearch (running on the JVM), and viewed via kibana. > And you do know that there are archaic things like documentation and own notes, right? Yes, hence his mentioning that he needs to Google to reference them.
- markbnj 10y ago
  A nearly perfect description, from someone who has obviously built similar systems :).
- dozzie 10y ago
  > Why does that sound like a badly written thing, you having no knowledge of the specific domain he's referencing? I don't have the knowledge, that's why I didn't claim the system was badly written, I only expressed my impression it was. I know, it's a subtle difference easy to be overlooked, especially if somebody is invested in similar architecture. And it still sounds like a bad idea to tie oneself to specific log transport and log storage, and to a single load balancer at a project time. Use of MongoDB when already using PostgreSQL (or vice versa) doesn't add much soundness, either.
  - CydeWeys 10y ago
    Why would you admit to not having knowledge of the domain in question, and then, right in the same post, continue making speculative disparaging comments about the technology stack in use, which you self-admittedly know little about? Knowing when your opinion is valuable to others, and withholding it when it is not, is a useful skill. > Use of MongoDB when already using PostgreSQL (or vice versa) doesn't add much soundness, either. Come on, they're entirely different kinds of databases. PostgreSQL is relational, MongoDB is not. It's very reasonable to use both kinds together in a larger project, each for the type of data that it is best at. I have worked on multiple software projects that combined some variant of SQL (e.g. MySQL, PostgreSQL, MS SQL) with some variant of NoSQL (e.g. MongoDB, DynamoDB, Datastore). Trying to shoehorn everything into a single schema would have been worse.
markbnj 10y ago
You think that list of technologies is unusual? It's not. And while I keep plenty of my own notes, I am not sure why you think trying to replicate the online documentation that can be quickly found through search tools would be a better strategy.
- dozzie 10y ago
  I'm aware it is not unusual. It is not unusual for software to be badly written, too (being wide-spread is no sign of being a proper way of doing things). For a single project (and OP said it was one project) depending on that many specific tools coming from that many fields is a strong sign that the project does too much. Most of the tools mentioned by OP are from the set to be decided in deployment time instead of design time (log transport, storage, and browser, deployment automation, stuff like that), and they should not be dependencies.
  - markbnj 10y ago
    The concept of a "single project" leaves a lot of wiggle room. In this case the "project" I refer to was a mobile application with all the front and back-end pieces that implies. You could carve the "project" up into sub-projects and view it however you wish, but since I was responsible for much of the back end pieces I experienced it the way I described.

by dozzie 10y ago

d23 10y ago

Why does that sound like a badly written thing, you having no knowledge of the specific domain he's referencing? Perhaps it's a web frontend in flask, load balanced by haproxy, uses redis heavily for caching, postgres for structured data, and mongo for analytics (yeah yeah, we can gripe about how bad mongo is). These applications run via kubernetes on GCE, the logs of which are forwarded to logstash, stored in elasticsearch (running on the JVM), and viewed via kibana. > And you do know that there are archaic things like documentation and own notes, right? Yes, hence his mentioning that he needs to Google to reference them.

markbnj 10y ago
A nearly perfect description, from someone who has obviously built similar systems :).
dozzie 10y ago
> Why does that sound like a badly written thing, you having no knowledge of the specific domain he's referencing? I don't have the knowledge, that's why I didn't claim the system was badly written, I only expressed my impression it was. I know, it's a subtle difference easy to be overlooked, especially if somebody is invested in similar architecture. And it still sounds like a bad idea to tie oneself to specific log transport and log storage, and to a single load balancer at a project time. Use of MongoDB when already using PostgreSQL (or vice versa) doesn't add much soundness, either.
- CydeWeys 10y ago
  Why would you admit to not having knowledge of the domain in question, and then, right in the same post, continue making speculative disparaging comments about the technology stack in use, which you self-admittedly know little about? Knowing when your opinion is valuable to others, and withholding it when it is not, is a useful skill. > Use of MongoDB when already using PostgreSQL (or vice versa) doesn't add much soundness, either. Come on, they're entirely different kinds of databases. PostgreSQL is relational, MongoDB is not. It's very reasonable to use both kinds together in a larger project, each for the type of data that it is best at. I have worked on multiple software projects that combined some variant of SQL (e.g. MySQL, PostgreSQL, MS SQL) with some variant of NoSQL (e.g. MongoDB, DynamoDB, Datastore). Trying to shoehorn everything into a single schema would have been worse.

markbnj 10y ago

You think that list of technologies is unusual? It's not. And while I keep plenty of my own notes, I am not sure why you think trying to replicate the online documentation that can be quickly found through search tools would be a better strategy.

dozzie 10y ago
I'm aware it is not unusual. It is not unusual for software to be badly written, too (being wide-spread is no sign of being a proper way of doing things). For a single project (and OP said it was one project) depending on that many specific tools coming from that many fields is a strong sign that the project does too much. Most of the tools mentioned by OP are from the set to be decided in deployment time instead of design time (log transport, storage, and browser, deployment automation, stuff like that), and they should not be dependencies.
- markbnj 10y ago
  The concept of a "single project" leaves a lot of wiggle room. In this case the "project" I refer to was a mobile application with all the front and back-end pieces that implies. You could carve the "project" up into sub-projects and view it however you wish, but since I was responsible for much of the back end pieces I experienced it the way I described.