3 ms·

> What part of VPC networking is challenging to you? It can be a source of friction for someone that is just trying to accomplish a specific goal, in this case

by sah2ed 7y ago

> What part of VPC networking is challenging to you? It can be a source of friction for someone that is just trying to accomplish a specific goal, in this case it might just be to use AWS to go live. If building a database-backed prototype is your goal, designing a schema is only tangentially related to that goal. Schema design ability can slow you down if you have limited familiarity with SQL and/or the entity relationship model used by RDBMS. NoSQL allows you to reach that specific goal faster than with an RDBMS by lowering the friction of creating database records and it does it by flipping the schema model from schema-on-write to schema-on-read which is a huge time saver. This made NoSQL great for building prototypes, especially for devs with little or no familiarity with SQL, because it eliminates the upfront effort of designing a schema (that is certain to change anyway) before you can begin to store records in the database. In other words, the use of a far more familiar language (JavaScript vs. SQL) and the simpler programming model (document-model vs. entity relationship model) is a big part of why MongoDB was able to earn mind share among devs evaluating NoSQLs databases.

world32 7y ago
I like how you managed to turn a question about VPC networking on AWS into an advertisement for MongoDB!
- sah2ed 7y ago
  LOL :-)! That was never my intention and I have absolutely no connection to them in any way whatsoever. MongoDB just turned out to be an example I felt a lot of HNers could readily relate to considering some of their guffaws as they grew made front-page like this one: https://stackoverflow.com/questions/16833100/why-does-the-mongodb-java-driver-use-a-random-number-generator-in-a-conditional https://stackoverflow.com/questions/16833100/why-does-the-mo...
takeda 7y ago
In both, NoSQL or cloud providers that provide predefined networking it is easy to stay, but it will vote you in the butt once your project becomes bigger. Using RDBMS has a stepper learning curve, but organizing your data right will pay back in bundles. VPC is similar. It's not that hard to configure it, it is actually very trivial compared to setting on prem data center, you can learn it or how someone who knows this. Same with RDBMS, learn it (it is not hard) or hire a DBA. I worked at one of popular car shopping sites, and while they used postgresql they stored car information as a thrift encoded blob. Every car trim was a separate row. A data had to be ETL to a NoSQL database to be extracted, that means to look up all car makes took 7 hours (the ETL happened through AMQP, don't ask) of course it was them stored in dedicated collection, but it was ridiculous. Out of curiosity I wrote code to convert the data from that form to relational database (postgresql) this actually helped to find that the were duplicates. The queries on postgresql database also had lower latency it didn't look like it would have a problem handling the traffic. This data it's also read only for users so it is trivial to scale out. They also had a database to map up and latitude/longitude to a zip code. The data on MongoDB was taking 30GB and they had to use instances with enough RAM to hold all the data otherwise MongoDB would choke. The same exact data when stored in postgresql using right types (PostGIS and ip4r) took a bit over 600MB. They also used Solr for storing inventory data. I'm sure you are shaking your head that you would not do such things, bit of you have NoSQL blinkers on you often will run into problems that aren't problems at all.
chousuke 7y ago
If you're just setting up a prototype environment then yes, sure, set up a VPC and go with the defaults and don't worry overmuch about what you're doing. However, when you start setting up a production environment, the story changes. If you're a developer creating network-facing applications, then I am convinced that at least a basic understanding of computer networking is not optional. I think the attitude that not everyone needs to know these things leads to people mistakenly thinking they can always have them be someone else's problem which furthermore results (needlessly!) in brittle systems. I'm not asking for every developer to become a networking expert, but learning at least the very basics about your dependencies (you depend on networking, after all) is what I think a professional should do. To address your NoSQL example, certainly it can be convenient to prototype with a dynamic schema, but you as a developer should still understand that even if it is dynamic, your data still has a schema and a concrete structure, which will affect your application in various ways. For rapid prototyping it's fine to ignore all this, but again once you start thinking about production, you no longer have the luxury.

by sah2ed 7y ago

world32 7y ago

I like how you managed to turn a question about VPC networking on AWS into an advertisement for MongoDB!

sah2ed 7y ago
LOL :-)! That was never my intention and I have absolutely no connection to them in any way whatsoever. MongoDB just turned out to be an example I felt a lot of HNers could readily relate to considering some of their guffaws as they grew made front-page like this one: https://stackoverflow.com/questions/16833100/why-does-the-mongodb-java-driver-use-a-random-number-generator-in-a-conditional https://stackoverflow.com/questions/16833100/why-does-the-mo...

takeda 7y ago

In both, NoSQL or cloud providers that provide predefined networking it is easy to stay, but it will vote you in the butt once your project becomes bigger. Using RDBMS has a stepper learning curve, but organizing your data right will pay back in bundles. VPC is similar. It's not that hard to configure it, it is actually very trivial compared to setting on prem data center, you can learn it or how someone who knows this. Same with RDBMS, learn it (it is not hard) or hire a DBA. I worked at one of popular car shopping sites, and while they used postgresql they stored car information as a thrift encoded blob. Every car trim was a separate row. A data had to be ETL to a NoSQL database to be extracted, that means to look up all car makes took 7 hours (the ETL happened through AMQP, don't ask) of course it was them stored in dedicated collection, but it was ridiculous. Out of curiosity I wrote code to convert the data from that form to relational database (postgresql) this actually helped to find that the were duplicates. The queries on postgresql database also had lower latency it didn't look like it would have a problem handling the traffic. This data it's also read only for users so it is trivial to scale out. They also had a database to map up and latitude/longitude to a zip code. The data on MongoDB was taking 30GB and they had to use instances with enough RAM to hold all the data otherwise MongoDB would choke. The same exact data when stored in postgresql using right types (PostGIS and ip4r) took a bit over 600MB. They also used Solr for storing inventory data. I'm sure you are shaking your head that you would not do such things, bit of you have NoSQL blinkers on you often will run into problems that aren't problems at all.

chousuke 7y ago

If you're just setting up a prototype environment then yes, sure, set up a VPC and go with the defaults and don't worry overmuch about what you're doing. However, when you start setting up a production environment, the story changes. If you're a developer creating network-facing applications, then I am convinced that at least a basic understanding of computer networking is not optional. I think the attitude that not everyone needs to know these things leads to people mistakenly thinking they can always have them be someone else's problem which furthermore results (needlessly!) in brittle systems. I'm not asking for every developer to become a networking expert, but learning at least the very basics about your dependencies (you depend on networking, after all) is what I think a professional should do. To address your NoSQL example, certainly it can be convenient to prototype with a dynamic schema, but you as a developer should still understand that even if it is dynamic, your data still has a schema and a concrete structure, which will affect your application in various ways. For rapid prototyping it's fine to ignore all this, but again once you start thinking about production, you no longer have the luxury.