11 ms·

Here's to hoping ignoring serverless will work as well as ignoring NoSQL worked for me.

by danharaj 8y ago

Xorlev 8y ago
But don't you want perfect elastic scalability for your blockchain^W distributed ledger project? /s
- baxtr 8y ago
  You forgot the AI part...
  - atonse 8y ago
    How about visualizing the realtime graph with VR? There, I tied it all in.
deleted 8y ago
[deleted]
giancarlostoro 8y ago
Did not Google already make a cloud platform that is cloud agnostic called OpenCloud which is a better name that doesn't target just "Serverless" aka Azure Functions, Amazon Lambdas. The use of "Serverless" is not having to deal with an "IT" guy at all who complains about setting up your app cause you updated the STACK and now it collides with everything else on the same server. Also makes it so you don't necessarily have to use containers.
- toomuchtodo 8y ago
  I think the problem is all of this tooling (Docker, Serverless, NoSQL) has been created to “support developer velocity”, which really just ends up as technical debt. You can’t magic away the need for experience and domain knowledge. Docker doesn’t replace the need to know how VMs work. Containers don’t magically allow you to scale to infinity (Although k8s shows a lot of potential). And you probably should be using PostgreSQL instead of NoSQL unless you’re absolutely sure you’re smart enough to know why PostgreSQL can’t work for your use case. Serverless is great if you want to replace a cron job, the value of the function firing is substantially higher than the cost to run it (“margins are crazy high, optimize to VMs later and ignore the AWS bill for now”), or you’re executing untrusted code in isolation for customers.
  - sbinthree 8y ago
    I am learning this lesson the hard way with dynamically typed languages on the server. If the documentation and database have to be statically typed, you should really use types in the code. So dynamically typed languages on the server is impossible (?).
    - cphoover 8y ago
      No offense but BS. What you have claimed is totally unsubstantiated, and not aligned with my experience working on highly-trafficked eCommerce applications. If you know what your doing you can write elegant, performance tuned, secure and maintainable code in a dynamic language. I've also seen poorly written code written in statically typed languages. It really comes to who is writing the code, what kind of standards they abide to, and their architectural prowess.
      - eropple 8y ago
        > If you know what your doing you can write elegant, performance tuned, secure and maintainable code in a dynamic language. You can! You totally can. But, statistically speaking? You probably won't. Neither will I. And that's why the minimal level of guardrails I'll put up with in 2018 is TypeScript and I'd really rather have better.
        always_good 8y ago
        You're rehashing the old dynamic- vs static-typed debate. But what the upstream comment said was just wrong: that because documentation and database are statically-typed, then the application must be. It doesn't really make sense. See their use of "impossible". For example, your database types or your application types aren't what your API documentation annotates. Your docs annotate your endpoint contracts, not the implementation detail behind them.
        eropple 8y ago
        If you take "must" in the pocket-protectory overly-literal way, yeah, sure. But when you act generously and decent and take it to reference what is tenable and acceptable, things take a different turn. And it is in that light that I approach the topic and the speaker, because generosity and decency are pretty good things.
Bahamut 8y ago
This comment is a little strange - NoSQL is prevalent, especially when dealing with large data sets, or certain problems such as search. That said, I’m not sold on serverless.
y4mi 8y ago
Serverless is at its heart - as I understand it - a dockerized microservice, abstracted away to a degree that the developer no longer has to think about anything but his application code. You'll definitely be able to ignore it and it probably won't be used in smallish companies for ages. It's just an easier way to to get your application to scale than homebuild docker images were.
- chii 8y ago
  > no longer has to think about anything but his application code. i mean, CGI has always existed. This serverless hype is basically rebrand of CGI with some fancy orchestration around autoscaling across boxes (which, tbh, isn't really that much work, and most people don't need the scale required to make this feasible anyway).
  - mmt 8y ago
    > isn't really that much work I suspect that it's this little parenthetical tidbit, and implicit disagreement with it (or differing definitions of "much") that drives the creation of this kind of abstraction. In some situations, I consider the gap to be legitimate, where it may be easy (not that much work) for an expert but difficult for everyone else, and, more importantly, becoming expert is non-trivial, even with training/mentorship from one. In other situations, I consider the gap to be merely one of perception/misestimation, either because it would actually be relatively easy for a non-expert who had actually tried, and/or the needed expertise is shallow enough that it can be quickly taught. I believe autoscaling is (or at least originally was) of the former category and that the availability of tools and abstractions around it has allowed a broad number of non-experts to leverage the wisdom of a much narrower group of expert practitioners. OTOH, I believe running hardware in a datacenter (as opposed to outsourcing it to a VPS or even cloud) is of the latter category. I routinely read comments like "have to hire 5 sysadmins" from non-experts when we experts know that estimate is around 20x too high for a scale of hundreds of servers. Even at higher scale, if hiring is necessary, the hardware-specific skills are easily taught, so junior staff is fine.
- xchaotic 8y ago
  "a degree that the developer no longer has to think about anything but his application code." a developer still has to understand the implications of resource consumption etc. For performance-critical pieces of code, IMO it's better to have direct access to the hardware - I had a recent first hand experience with this debugging NUMA related performance issue.
autotune 8y ago
Serverless has its place, that said I'm not sure how well this company is going to end up doing. Might be a bit biased but the most common serverless applications I've seen are all about integrating cloud services with each other, which is done on their own platform.
cphoover 8y ago
NoSQL has been and continues to be hugely influential. All major cloud players provide document/object based storage, as well as other NoSQL Solutions. The term "NoSQL" was dumb and overhyped... But I think it's really about using the correct storage solution for the job. Non relational data should be stored in a non rdbms. Key-Value stores like Redis are immensely useful as caching layers (but they offer so many more features). Graph databases can be used for data with complex relationships that are not easily modeled. They are also good for seeking strong correlations between related items. (think person A. called person B. called person C. (palantir type searches).). Searches can be done way more effectively in a specialized index, like an inverted index used by lucene/elasticsearch, which also supports things like stemming, synonyms, and numerous other features. These are all "NoSQL" NoSQL is not just mongodb (which isn't nearly as bad as people make it out to be btw). Even traditional RDBMS are seeing an influx of NOSQLesque features. Like JSON types and operations in postgres. The reason "NoSQL" dbs got popular are because in my experience monolithic large relational databases are hard to scale, and manage once they become too complex. When you have one large database with tons of interdependencies, it makes migrating data, and making schema changes much harder. This in my opinion is the biggest issue (moreso than performance problems associated with doing joins to the n-th degree., which is also an issue.) It also makes separating concerns of the application more difficult when one SQL connection can query/join all entities. In theory better application design would have separate upstream data services fetch the resources they are responsible for. That data can be stored in a RDBMS or NOSQL, but NOSQL forces your hand in that direction. As it goes for serverless, this just seems like a natural progression from containerization, I'm interested to see where the space goes. Personally I think it's foolish to put your head in the sand when the industry is changing, or learning new concepts.
- danharaj 8y ago
  > The reason "NoSQL" dbs got popular are because in my experience Monolithic large relational databases are hard to scale. I've met a lot of people whomst thought they had to scale that big. Very few handled anything that couldn't run off a beefy postgres installation. The purpose of a system is what it does. People don't use nosql to scale because they don't need to scale, so what does it do? People use nosql to not write schemas. That's what it's for, for the majority of users. If I need a key value store, I use a key value store. There's no flashy paradigm there. If I need to put a container up on the interwebs, I do it. What's serverless? Nosql is an "idea", "paradigm", "revolution", or at least the branding of one. Just the same, serverless. I will continue to ignore nosql and serverless. The industry sure does change, but do you know how much of that is moving in a real direction and how much is a merry-go-round? Let's brand it "Carousel" and raise 10 million. And in 20 years we can talk about serverless being the new hotness, again.
exabrial 8y ago
This is anecdotal, and I've read cases of the opposite, so I know there are downvotes incoming. I've yet to work on a system where NoSQL was I was like "thank goodness we didn't use a structured database!". Instead, every time it's been the HIPPO trying to defend the decision while everyone else just deals with it. NoSQL seems to be taking a giant loan... You're going to need to organize and parse your data at some point (or why would you keep the data?). Putting that decision into the future just makes it harder on everyone.
- zaarn 8y ago
  Schemaless definitely has a few applications, usually systems related to tagging. Luckily you can easily integrate schemaless into your Postgres database with no performance downside all thanks to the magic of JSONB or FDW, depending on which way you swing. The very few pure schemaless databases that continue to exist and where I'm convinced they will continue to exist for a long while are those that specialize a lot (ie, Redis, Elasticsearch, a lot of the Timeseries databases).

Here's to hoping ignoring serverless will work as well as ignoring NoSQL worked for me.

Xorlev 8y ago

But don't you want perfect elastic scalability for your blockchain^W distributed ledger project? /s

baxtr 8y ago
You forgot the AI part...
- atonse 8y ago
  How about visualizing the realtime graph with VR? There, I tied it all in.

deleted 8y ago

[deleted]

giancarlostoro 8y ago

Did not Google already make a cloud platform that is cloud agnostic called OpenCloud which is a better name that doesn't target just "Serverless" aka Azure Functions, Amazon Lambdas. The use of "Serverless" is not having to deal with an "IT" guy at all who complains about setting up your app cause you updated the STACK and now it collides with everything else on the same server. Also makes it so you don't necessarily have to use containers.

toomuchtodo 8y ago
I think the problem is all of this tooling (Docker, Serverless, NoSQL) has been created to “support developer velocity”, which really just ends up as technical debt. You can’t magic away the need for experience and domain knowledge. Docker doesn’t replace the need to know how VMs work. Containers don’t magically allow you to scale to infinity (Although k8s shows a lot of potential). And you probably should be using PostgreSQL instead of NoSQL unless you’re absolutely sure you’re smart enough to know why PostgreSQL can’t work for your use case. Serverless is great if you want to replace a cron job, the value of the function firing is substantially higher than the cost to run it (“margins are crazy high, optimize to VMs later and ignore the AWS bill for now”), or you’re executing untrusted code in isolation for customers.
- sbinthree 8y ago
  I am learning this lesson the hard way with dynamically typed languages on the server. If the documentation and database have to be statically typed, you should really use types in the code. So dynamically typed languages on the server is impossible (?).
  - cphoover 8y ago
    No offense but BS. What you have claimed is totally unsubstantiated, and not aligned with my experience working on highly-trafficked eCommerce applications. If you know what your doing you can write elegant, performance tuned, secure and maintainable code in a dynamic language. I've also seen poorly written code written in statically typed languages. It really comes to who is writing the code, what kind of standards they abide to, and their architectural prowess.
    - eropple 8y ago
      > If you know what your doing you can write elegant, performance tuned, secure and maintainable code in a dynamic language. You can! You totally can. But, statistically speaking? You probably won't. Neither will I. And that's why the minimal level of guardrails I'll put up with in 2018 is TypeScript and I'd really rather have better.
      - always_good 8y ago
        You're rehashing the old dynamic- vs static-typed debate. But what the upstream comment said was just wrong: that because documentation and database are statically-typed, then the application must be. It doesn't really make sense. See their use of "impossible". For example, your database types or your application types aren't what your API documentation annotates. Your docs annotate your endpoint contracts, not the implementation detail behind them.
        eropple 8y ago
        If you take "must" in the pocket-protectory overly-literal way, yeah, sure. But when you act generously and decent and take it to reference what is tenable and acceptable, things take a different turn. And it is in that light that I approach the topic and the speaker, because generosity and decency are pretty good things.

Bahamut 8y ago

This comment is a little strange - NoSQL is prevalent, especially when dealing with large data sets, or certain problems such as search. That said, I’m not sold on serverless.

y4mi 8y ago

Serverless is at its heart - as I understand it - a dockerized microservice, abstracted away to a degree that the developer no longer has to think about anything but his application code. You'll definitely be able to ignore it and it probably won't be used in smallish companies for ages. It's just an easier way to to get your application to scale than homebuild docker images were.

chii 8y ago
> no longer has to think about anything but his application code. i mean, CGI has always existed. This serverless hype is basically rebrand of CGI with some fancy orchestration around autoscaling across boxes (which, tbh, isn't really that much work, and most people don't need the scale required to make this feasible anyway).
- mmt 8y ago
  > isn't really that much work I suspect that it's this little parenthetical tidbit, and implicit disagreement with it (or differing definitions of "much") that drives the creation of this kind of abstraction. In some situations, I consider the gap to be legitimate, where it may be easy (not that much work) for an expert but difficult for everyone else, and, more importantly, becoming expert is non-trivial, even with training/mentorship from one. In other situations, I consider the gap to be merely one of perception/misestimation, either because it would actually be relatively easy for a non-expert who had actually tried, and/or the needed expertise is shallow enough that it can be quickly taught. I believe autoscaling is (or at least originally was) of the former category and that the availability of tools and abstractions around it has allowed a broad number of non-experts to leverage the wisdom of a much narrower group of expert practitioners. OTOH, I believe running hardware in a datacenter (as opposed to outsourcing it to a VPS or even cloud) is of the latter category. I routinely read comments like "have to hire 5 sysadmins" from non-experts when we experts know that estimate is around 20x too high for a scale of hundreds of servers. Even at higher scale, if hiring is necessary, the hardware-specific skills are easily taught, so junior staff is fine.
xchaotic 8y ago
"a degree that the developer no longer has to think about anything but his application code." a developer still has to understand the implications of resource consumption etc. For performance-critical pieces of code, IMO it's better to have direct access to the hardware - I had a recent first hand experience with this debugging NUMA related performance issue.

autotune 8y ago

Serverless has its place, that said I'm not sure how well this company is going to end up doing. Might be a bit biased but the most common serverless applications I've seen are all about integrating cloud services with each other, which is done on their own platform.

cphoover 8y ago

NoSQL has been and continues to be hugely influential. All major cloud players provide document/object based storage, as well as other NoSQL Solutions. The term "NoSQL" was dumb and overhyped... But I think it's really about using the correct storage solution for the job. Non relational data should be stored in a non rdbms. Key-Value stores like Redis are immensely useful as caching layers (but they offer so many more features). Graph databases can be used for data with complex relationships that are not easily modeled. They are also good for seeking strong correlations between related items. (think person A. called person B. called person C. (palantir type searches).). Searches can be done way more effectively in a specialized index, like an inverted index used by lucene/elasticsearch, which also supports things like stemming, synonyms, and numerous other features. These are all "NoSQL" NoSQL is not just mongodb (which isn't nearly as bad as people make it out to be btw). Even traditional RDBMS are seeing an influx of NOSQLesque features. Like JSON types and operations in postgres. The reason "NoSQL" dbs got popular are because in my experience monolithic large relational databases are hard to scale, and manage once they become too complex. When you have one large database with tons of interdependencies, it makes migrating data, and making schema changes much harder. This in my opinion is the biggest issue (moreso than performance problems associated with doing joins to the n-th degree., which is also an issue.) It also makes separating concerns of the application more difficult when one SQL connection can query/join all entities. In theory better application design would have separate upstream data services fetch the resources they are responsible for. That data can be stored in a RDBMS or NOSQL, but NOSQL forces your hand in that direction. As it goes for serverless, this just seems like a natural progression from containerization, I'm interested to see where the space goes. Personally I think it's foolish to put your head in the sand when the industry is changing, or learning new concepts.

danharaj 8y ago
> The reason "NoSQL" dbs got popular are because in my experience Monolithic large relational databases are hard to scale. I've met a lot of people whomst thought they had to scale that big. Very few handled anything that couldn't run off a beefy postgres installation. The purpose of a system is what it does. People don't use nosql to scale because they don't need to scale, so what does it do? People use nosql to not write schemas. That's what it's for, for the majority of users. If I need a key value store, I use a key value store. There's no flashy paradigm there. If I need to put a container up on the interwebs, I do it. What's serverless? Nosql is an "idea", "paradigm", "revolution", or at least the branding of one. Just the same, serverless. I will continue to ignore nosql and serverless. The industry sure does change, but do you know how much of that is moving in a real direction and how much is a merry-go-round? Let's brand it "Carousel" and raise 10 million. And in 20 years we can talk about serverless being the new hotness, again.

exabrial 8y ago

This is anecdotal, and I've read cases of the opposite, so I know there are downvotes incoming. I've yet to work on a system where NoSQL was I was like "thank goodness we didn't use a structured database!". Instead, every time it's been the HIPPO trying to defend the decision while everyone else just deals with it. NoSQL seems to be taking a giant loan... You're going to need to organize and parse your data at some point (or why would you keep the data?). Putting that decision into the future just makes it harder on everyone.

zaarn 8y ago
Schemaless definitely has a few applications, usually systems related to tagging. Luckily you can easily integrate schemaless into your Postgres database with no performance downside all thanks to the magic of JSONB or FDW, depending on which way you swing. The very few pure schemaless databases that continue to exist and where I'm convinced they will continue to exist for a long while are those that specialize a lot (ie, Redis, Elasticsearch, a lot of the Timeseries databases).