3 ms·

> Most app servers shouldn’t be “caching state locally” anyway. Once you do that it makes scaling horizontally harder - you have to Implement sticky sessions.

> Most app servers shouldn’t be “caching state locally” anyway. Once you do that it makes scaling horizontally harder - you have to Implement sticky sessions. Why not? In-memory caches are fast and fairly uncomplicated to write.

LASR 8y ago
Yes, but you want applications to be entirely stateless. The only caching you should be doing is for static data that never changes. Alternatively, a lot of interpreted/JIT'ed languages, for example will have some form of a module cache. These are perfect, so your request latency is considerably lower than the first request latency.
dlhavema 8y ago
The sticky session thing is about a clients session. You store the identifier on one machines and if the same client hits another machine it knows nothing of that session I'd. This is one example but in memory only works well in this setup for client agnostic data. Type table lookups, etc
adrianmonk 8y ago
Because when you have horizontally scaled to 1000 servers, when the next request comes in, it's very likely to hit a different server from last time. So your cache hit rate is going to be horrendous. Take the cache hit rate you'd have for a single server, and divide it by 1000, the odds of getting the same server again. So if your cache hit rate was 80%, now it's 0.08%. Of course you could arrange for a user's requests to hit the same server as last time, but that's sticky sessions.
- IgorPartola 8y ago
  Wait so how is Lambda making this worse? In this case it’s pretty clear you want a caching server instead of an in memory cache. I don’t see why the 1000 serverless functions is worse than 1000 long running processes.
  - scarface74 8y ago
    That’s kind of the point. People act like lambda is some weird thing that completely changes your architecture. Even when I’m hosting things on VMs, I still consider them as disposable. I don’t store state on them, logs are all sent to a central store like Cloudwatch or ElasticSearch, if a health check fails, autoscaling just kills the instance and initiates another one based on a custom prebaked image and a startup script, etc. Long lived data is stored in a database and cached with either Memcache or Redis. We had an old school Devops guy that wanted me to list the names and ip addresses of some servers that I was using. I told him that I would have no idea and even if I did give him a list, it would change tomorrow. I have some windows servers that in non prod environments that host some Windows services based on queues. The servers are automatically terminated when nothing is in the queue and it may spin up a dozen when needed. My servers don’t have names - they have tags. How is this any different than using a Lambda?
    - tatersolid 8y ago
      > Hiw is this any different from Lamba? You had to write all that orchestration code yourself. That said, I completely agree with you, and utilize the same arch at dayjob. There are “app tier” cattle, and there are a few special pets (databases) that are backed up, loved, hand-held, etc. because database failover is still painful for users for seconds-to-minutes. When you’re serving thousands of requests per second, that’s a lot of angry users. RDS, Azure SQL, Cassandra, Aurora, whatever still cause measurable pain for the end user during node failure.
      - scarface74 8y ago
        You had to write all that orchestration code yourself. With the Windows VMs I had to: - integrate TopShelf to make it a Windows Service - write the code to poll the queue (no big deal but still) - create two Cloudwatch alarms to monitor the queue to tell the autoscaling group when to scale in and out - define the launch configuration for the autoscsling group - define the autoscaling group - add health check functionality to the Windows service to ensure that the Windows service was running that could be integrated with the sutoscaling group (well I didn’t have too but to be complete) Alternatively with lambda all I had to do was - add a method that took in two parameters - the SQS event and the lambda context. - add the SQS event to trigger the lambda. Of course both stacks were created with Cloudformstion. One I had to do myself, the other I just basically set everything up in the web console, exported the CloudFormation template and made some minor changes.
hinkley 8y ago
You always have to apply pressure to keep the system decoupled or you get a Ball of Mud. If you switch to micro services and serverless and then have a high fanout and caches everywhere just to make the numbers work, you haven’t gotten rid of the Ball of Mud, you’ve just hidden it in plain sight. Serverless and embarrassingly parallel problems look like pure functions punctuated by one or two state changes. To get that you should probably be passing all the data in, instead of communicating out of hand with a cache to keep your fanout from looking awful (again, hiding the problem).
- dahauns 8y ago
  But who is doing the "passing-in" then, and where does the data come from? This still sounds like hiding the problem...or more specifically, making it someone else's problem.
  - scarface74 8y ago
    Usually the data comes from outside the system - an api integration, user interactions with a website, an ETL process involving a file, IOT devices, etc.
  - hinkley 8y ago
    Who does the passing in in development mode? Where does the data come from? You have this problem anyway, and it blocks you from launching new code. These out of band communication mechanisms always lead to problems with developer ergonomics and repeatability. It's essentially a larger version of the problem of testing pure functions versus heavy usage of mocks. Passing the data in directly makes testing the callee very simple.

LASR 8y ago

Yes, but you want applications to be entirely stateless. The only caching you should be doing is for static data that never changes. Alternatively, a lot of interpreted/JIT'ed languages, for example will have some form of a module cache. These are perfect, so your request latency is considerably lower than the first request latency.

dlhavema 8y ago

The sticky session thing is about a clients session. You store the identifier on one machines and if the same client hits another machine it knows nothing of that session I'd. This is one example but in memory only works well in this setup for client agnostic data. Type table lookups, etc

adrianmonk 8y ago

Because when you have horizontally scaled to 1000 servers, when the next request comes in, it's very likely to hit a different server from last time. So your cache hit rate is going to be horrendous. Take the cache hit rate you'd have for a single server, and divide it by 1000, the odds of getting the same server again. So if your cache hit rate was 80%, now it's 0.08%. Of course you could arrange for a user's requests to hit the same server as last time, but that's sticky sessions.

IgorPartola 8y ago
Wait so how is Lambda making this worse? In this case it’s pretty clear you want a caching server instead of an in memory cache. I don’t see why the 1000 serverless functions is worse than 1000 long running processes.
- scarface74 8y ago
  That’s kind of the point. People act like lambda is some weird thing that completely changes your architecture. Even when I’m hosting things on VMs, I still consider them as disposable. I don’t store state on them, logs are all sent to a central store like Cloudwatch or ElasticSearch, if a health check fails, autoscaling just kills the instance and initiates another one based on a custom prebaked image and a startup script, etc. Long lived data is stored in a database and cached with either Memcache or Redis. We had an old school Devops guy that wanted me to list the names and ip addresses of some servers that I was using. I told him that I would have no idea and even if I did give him a list, it would change tomorrow. I have some windows servers that in non prod environments that host some Windows services based on queues. The servers are automatically terminated when nothing is in the queue and it may spin up a dozen when needed. My servers don’t have names - they have tags. How is this any different than using a Lambda?
  - tatersolid 8y ago
    > Hiw is this any different from Lamba? You had to write all that orchestration code yourself. That said, I completely agree with you, and utilize the same arch at dayjob. There are “app tier” cattle, and there are a few special pets (databases) that are backed up, loved, hand-held, etc. because database failover is still painful for users for seconds-to-minutes. When you’re serving thousands of requests per second, that’s a lot of angry users. RDS, Azure SQL, Cassandra, Aurora, whatever still cause measurable pain for the end user during node failure.
    - scarface74 8y ago
      You had to write all that orchestration code yourself. With the Windows VMs I had to: - integrate TopShelf to make it a Windows Service - write the code to poll the queue (no big deal but still) - create two Cloudwatch alarms to monitor the queue to tell the autoscaling group when to scale in and out - define the launch configuration for the autoscsling group - define the autoscaling group - add health check functionality to the Windows service to ensure that the Windows service was running that could be integrated with the sutoscaling group (well I didn’t have too but to be complete) Alternatively with lambda all I had to do was - add a method that took in two parameters - the SQS event and the lambda context. - add the SQS event to trigger the lambda. Of course both stacks were created with Cloudformstion. One I had to do myself, the other I just basically set everything up in the web console, exported the CloudFormation template and made some minor changes.

hinkley 8y ago

You always have to apply pressure to keep the system decoupled or you get a Ball of Mud. If you switch to micro services and serverless and then have a high fanout and caches everywhere just to make the numbers work, you haven’t gotten rid of the Ball of Mud, you’ve just hidden it in plain sight. Serverless and embarrassingly parallel problems look like pure functions punctuated by one or two state changes. To get that you should probably be passing all the data in, instead of communicating out of hand with a cache to keep your fanout from looking awful (again, hiding the problem).

dahauns 8y ago
But who is doing the "passing-in" then, and where does the data come from? This still sounds like hiding the problem...or more specifically, making it someone else's problem.
- scarface74 8y ago
  Usually the data comes from outside the system - an api integration, user interactions with a website, an ETL process involving a file, IOT devices, etc.
- hinkley 8y ago
  Who does the passing in in development mode? Where does the data come from? You have this problem anyway, and it blocks you from launching new code. These out of band communication mechanisms always lead to problems with developer ergonomics and repeatability. It's essentially a larger version of the problem of testing pure functions versus heavy usage of mocks. Passing the data in directly makes testing the callee very simple.