11 ms·

Building a Bank with Kubernetes [slides]

375 points by renaudg 10y ago

coleca 10y ago
Here is the video from the presentation. Love to see real world use cases for K8S. https://skillsmatter.com/skillscasts/9146-building-a-microservices-with-kubernetes https://skillsmatter.com/skillscasts/9146-building-a-microse...
0xmohit 10y ago
Is there a video available elsewhere? Last I checked, it wasn't possible to download those from SkillsMatter.
- harshreality 10y ago
  % youtube-dl 'https://skillsmatter.com/skillscasts/9146-building-a-microservices-with-kubernetes' [generic] 9146-building-a-microservices-with-kubernetes: Requesting header WARNING: Falling back on generic information extractor. [generic] 9146-building-a-microservices-with-kubernetes: Downloading webpage [generic] 9146-building-a-microservices-with-kubernetes: Extracting information [vimeo] 188042022: Downloading webpage [vimeo] 188042022: Extracting information [vimeo] 188042022: Downloading JSON metadata [vimeo] 188042022: Downloading m3u8 information [download] Destination: Meetups_Oct19_19-43-44-188042022.mp4 [download] 7.9% of 152.75MiB at 82.49KiB/s ETA 29:05
  - runeks 10y ago
    Cool! Although, seems like "youtube-dl" may not be the right name for that app anymore.
    - timothyb89 10y ago
      youtube-dl is still plenty active, its last commit was just a few hours ago: https://github.com/rg3/youtube-dl/ https://github.com/rg3/youtube-dl/
      - karakal 10y ago
        How come this project doesn't get a takedown request?
        timothyb89 10y ago
        No idea, but I'm not going to complain. It's incredibly useful.
avitzurel 10y ago
It can be called: "How we built 'x' with Kubernetes". Really the only thing that is specific to a bank (as I see it) is that they use separate linkerd in order to do the secure stuff. Which is essentially what banks have been doing for ages. I commented before on how Kube has just taken over and beat mesos/marathon stack. This talk is an example to that. You can see how many people jumped on the Kube stack and running successful deployments on it.
- graffitici 10y ago
  But what about "Big Data" workloads? Running Spark or Cassandra clusters say? My understanding is that having a custom scheduler makes mesos more attractive for those tasks? Has your experience been different?
  - avitzurel 10y ago
    Personally I always ran big data on Amazon stack so never really needed that ability
  - agibsonccc 10y ago
    On prem aka "non aws" it does tend to matter yes.
- agibsonccc 10y ago
  Disclosure: I mainly use DC/OS mesos myself. I've evaluated k8s for our use case and didn't find it was quite what we were looking for. Our customers and stack are mainly JVM based. We do on prem deployments not cloud where GCE is already doing pretty well. We also mainly work with the microsoft side of things (azure,enterprise stuff) Not convinced of this. Direct mesos and yarn integration with spark (not to mention a lot of the software already being built on top) is going to keep mesos/marathon relevant for a long time. It's definitely better for big data workloads. I think a lot of startups will jump to this for sure. Many startups don't actually have big data stacks and prefer to use go based stuff (mainly because it's simpler). In that case k8s makes sense for that. A lot of these companies will likely prefer DC/OS and mesos/marathon because they have in house zookeeper expertise already. ZK is a dependency for much of the big data ecosystem as well a kafka and mesos. The synergy is a lot better. That being said: Many here will disagree. I definitely think k8s is winning developer mindshare overall, but I don't think it will have 100% of the market.
hanief 10y ago
But will it run FORTRAN or COBOL? ;)
- nickpsecurity 10y ago
  And does it support decimal math or will it have floating point errors instead? ;)
obeattie 10y ago
Author of the talk here. Happy to answer any questions anyone has. This post also contains more info on how we build our systems: https://monzo.com/blog/2016/09/19/building-a-modern-bank-backend/ https://monzo.com/blog/2016/09/19/building-a-modern-bank-bac... And as an aside, I'll also be giving a longer talk at Kubecon going into more detail on some of this stuff :-)
- olalonde 10y ago
  I hadn't heard of linkerd before. How does it compare to etcd or consul and why did you chose it?
  - lobster_johnson 10y ago
    Linkerd is not very similar to Etcd or Consul. It's a proxy that sits between your microservices/apps to act as a middleman for RPC calls. Being a proxy, it can handle things like load balancing, throttling and failure handling, meaning you don't have to build that into each app. When used together with Kubernetes, Linkerd will use Kubernetes for discovery, so you don't need Etcd or Consul directly (K8s itself relies on Etcd, though).
- KirinDave 10y ago
  Not that I don't appreciate the subject matter, I wave the flag for K8S all the time and I've shipped bank product presentation stuff on it... But... The fundamental challenges of building a bank are almost entirely orthogonal to things like distributed system uptime and resiliency (unless, I suppose, you could lose consistency during the types service loss Kubernetes makes easy to ameliorate). Evidence for this abounds: nearly every major bank out there is at least 10 years behind the tech we're discussing here. Sure, banks are getting savvy to modern techniques for their presentation and API layers (my employer, Capital One, for example). But the actual challenges are data consistency and liveliness and audit-ability are preserved. I'm really curious if you're using a novel technique to achieve this that Docker and self-managed micro-app swarms can deliver on better. Because what really limits most financial institutions from embracing a lot more modern tech is their core systems of record AND the acceptance of said systems by their governing agencies. So when we talk about basic datacenter ops, that's all great. But I don't think they're the things that make K8S great for a bank. I think most people in a position to evaluate a financial institution roadmap would be unmoved by this deck. Now, if you talk about other things we both know K8S is great for... Things like discovering the genesis of a database action by preserving it throughout the chain of responders, or being able to rapidly respond to site exploits with rolling restarts and a built in mechanism for feature flagging, or having a really great way to offer data scientists the environments they need without risk for data theft, or being able to use traditional CI/CD methodologies but end up with a single deployable unit that is amenable to both automated an manual review and mechanical deployment in spite of the tooling used within. Not that I think selling K8S is your job. But I thought I'd mention the perspective of someone doing inf modernization and product work at a major bank. And of course, as always: the opinions above are my own and not those of my employer or co-workers.
hummingurban 10y ago
can someone explain the business pitch for using kubernetes at all? I'm confused how it differs from docker.
trastentrasten 10y ago
What you using for persistence of user data?
isostatic 10y ago
Do you completely rely on AWS? I.e. if amazon goes bust your company also dies? Or are you just using AWS as a provider of VMs, and could move to rackspace or linode.
- wstrange 10y ago
  Kubernetes provides a great deal of insulation for this exact problem. It runs just fine on all the major clouds, bare metal, etc.
- mandudebruh 10y ago
  Kubernetes has tutorials for various other cloud providers http://kubernetes.io/docs/getting-started-guides#turn-key-cloud-solutions http://kubernetes.io/docs/getting-started-guides#turn-key-cl...
  - elcct 10y ago
    some of those documents are quite outdated
deleted 10y ago
[deleted]
akramhussein 10y ago
As an aside, as a Monzo beta tester I can't say enough good things. Currently travelling around Asia and the card works almost everywhere and I pay 0 fees and get the exact Master Card rate. Twice now I've landed with 0 cash in the airport and been fine. I've used the card in over 10 countries outside UK with <1% issues, and often this is better than my Barclays or Amex. This is just a small benefit but when you put the whole product/experience together from the app + in-app customer service + the card + etc, it just works so well and really comes into a category of it's own. A lot of people say "yeah buy my bank does X too" and while true, they way I look at this is Monzo is like the iPod - other MP3 players had same functionality but this one just works and works damn well.
- ed_blackburn 10y ago
  Simply using it as a bog standard card in the UK it is head and shoulders above other high street banks. Kudos for having the brass balls to actually execute a conversation that has probably happened a million times in every City pub.
redwood 10y ago
Anyone here using k8s in prod? If so, what's your use case? What parts of the stack run inside k8s? Even state? What kind of org do you work for? Mission critical?
kokey 10y ago
Well, it's not a bank, yet. Their banking license restrictions could be lifted next year. That said if they are successful it will probably go a long way towards disrupting the customer experience of retail banking. The banks will be able to compete with this, but having something like this to model their improved experience on is good for them and good for customers in general. I am, however, a bit cynical about this. I think if Monzo takes off they will hit a hurdle or event along the way that can sink the company. Most fintech companies are overly vulnerable to making the same mistakes as most banks made many decades ago. In the end it will probably mean they will get salvaged by being bought by a bank. Then the customer experience won't improve so fast any more, while the back end and processes are being made robust to allow it to remain in business long term. The culture in the company will also change to be somewhat more, uhm, traditional and boring. The result will still be that the customer experience has been pushed forward in general and that is a good thing.
- elcct 10y ago
  I was trying to find an analogy, and I think this company is doing something like making TV guide more accessible and not having in mind that in a few years TV will be obsolete. Probably this will be a good business for the next decade, but it is not doing anything really disruptive.

375 points by renaudg 10y ago

Here is the video from the presentation. Love to see real world use cases for K8S. https://skillsmatter.com/skillscasts/9146-building-a-microservices-with-kubernetes https://skillsmatter.com/skillscasts/9146-building-a-microse...

0xmohit 10y ago

Is there a video available elsewhere? Last I checked, it wasn't possible to download those from SkillsMatter.

harshreality 10y ago
% youtube-dl 'https://skillsmatter.com/skillscasts/9146-building-a-microservices-with-kubernetes' [generic] 9146-building-a-microservices-with-kubernetes: Requesting header WARNING: Falling back on generic information extractor. [generic] 9146-building-a-microservices-with-kubernetes: Downloading webpage [generic] 9146-building-a-microservices-with-kubernetes: Extracting information [vimeo] 188042022: Downloading webpage [vimeo] 188042022: Extracting information [vimeo] 188042022: Downloading JSON metadata [vimeo] 188042022: Downloading m3u8 information [download] Destination: Meetups_Oct19_19-43-44-188042022.mp4 [download] 7.9% of 152.75MiB at 82.49KiB/s ETA 29:05
- runeks 10y ago
  Cool! Although, seems like "youtube-dl" may not be the right name for that app anymore.
  - timothyb89 10y ago
    youtube-dl is still plenty active, its last commit was just a few hours ago: https://github.com/rg3/youtube-dl/ https://github.com/rg3/youtube-dl/
    - karakal 10y ago
      How come this project doesn't get a takedown request?
      - timothyb89 10y ago
        No idea, but I'm not going to complain. It's incredibly useful.

avitzurel 10y ago

It can be called: "How we built 'x' with Kubernetes". Really the only thing that is specific to a bank (as I see it) is that they use separate linkerd in order to do the secure stuff. Which is essentially what banks have been doing for ages. I commented before on how Kube has just taken over and beat mesos/marathon stack. This talk is an example to that. You can see how many people jumped on the Kube stack and running successful deployments on it.

graffitici 10y ago
But what about "Big Data" workloads? Running Spark or Cassandra clusters say? My understanding is that having a custom scheduler makes mesos more attractive for those tasks? Has your experience been different?
- avitzurel 10y ago
  Personally I always ran big data on Amazon stack so never really needed that ability
- agibsonccc 10y ago
  On prem aka "non aws" it does tend to matter yes.
agibsonccc 10y ago
Disclosure: I mainly use DC/OS mesos myself. I've evaluated k8s for our use case and didn't find it was quite what we were looking for. Our customers and stack are mainly JVM based. We do on prem deployments not cloud where GCE is already doing pretty well. We also mainly work with the microsoft side of things (azure,enterprise stuff) Not convinced of this. Direct mesos and yarn integration with spark (not to mention a lot of the software already being built on top) is going to keep mesos/marathon relevant for a long time. It's definitely better for big data workloads. I think a lot of startups will jump to this for sure. Many startups don't actually have big data stacks and prefer to use go based stuff (mainly because it's simpler). In that case k8s makes sense for that. A lot of these companies will likely prefer DC/OS and mesos/marathon because they have in house zookeeper expertise already. ZK is a dependency for much of the big data ecosystem as well a kafka and mesos. The synergy is a lot better. That being said: Many here will disagree. I definitely think k8s is winning developer mindshare overall, but I don't think it will have 100% of the market.

hanief 10y ago

But will it run FORTRAN or COBOL? ;)

nickpsecurity 10y ago
And does it support decimal math or will it have floating point errors instead? ;)

obeattie 10y ago

Author of the talk here. Happy to answer any questions anyone has. This post also contains more info on how we build our systems: https://monzo.com/blog/2016/09/19/building-a-modern-bank-backend/ https://monzo.com/blog/2016/09/19/building-a-modern-bank-bac... And as an aside, I'll also be giving a longer talk at Kubecon going into more detail on some of this stuff :-)

olalonde 10y ago
I hadn't heard of linkerd before. How does it compare to etcd or consul and why did you chose it?
- lobster_johnson 10y ago
  Linkerd is not very similar to Etcd or Consul. It's a proxy that sits between your microservices/apps to act as a middleman for RPC calls. Being a proxy, it can handle things like load balancing, throttling and failure handling, meaning you don't have to build that into each app. When used together with Kubernetes, Linkerd will use Kubernetes for discovery, so you don't need Etcd or Consul directly (K8s itself relies on Etcd, though).
KirinDave 10y ago
Not that I don't appreciate the subject matter, I wave the flag for K8S all the time and I've shipped bank product presentation stuff on it... But... The fundamental challenges of building a bank are almost entirely orthogonal to things like distributed system uptime and resiliency (unless, I suppose, you could lose consistency during the types service loss Kubernetes makes easy to ameliorate). Evidence for this abounds: nearly every major bank out there is at least 10 years behind the tech we're discussing here. Sure, banks are getting savvy to modern techniques for their presentation and API layers (my employer, Capital One, for example). But the actual challenges are data consistency and liveliness and audit-ability are preserved. I'm really curious if you're using a novel technique to achieve this that Docker and self-managed micro-app swarms can deliver on better. Because what really limits most financial institutions from embracing a lot more modern tech is their core systems of record AND the acceptance of said systems by their governing agencies. So when we talk about basic datacenter ops, that's all great. But I don't think they're the things that make K8S great for a bank. I think most people in a position to evaluate a financial institution roadmap would be unmoved by this deck. Now, if you talk about other things we both know K8S is great for... Things like discovering the genesis of a database action by preserving it throughout the chain of responders, or being able to rapidly respond to site exploits with rolling restarts and a built in mechanism for feature flagging, or having a really great way to offer data scientists the environments they need without risk for data theft, or being able to use traditional CI/CD methodologies but end up with a single deployable unit that is amenable to both automated an manual review and mechanical deployment in spite of the tooling used within. Not that I think selling K8S is your job. But I thought I'd mention the perspective of someone doing inf modernization and product work at a major bank. And of course, as always: the opinions above are my own and not those of my employer or co-workers.

hummingurban 10y ago

can someone explain the business pitch for using kubernetes at all? I'm confused how it differs from docker.

trastentrasten 10y ago

What you using for persistence of user data?

isostatic 10y ago

Do you completely rely on AWS? I.e. if amazon goes bust your company also dies? Or are you just using AWS as a provider of VMs, and could move to rackspace or linode.

wstrange 10y ago
Kubernetes provides a great deal of insulation for this exact problem. It runs just fine on all the major clouds, bare metal, etc.
mandudebruh 10y ago
Kubernetes has tutorials for various other cloud providers http://kubernetes.io/docs/getting-started-guides#turn-key-cloud-solutions http://kubernetes.io/docs/getting-started-guides#turn-key-cl...
- elcct 10y ago
  some of those documents are quite outdated

deleted 10y ago

[deleted]

akramhussein 10y ago

As an aside, as a Monzo beta tester I can't say enough good things. Currently travelling around Asia and the card works almost everywhere and I pay 0 fees and get the exact Master Card rate. Twice now I've landed with 0 cash in the airport and been fine. I've used the card in over 10 countries outside UK with <1% issues, and often this is better than my Barclays or Amex. This is just a small benefit but when you put the whole product/experience together from the app + in-app customer service + the card + etc, it just works so well and really comes into a category of it's own. A lot of people say "yeah buy my bank does X too" and while true, they way I look at this is Monzo is like the iPod - other MP3 players had same functionality but this one just works and works damn well.

ed_blackburn 10y ago
Simply using it as a bog standard card in the UK it is head and shoulders above other high street banks. Kudos for having the brass balls to actually execute a conversation that has probably happened a million times in every City pub.

redwood 10y ago

Anyone here using k8s in prod? If so, what's your use case? What parts of the stack run inside k8s? Even state? What kind of org do you work for? Mission critical?

kokey 10y ago

Well, it's not a bank, yet. Their banking license restrictions could be lifted next year. That said if they are successful it will probably go a long way towards disrupting the customer experience of retail banking. The banks will be able to compete with this, but having something like this to model their improved experience on is good for them and good for customers in general. I am, however, a bit cynical about this. I think if Monzo takes off they will hit a hurdle or event along the way that can sink the company. Most fintech companies are overly vulnerable to making the same mistakes as most banks made many decades ago. In the end it will probably mean they will get salvaged by being bought by a bank. Then the customer experience won't improve so fast any more, while the back end and processes are being made robust to allow it to remain in business long term. The culture in the company will also change to be somewhat more, uhm, traditional and boring. The result will still be that the customer experience has been pushed forward in general and that is a good thing.

elcct 10y ago
I was trying to find an analogy, and I think this company is doing something like making TV guide more accessible and not having in mind that in a few years TV will be obsolete. Probably this will be a good business for the next decade, but it is not doing anything really disruptive.