7 ms·

New setup for 2020

117 points by fagnerbrack 6y ago

fairramone 6y ago
Is this really “simpler” as they claim? It reads a bit like the honeymoon phase and I’m a lot more interested in how they feel about the new stack 1 or 2 years down the line.
skinnyarms 6y ago
I'm a fan of the show, it was really entertaining to listen to knocking the site over "live".
- gerhardlazu 6y ago
  That was my favourite part too! Yes, we could have mitigated that entirely with CDN stale caching, but it was good to see what happens today, and then iterate towards better Fastly integration.
js4ever 6y ago
I was baffled by this: "The worst part is that serving the same files from disks local to the VMs is 44x faster than from persistent volumes (267MB/s vs 6MB/s)." Is it a configuration issue on their side or do the LKE volumes are really limited to 6MB/s on linode? How can you be happy with this for production??
- gerhardlazu 6y ago
  Block storage is an area that we are working with Linode to improve. That's the random read/write performance, as measured by fio. We have mostly sequential reads & writes (mp3 files) that peak at 50MB/s, then rely on CDN caching (Fastly makes us happy in this respect). CDN caching is something that we are currently improving, which will make things quicker and more reliable. The focus is on reality vs the ideal, and the path that we are taking to improving not just changelog.com, but also our sponsors' products. No managed K8S or IaaS is perfect, but we enjoy the Linode partnership & collaboration ;)
- sweeneyrod 6y ago
  Hey, that speed was good enough in ~1995!
- tracker1 6y ago
  It's a relatively common problem... if you need more IO, you may want to try a stripe of 4-8 block storage connections. Then again, depends on what you're doing.
  - gerhardlazu 6y ago
    For what it's worth, Rook, OpenEBS or Longhorn are worth exploring.
johnchristopher 6y ago
It seemed interesting and I tried to subscribe but the process doesn't work without disabling ublock/adblock. I am okay with that so I disabled it but then I failed two captchas round (find bikes in thumbnails, some thumbnails are upside down ?!) :(.
- jerodsanto 6y ago
  Sorry for the hassle! We’ve been hit by lots of spammers lately so I battened down the hatches. Unfortunately this has the side effect of also blocking some legit humans as well. :(
  - johnchristopher 6y ago
    Could 2FA or a mobile number be a good/practical alternative ?
remram 6y ago
Isn't changelog.com mostly a static site? What kind of workloads do they run/monitor/update with all this infrastructure?
- awinter-py 6y ago
  I think they're sponsored by linode, and they're developer-themed -- there may be team / content reasons to use a lot of unnecessary tools in order to review them
  - jerodsanto 6y ago
    This quote from Adam on our episode about the setup explains some of our motivations here: > It’s worth noting that we don’t really need what we have around Kubernetes. This is for fun, to some degree. One, we love Linode, they’re a great partner… Two, we love you, Gerhard, and all the work you’ve done here… We don’t really need this setup. One, it’s about learning ourselves, but then also sharing that. Obviously, Changelog.com is open source, so if you’re curious how this is implemented, you can look in our codebase. But beyond that, I think it’s important to remind our audience that we don’t really need this; it’s fun to have, and actually a worthwhile investment for us, because this does cost us money (Gerhard does not work for free), and it’s part of this desire to learn for ourselves, and then also to share it with everyone else… So that’s fun. It’s fun to do.
mdoms 6y ago
Isn't this just a podcast website, or is there something more to it? Why on earth would a podcast website need Kube?
- notsureaboutpg 6y ago
  A good portion of the article (the beginning portion no less!) is explicitly meant to answer exactly this question. When the writer anticipates this question and preemptively responds to it, least we could do is discuss their answer rather than repeat the question.
- that_guy_iain 6y ago
  Yea that is what I was thinking. And the next thing was what is the price difference. A bit of me thinks this is just some people doing some really cool tech work for the sake of it, which I can't fault them for.
- jerodsanto 6y ago
  The site does a lot more than merely hosting podcasts. There’s a very active news feed with submissions, commenting, newsletter subscriptions and management, a blog, episode requests, live streams, etc. Check out the source to see what all the app does: https://github.com/thechangelog/changelog.com https://github.com/thechangelog/changelog.com
- jlgaddis 6y ago
  Not knowing what changelog.com is, I was wondering the same thing... from the header of the page, I could tell that there's a blog, a podcast, and a newsletter. Without some additional insight that I don't have, it does seem that this is an enormously over-engineered "solution" for a website -- I've NFI why it requires 99.99% uptime! Perhaps they look it as a goal or challenge, an opportunity to showcase their knowledge and skills to potential customers, or, hell, maybe they just enjoy that kind of thing? If that's the case, I completely understand and can even relate (my home network is a textbook example of an "over-engineered solution": close to a dozen "enterprise-class" servers in the basement, ~35 various subnets, VMware Enterprise Plus clusters, BGP for anycast, and so on). AFAICT, though, this is just some developers running a blog and podcasts aimed at other developers? I mean, we're not exactly talking about a "mission critical" web site that's going to result in death and destruction the next time it goes down or Linode shits itself, right? Or am I missing something? -- EDIT: I've read through the rest of the comments now ... > This is for fun, to some degree ... We don’t really need this setup. One, it’s about learning ourselves, but then also sharing that ... It’s fun to do. ... and I completely understand!
jacques_chester 6y ago
I'd be interested in learning more about the move from Concourse to Circle (I'm a notorious Concourse fanboy). What went well, what didn't, what you miss, what prompted it -- that sort of thing.
- gerhardlazu 6y ago
  The primary reason behind the move was not wanting to manage CI. Since there were no options for a managed Concourse in 2018, we migrated to Circle, one of the Changelog sponsors at the time. Concourse worked well for us, we didn't have any issues that were being enough to remember. You may be interested in this screenshot that captured the changelog.com pipeline from 2017: https://pipeline.gerhard.io/images/small-oss.png https://pipeline.gerhard.io/images/small-oss.png I missed the simple Concourse pipeline view at first, but CircleCI improved by leaps and bounds in 2020, and the new Circle pipeline view equivalent is even better (compared to Concourse, clicking on jobs always works): https://app.circleci.com/pipelines/github/thechangelog/changelog.com/251/workflows/5836f363-060a-4bc5-8b9a-29988e59f0ad https://app.circleci.com/pipelines/github/thechangelog/chang... The Circle feature which I didn't expect to like as much as I do today, is the dashboard view (list of all pipeline/workflow runs). This is something that Concourse is still missing: https://app.circleci.com/pipelines/github/thechangelog https://app.circleci.com/pipelines/github/thechangelog My favourite Circle 2020 feature is the Insights: https://app.circleci.com/insights/github/thechangelog/changelog.com/workflows/build/overview https://app.circleci.com/insights/github/thechangelog/change.... Yup, we were one of the first ones to ask for it in 2019. In 2021, I expect us to spend one migration credit on GitHub Actions, as a Circle replacement. Argo comes as a second close, but that requires an innovation credit which is more precious to us. Because we are already using GitHub Actions for some automation, it would make sense to consolidate, and also leverage the GitHub Container Registry, as a migration from Docker Hub. Watch https://github.com/thechangelog/changelog.com https://github.com/thechangelog/changelog.com to see what happens : )
  - jacques_chester 6y ago
    Thanks, a very comprehensive answer. I agree especially that the lack of a hosted option has hamstrung Concourse's success. I couldn't use the Circle CI links, they're auth gated.
awinter-py 6y ago
woo self-managed postgres on k8s vendor-managed DBs limit plugins + versions, excited to see this space advance
lifeisstillgood 6y ago
>>> We no longer provision load balancers, or configure DNS; we simply describe the resources that we need, and Kubernetes makes it happen. This is (part) of what keeps me in the stone age. You are provisioning load balancers and DNS - but just one step removed through k8s And my prior is that we need to understand that, be aaare of it, have a model of what is going on to help develop and debug. And so it feels a bit like "magic abstraction". And then to peek through the abstraction you suddenly not only need to know about DNS and which machine is running bind, but also how kubernetes internally stores it's DNS config and how it spits that out and what version they changed it with. In other words you have to become expect in two things to debug it. And maybe it's worth it - but I struggle to see why it's not simpler to keep my install scripts going. (OK I guess I am writing my own answer - but surely the point is what is the simplest level of thin install scripts needed to deploy containers?)
- kbar13 6y ago
  i think the idea is that k8s does away with having to glue all those pieces of infra together, not that you lose understanding of how it all works together. part of the headache with managing infra is that it rots over time... things come and go (sysv to systemd, apt/snap/whatever, config files change, things break). it's easier to keep up to date on k8s than all the disparate parts of the OS and provider-specific APIs and whatnot
  - lifeisstillgood 6y ago
    That's an interesting perspective I have not heard before. Does this imply there is a cloud abstract layer that should come (assuming all providers can put aside commercial interests etc) And is k8s the simplest possible abstraction? And if not - what is?
    - gerhardlazu 6y ago
      > Does this imply there is a cloud abstract layer that should come crossplane.io comes closest afaik > And is k8s the simplest possible abstraction? And if not - what is? If you are asking about the simplest possible abstraction for container scheduling and orchestration, then I believe Nomad from HashiCorp or Docker Swarm are simpler. As for managed solutions with wide adoption in all types of environments and the largest investment to date, I am not aware of anything on par with K8S.
manigandham 6y ago
The static files should definitely be on some kind of object storage like S3, that's what it's built for. Much faster, more reliable, more scalable, and likely much cheaper too. As for persistent volumes, might be better to just offload Postgres to a managed DB service and downsize the K8S instances, or use something like CockroachDB which is natively distributed and can make use of local volumes instead.
- gerhardlazu 6y ago
  Yes, it does make sense to move static files to object storage, especially the mp3s. There is some ffmpeg-related refactoring that we need to do before we can do this though, and it's not a quick & easy task, so we have been deferring it since it's not that high priority, and there are simpler solutions to this particular problem (i.e. improved CDN caching). Other static files such as css, js, txt make sense to remain bundled with the app image, which is stateless and a prime candidate for horizontal scaling. Also, CDN caching makes small static files that change infrequently a non issue, regardless of their origin. The managed Postgres service from Linode's 2021 roadmap is definitely something that we are looking forward to, but the simplest thing might be to provision Postgres with local volumes instead. We are already using a replicated Postgres via the Crunchy PostgreSQL Operator, so I'm looking forward to trying this approach out first. CockroachDB is on my list of cool tech to experiment with, but that will use an innovation token, and we only have a few left for 2021, so I don't want to spend them all at once.
  - manigandham 6y ago
    Yea the small static files that are part of your webapp can stay with it, but media files are best on S3. If you need a block interface though, I recommend something like ObjectiveFS: https://objectivefs.com/ https://objectivefs.com/ If you're using an operator then local volumes is a good middleground if it automates the replication already. CockroachDB also has a kubernetes operator although it's only for GKE currently. There are also other options like YugabyteDB which is another cloud-native postgres-compatible DB.
    - 6y ago
PietKachelhout 6y ago
I'm disappointed that blogs and podcasts keep promoting Linode. I currently maintain about 30 Linodes and I have been doing so for the past 2 years. Some things I noticed: * The internal network is not private. But people don't realise it. You share a /16 with other Linodes. So many open databases, file shares and other services in there. * Block storage performance is really poor, around 100 iops. Same as a SATA disk from 10 years ago. * No proper snapshot / image functionality. * Linode Kubeternetes Engine was based on Debian Oldstable when it launched. * Excessive CPU steal, even on dedicated cores. 25% CPU steal is considered normal. Over 50% happens a lot. * Problems with their hosts. I can only guess what the reason is but 4 to 8 hours of unannounced downtime of a VM happend to me 6 times in the past 2 years. Yes, support is friendly. But my international phone bill is huge because the fastest way to get them to do something is to call.
- brobinson 6y ago
  With all these negatives, there must be a really compelling reason to stay. What is it?
  - PietKachelhout 6y ago
    The owner of the company thinks they are great because you can call them when there is a problem. I'm having a hard time convincing him that these are issues I've never had at other providers. Certainly not so many. We are moving everything away. Most of our servers are with another provider already. And we haven't had any similar issues there. I've never called them! And I forgot to mention the connectivity issues at Linode. When the whole London datacenter was unreachable for 2 hours we lost some customers.
    - tgsovlerkhgsel 6y ago
      This is something more companies need to realize: Yes, you should be easily and quickly reachable by phone so that when (not if) things go wrong, I can get good service and resolve it quickly. But if I'm calling you, you have most likely already failed. If I'm calling for information, your documentation has failed to make that information accessible (it either wasn't documented, or not easy enough to find). If I'm calling to resolve an issue (technical or billing), it would have been a lot better if it didn't happen in the first place.
SergeAx 6y ago
I browsed the site swiftly. Isn't it a bunch of content items, like WordPress site? So, here we are, deploying WordPress in the cloud via Kubernetes, for real, not as a funny meme?
- gerhardlazu 6y ago
  changelog.com used to be WordPress, then became a Phoenix app because it needed features that were hacky to implement & then manage in WP. It's more of a podcasting platform these days rather than a CMS. The code in this repo tells the truth about what it is, and even shows how it works: https://github.com/thechangelog/changelog.com https://github.com/thechangelog/changelog.com

New setup for 2020

117 points by fagnerbrack 6y ago

Is this really “simpler” as they claim? It reads a bit like the honeymoon phase and I’m a lot more interested in how they feel about the new stack 1 or 2 years down the line.

skinnyarms 6y ago

I'm a fan of the show, it was really entertaining to listen to knocking the site over "live".

gerhardlazu 6y ago
That was my favourite part too! Yes, we could have mitigated that entirely with CDN stale caching, but it was good to see what happens today, and then iterate towards better Fastly integration.

js4ever 6y ago

I was baffled by this: "The worst part is that serving the same files from disks local to the VMs is 44x faster than from persistent volumes (267MB/s vs 6MB/s)." Is it a configuration issue on their side or do the LKE volumes are really limited to 6MB/s on linode? How can you be happy with this for production??

gerhardlazu 6y ago
Block storage is an area that we are working with Linode to improve. That's the random read/write performance, as measured by fio. We have mostly sequential reads & writes (mp3 files) that peak at 50MB/s, then rely on CDN caching (Fastly makes us happy in this respect). CDN caching is something that we are currently improving, which will make things quicker and more reliable. The focus is on reality vs the ideal, and the path that we are taking to improving not just changelog.com, but also our sponsors' products. No managed K8S or IaaS is perfect, but we enjoy the Linode partnership & collaboration ;)
sweeneyrod 6y ago
Hey, that speed was good enough in ~1995!
tracker1 6y ago
It's a relatively common problem... if you need more IO, you may want to try a stripe of 4-8 block storage connections. Then again, depends on what you're doing.
- gerhardlazu 6y ago
  For what it's worth, Rook, OpenEBS or Longhorn are worth exploring.

johnchristopher 6y ago

It seemed interesting and I tried to subscribe but the process doesn't work without disabling ublock/adblock. I am okay with that so I disabled it but then I failed two captchas round (find bikes in thumbnails, some thumbnails are upside down ?!) :(.

jerodsanto 6y ago
Sorry for the hassle! We’ve been hit by lots of spammers lately so I battened down the hatches. Unfortunately this has the side effect of also blocking some legit humans as well. :(
- johnchristopher 6y ago
  Could 2FA or a mobile number be a good/practical alternative ?

remram 6y ago

Isn't changelog.com mostly a static site? What kind of workloads do they run/monitor/update with all this infrastructure?

awinter-py 6y ago
I think they're sponsored by linode, and they're developer-themed -- there may be team / content reasons to use a lot of unnecessary tools in order to review them
- jerodsanto 6y ago
  This quote from Adam on our episode about the setup explains some of our motivations here: > It’s worth noting that we don’t really need what we have around Kubernetes. This is for fun, to some degree. One, we love Linode, they’re a great partner… Two, we love you, Gerhard, and all the work you’ve done here… We don’t really need this setup. One, it’s about learning ourselves, but then also sharing that. Obviously, Changelog.com is open source, so if you’re curious how this is implemented, you can look in our codebase. But beyond that, I think it’s important to remind our audience that we don’t really need this; it’s fun to have, and actually a worthwhile investment for us, because this does cost us money (Gerhard does not work for free), and it’s part of this desire to learn for ourselves, and then also to share it with everyone else… So that’s fun. It’s fun to do.

mdoms 6y ago

Isn't this just a podcast website, or is there something more to it? Why on earth would a podcast website need Kube?

notsureaboutpg 6y ago
A good portion of the article (the beginning portion no less!) is explicitly meant to answer exactly this question. When the writer anticipates this question and preemptively responds to it, least we could do is discuss their answer rather than repeat the question.
that_guy_iain 6y ago
Yea that is what I was thinking. And the next thing was what is the price difference. A bit of me thinks this is just some people doing some really cool tech work for the sake of it, which I can't fault them for.
jerodsanto 6y ago
The site does a lot more than merely hosting podcasts. There’s a very active news feed with submissions, commenting, newsletter subscriptions and management, a blog, episode requests, live streams, etc. Check out the source to see what all the app does: https://github.com/thechangelog/changelog.com https://github.com/thechangelog/changelog.com
jlgaddis 6y ago
Not knowing what changelog.com is, I was wondering the same thing... from the header of the page, I could tell that there's a blog, a podcast, and a newsletter. Without some additional insight that I don't have, it does seem that this is an enormously over-engineered "solution" for a website -- I've NFI why it requires 99.99% uptime! Perhaps they look it as a goal or challenge, an opportunity to showcase their knowledge and skills to potential customers, or, hell, maybe they just enjoy that kind of thing? If that's the case, I completely understand and can even relate (my home network is a textbook example of an "over-engineered solution": close to a dozen "enterprise-class" servers in the basement, ~35 various subnets, VMware Enterprise Plus clusters, BGP for anycast, and so on). AFAICT, though, this is just some developers running a blog and podcasts aimed at other developers? I mean, we're not exactly talking about a "mission critical" web site that's going to result in death and destruction the next time it goes down or Linode shits itself, right? Or am I missing something? -- EDIT: I've read through the rest of the comments now ... > This is for fun, to some degree ... We don’t really need this setup. One, it’s about learning ourselves, but then also sharing that ... It’s fun to do. ... and I completely understand!

jacques_chester 6y ago

I'd be interested in learning more about the move from Concourse to Circle (I'm a notorious Concourse fanboy). What went well, what didn't, what you miss, what prompted it -- that sort of thing.

gerhardlazu 6y ago
The primary reason behind the move was not wanting to manage CI. Since there were no options for a managed Concourse in 2018, we migrated to Circle, one of the Changelog sponsors at the time. Concourse worked well for us, we didn't have any issues that were being enough to remember. You may be interested in this screenshot that captured the changelog.com pipeline from 2017: https://pipeline.gerhard.io/images/small-oss.png https://pipeline.gerhard.io/images/small-oss.png I missed the simple Concourse pipeline view at first, but CircleCI improved by leaps and bounds in 2020, and the new Circle pipeline view equivalent is even better (compared to Concourse, clicking on jobs always works): https://app.circleci.com/pipelines/github/thechangelog/changelog.com/251/workflows/5836f363-060a-4bc5-8b9a-29988e59f0ad https://app.circleci.com/pipelines/github/thechangelog/chang... The Circle feature which I didn't expect to like as much as I do today, is the dashboard view (list of all pipeline/workflow runs). This is something that Concourse is still missing: https://app.circleci.com/pipelines/github/thechangelog https://app.circleci.com/pipelines/github/thechangelog My favourite Circle 2020 feature is the Insights: https://app.circleci.com/insights/github/thechangelog/changelog.com/workflows/build/overview https://app.circleci.com/insights/github/thechangelog/change.... Yup, we were one of the first ones to ask for it in 2019. In 2021, I expect us to spend one migration credit on GitHub Actions, as a Circle replacement. Argo comes as a second close, but that requires an innovation credit which is more precious to us. Because we are already using GitHub Actions for some automation, it would make sense to consolidate, and also leverage the GitHub Container Registry, as a migration from Docker Hub. Watch https://github.com/thechangelog/changelog.com https://github.com/thechangelog/changelog.com to see what happens : )
- jacques_chester 6y ago
  Thanks, a very comprehensive answer. I agree especially that the lack of a hosted option has hamstrung Concourse's success. I couldn't use the Circle CI links, they're auth gated.

awinter-py 6y ago

woo self-managed postgres on k8s vendor-managed DBs limit plugins + versions, excited to see this space advance

lifeisstillgood 6y ago

>>> We no longer provision load balancers, or configure DNS; we simply describe the resources that we need, and Kubernetes makes it happen. This is (part) of what keeps me in the stone age. You are provisioning load balancers and DNS - but just one step removed through k8s And my prior is that we need to understand that, be aaare of it, have a model of what is going on to help develop and debug. And so it feels a bit like "magic abstraction". And then to peek through the abstraction you suddenly not only need to know about DNS and which machine is running bind, but also how kubernetes internally stores it's DNS config and how it spits that out and what version they changed it with. In other words you have to become expect in two things to debug it. And maybe it's worth it - but I struggle to see why it's not simpler to keep my install scripts going. (OK I guess I am writing my own answer - but surely the point is what is the simplest level of thin install scripts needed to deploy containers?)

kbar13 6y ago
i think the idea is that k8s does away with having to glue all those pieces of infra together, not that you lose understanding of how it all works together. part of the headache with managing infra is that it rots over time... things come and go (sysv to systemd, apt/snap/whatever, config files change, things break). it's easier to keep up to date on k8s than all the disparate parts of the OS and provider-specific APIs and whatnot
- lifeisstillgood 6y ago
  That's an interesting perspective I have not heard before. Does this imply there is a cloud abstract layer that should come (assuming all providers can put aside commercial interests etc) And is k8s the simplest possible abstraction? And if not - what is?
  - gerhardlazu 6y ago
    > Does this imply there is a cloud abstract layer that should come crossplane.io comes closest afaik > And is k8s the simplest possible abstraction? And if not - what is? If you are asking about the simplest possible abstraction for container scheduling and orchestration, then I believe Nomad from HashiCorp or Docker Swarm are simpler. As for managed solutions with wide adoption in all types of environments and the largest investment to date, I am not aware of anything on par with K8S.

manigandham 6y ago

The static files should definitely be on some kind of object storage like S3, that's what it's built for. Much faster, more reliable, more scalable, and likely much cheaper too. As for persistent volumes, might be better to just offload Postgres to a managed DB service and downsize the K8S instances, or use something like CockroachDB which is natively distributed and can make use of local volumes instead.

gerhardlazu 6y ago
Yes, it does make sense to move static files to object storage, especially the mp3s. There is some ffmpeg-related refactoring that we need to do before we can do this though, and it's not a quick & easy task, so we have been deferring it since it's not that high priority, and there are simpler solutions to this particular problem (i.e. improved CDN caching). Other static files such as css, js, txt make sense to remain bundled with the app image, which is stateless and a prime candidate for horizontal scaling. Also, CDN caching makes small static files that change infrequently a non issue, regardless of their origin. The managed Postgres service from Linode's 2021 roadmap is definitely something that we are looking forward to, but the simplest thing might be to provision Postgres with local volumes instead. We are already using a replicated Postgres via the Crunchy PostgreSQL Operator, so I'm looking forward to trying this approach out first. CockroachDB is on my list of cool tech to experiment with, but that will use an innovation token, and we only have a few left for 2021, so I don't want to spend them all at once.
- manigandham 6y ago
  Yea the small static files that are part of your webapp can stay with it, but media files are best on S3. If you need a block interface though, I recommend something like ObjectiveFS: https://objectivefs.com/ https://objectivefs.com/ If you're using an operator then local volumes is a good middleground if it automates the replication already. CockroachDB also has a kubernetes operator although it's only for GKE currently. There are also other options like YugabyteDB which is another cloud-native postgres-compatible DB.
  - 6y ago

PietKachelhout 6y ago

I'm disappointed that blogs and podcasts keep promoting Linode. I currently maintain about 30 Linodes and I have been doing so for the past 2 years. Some things I noticed: * The internal network is not private. But people don't realise it. You share a /16 with other Linodes. So many open databases, file shares and other services in there. * Block storage performance is really poor, around 100 iops. Same as a SATA disk from 10 years ago. * No proper snapshot / image functionality. * Linode Kubeternetes Engine was based on Debian Oldstable when it launched. * Excessive CPU steal, even on dedicated cores. 25% CPU steal is considered normal. Over 50% happens a lot. * Problems with their hosts. I can only guess what the reason is but 4 to 8 hours of unannounced downtime of a VM happend to me 6 times in the past 2 years. Yes, support is friendly. But my international phone bill is huge because the fastest way to get them to do something is to call.

brobinson 6y ago
With all these negatives, there must be a really compelling reason to stay. What is it?
- PietKachelhout 6y ago
  The owner of the company thinks they are great because you can call them when there is a problem. I'm having a hard time convincing him that these are issues I've never had at other providers. Certainly not so many. We are moving everything away. Most of our servers are with another provider already. And we haven't had any similar issues there. I've never called them! And I forgot to mention the connectivity issues at Linode. When the whole London datacenter was unreachable for 2 hours we lost some customers.
  - tgsovlerkhgsel 6y ago
    This is something more companies need to realize: Yes, you should be easily and quickly reachable by phone so that when (not if) things go wrong, I can get good service and resolve it quickly. But if I'm calling you, you have most likely already failed. If I'm calling for information, your documentation has failed to make that information accessible (it either wasn't documented, or not easy enough to find). If I'm calling to resolve an issue (technical or billing), it would have been a lot better if it didn't happen in the first place.

SergeAx 6y ago

I browsed the site swiftly. Isn't it a bunch of content items, like WordPress site? So, here we are, deploying WordPress in the cloud via Kubernetes, for real, not as a funny meme?

gerhardlazu 6y ago
changelog.com used to be WordPress, then became a Phoenix app because it needed features that were hacky to implement & then manage in WP. It's more of a podcasting platform these days rather than a CMS. The code in this repo tells the truth about what it is, and even shows how it works: https://github.com/thechangelog/changelog.com https://github.com/thechangelog/changelog.com