7 ms·

The Guardian goes all-in on AWS public cloud after OpenStack 'disaster’

79 points by robzyb 11y ago

Gigablah 11y ago
Going off-topic here, but the AWS logo used in the article seriously looks like a cheap knock-off version. Edit: apparently they sourced it from Wikimedia Commons, which has a rather... shoddy replica.
melted 11y ago
Sounds like they'll be writing that AWS is a disaster 2 years down the road.
jdubs 11y ago
After developing a product on openstack and dealing with so many issues; I miss AWS so much.
MrTonyD 11y ago
As part of my job I teach some classes and give the occasional conference talk. As a result, I've spoken to dozens of people who have tried to adopt OpenStack and eventually gave up. There are a lot of reasons. But I also happen to know a very senior technical person with OpenStack - and he described a completely dysfunctional organization. I wish that there were some way to stop these organizations that are wasting people's time - maybe somebody could sue them out of business for the good of the industry?
- johansch 11y ago
  "More than 500 companies have joined the project, including AppFormix, Arista Networks, AT&T, AMD, Avaya, Brocade, Canonical, Cisco, Citrix, Comcast, Cray, Dell, Dreamhost, EMC, Ericsson, Fujitsu, Go Daddy, Google, Hewlett-Packard, Hitachi Data Systems, Huawei, IBM, Intel, Internap, Juniper Networks, Mellanox, Mirantis, NEC, NetApp, Nexenta, Oracle, PLUMgrid, Pure Storage, Qosmos, Red Hat, Solidfire, SUSE Linux, VMware, VMTurbo and Yahoo!" Yeah, I am sure it is quite easy to make them agree on the directions OS should take...
alrs 11y ago
Andrew Clay Shafer called it in 2013: https://stochasticresonance.wordpress.com/2013/11/04/openstack-a-plea/ https://stochasticresonance.wordpress.com/2013/11/04/opensta... In a former life, I was frustrated by the futility of trying to be involved with the Openstack project. My next phase was to warn people away. Now I wonder why I bothered. Years later it's apparent that I was right, but what benefit do I see from that? Better to aggressively ignore bad technology and spend effort on the stuff that looks promising.
- rwmj 11y ago
  He mentions Ceilometer which was recently rewritten (https://wiki.openstack.org/wiki/Gnocchi https://wiki.openstack.org/wiki/Gnocchi). Of course whether the replacement will work any better is an open question at this point. Never really understood why they don't just use rrdtool and have done. Networking which he also mentions in passing is also being rewritten (https://www.openstack.org/summit/vancouver-2015/summit-videos/presentation/ovn-native-virtual-networking-for-open-vswitch https://www.openstack.org/summit/vancouver-2015/summit-video...). I generally agree with his final points - OpenStack really needs to work for the end users, not for the many competing vendors working on it.
  - alrs 11y ago
    They're not competing, they're cooperatively building a potemkin village so that they have a cloud story to tell Wall Street and the tech press. AWS is steamrollering everyone, but most of them incorrectly imagine AWS to be a mashup between VMware and a payday lender scam. They've been content to put people on stage every six months to peddle this-actually-works happy-talk. Dell tells the story. February 5, 2013: Dell announces plans to go private. May 20, 2013: Dell abandons Openstack.
    - sinxoveretothex 11y ago
      Dell abandoned Openstack in May 2013? Better tell ComputerWorldUK then, they seem out of date by 9 months because they linked me to this story from December 2013: http://www.computerworlduk.com/news/it-vendors/dell-and-red-hat-team-to-sell-enterprise-openstack-3493664/ http://www.computerworlduk.com/news/it-vendors/dell-and-red-...
dankohn1 11y ago
"we have had to host it on physical tin". Is this an English English thing? In America, we would say "host on our own iron". Do other countries use different metals?
- drrob 11y ago
  I'm English, and I have never used or even heard the phrase "host it on physical tin" before. Then again, I just tend to say "hosted on our own hardware" or just simply "self-hosted".
- petewailes 11y ago
  British person: I don't think this is one I've come across.
  - f3y67j 11y ago
    Another brit: just once, three years ago by a Programme Director determined to eradicate our IT dept "We are not going to feed and water our own tin!". I was equally bemused. However the outcome has been very detrimental to our org.
- techterrier 11y ago
  probably cornish :D
  - jinst8gmi 11y ago
    They do make great pies :)
    - techterrier 11y ago
      don't forget the ice cream!
- gaius 11y ago
  Yep pretty common in the UK. The chargeback the IT department makes to other groups or projects for kit is called the "tin tax" too.
pastyboy 11y ago
money no object then? Of course not. Safe harbour is a huge problem in EU AWS breaking the rules. The G could find itself in a sticky regulation mess when people start suing for data breeches.
- vidarh 11y ago
  Money probably no object given that they were using Cisco UCS and Netapp storage to begin with. With that as a starting point, a perceived disaster brewing and what's likely to be fairly steep discounts from AWS to get them to move given their size I'm sure it looked attractive (if you're paying list prices for AWS at any kind of size, you're leaving money on the table; if you're first going to use AWS at least make sure to negotiate).
jakozaur 11y ago
Related read: https://www.packet.net/blog/how-we-failed-at-openstack/ https://www.packet.net/blog/how-we-failed-at-openstack/
vidarh 11y ago
It's a bizarre article. On one hand I understand their frustration with OpenStack. On the other hand, if they expected to pretty much build their own AWS without any warning bells going off all over the place I find it very hard to be sympathetic - they're not remotely large enough for that. If they'd set their goals lower they could have had a very capable system fairly easily. That they're having problems getting load balancing working properly is a real sign that their ops team just didn't know what they're doing.
- rrmm 11y ago
  if the choice is "let's deploy software on AWS," vs "let's build an AWS work-a-like and then deploy software on it," it'll always be easier to use AWS. Managing physical servers is a pain. I've been in a similar situation in a small research group, but AWS doesn't fly for privacy and regulatory reasons. Managing hardware and infrastructure is a huge amount of overhead that doesn't directly relate to the core mission.
  - vidarh 11y ago
    > if the choice is "let's deploy software on AWS," vs "let's build an AWS work-a-like and then deploy software on it," it'll always be easier to use AWS Yes, but that's a false equivalence. You don't need an AWS work-a-like unless you plan on competing with AWS. So much of AWS is dedicated to doing things you don't need to do in most single tenant systems. But if managing hardware and infrastructure is a huge amount of overhead, what in the world are you doing? If it's more than sliding a server into a rack, hooking up network and power, and leaving it to itself until/unless there are hardware failures or SMART alerts, then you've missed out on automation steps somewhere. For server environments I set up, I spend perhaps on average 30m per server in a datacentre per year, including travel. On top of that we estimate on average perhaps 10m of remote hands per server per year.
- _Codemonkeyism 11y ago
  "they're not remotely large enough for that." Second that. Maybe there are only ten companies in the world large enough to have positive ROI to build an internal AWS, like Google, eBay, Facebook, ... who don't want to depend on Amazon.
hoodoof 11y ago
"That software we've got causes us so many problems, it's disaster! When we get the new software everything will be fixed." X to Y Y to X A to B B to C MongoDB to this MySQL to that Why we switched Why we switched back blah blah It's one of the oldest themes in computers - man, this software is crap - that new software will fix all our problems! Until you find the problems with the new. Precisely the same age is the new software vendor harnessing all that negative energy about the competitor and megaphoning the "man, the new will fix the old, we're so excited!" Having said that, I have worked with the compute servers of all the major cloud vendors and Amazon must be credited for the quality and consistency of its AWS systems, and Google and Microsoft's cloud computing is in almost all respects equally good - there is as far as I can tell absolutely no reason to choose Amazon over Microsoft Azure (yes, even for Linux systems) or Google Compute Engine. In some areas of functionality GCE and Azure I found to work much more easily and smoothly than Amazon. When I went to work with OpenStack I immediately found the lack of completeness, inconsistency and lack of polish characteristic of many open source projects - trying to do simple things instantly became hard problems.
- robzyb 11y ago
  > "That software we've got causes us so many problems, it's disaster! When we get the new software everything will be fixed." This way of thinking isn't exclusive to software.
  - hoodoof 11y ago
    Do you mean life partners? Anyhow its especially prevalent in software. Just search Google for "why we switched from". If anyone is thinking of writing yet another "why we switched from X to Y" blog post, I can save you the effort of a long blog post explaining the reasons. It's because "the old software was shit and the new is fricking awesome!" You don't need to explain the details.
    - robzyb 11y ago
      > Do you mean life partners? I mean everything. Almost literally everything. The most recent thing I can think of is seating arrangements. At my current workplace seating arrangements are mostly random. Someone here had the idea that we should group people by seniority so that people with related issues will be in close proximity. However, at my previous workplace we grouped by seniority, and someone had the idea that we should mix it up so that project teams would be in close proximity. [edit]: And lets not forget outsourcing vs. insourcing, grouping departments by geography vs. grouping departments by business type, etc., etc.
DanielBMarkham 11y ago
You know, there's probably a sweet spot between "AWS all the things!" and "We will build our own cloud" I mean geesh, people have been building small clouds since there were servers. That's the way mom and dad did it, and by gummity it oughta be good enough for you. The "new" stuff was autoscaling, PaaS, and so forth. So build out a few servers for content management and publishing, then write very small amount of code to push what you have out to a CDN. If you want realtime data capture, capture it using AWS (or whatnot) and pull it back locally. I'm not saying that's optimum for every solution, just that the all-or-nothing kind of thinking is probably what lured them into building their own cloud in the first place. You need a cloud for some stuff, so use a cloud. But you don't need a cloud for every freaking thing the company does. Its assets in the form of text content, internal docs, and branding are probably extremely small in modern terms.
smcl 11y ago
There's little substance to this other than "OpenStack is hard, let's go AWS!" - I would have liked to hear where exactly they ran into difficulties and what the problem was in detail. This is for two reasons: 1. "We have switched some components in our software stack" without much info is not an interesting story to read. 2. I have friends who work on OpenStack in RH who would I'm sure be very interested to know the sort of troubles that The Guardian had so if there is a usability or functionality issue it can be addressed.
- deleted 11y ago
  [deleted]
danieltillett 11y ago
I know cloud everything is the modern way to go, but isn't the Guardian just a glorified blog? Why could they not just push out their content to a series of load balanced static iron severs? Call a cron job every 5 minutes and be done with it.
- illumen 11y ago
  They are a dating website amongst other things. It's a large billion pound media organisation now with many different units.
  - danieltillett 11y ago
    I am not sure why they are running a dating site, but even if they consider this important why one system to rule them all. Hardware is cheap compared to developers.
    - DanBC 11y ago
      > I am not sure why they are running a dating site Many newspapers (at least in UK) did. Guardian is one that stuck around to digital age. It makes them money.
- 72deluxe 11y ago
  Perhaps they wanted a more "exciting" way of doing it?
- jsmeaton 11y ago
  First thing off the top of my head is tracking engagement etc. Also search? Not many things are simple anymore, especially when you're selling advertising space.
saintfiends 11y ago
It's a bit scary that Amazon AWS is the only complete solution in this space now. There are so many open projects in the cloud space that one would think it's a solved problem. Here's a list off the top of my head: 1. Docker - Container implementation. 2. Kubernets - Manage a cluster of Linux containers. 3. Mesos - Manage a cluster of resources (not just containers, I'm guessing there are some feature overlap with Kubernets). 4. CoreOS - An OS specialized in running containers. 5. RethinkDB, CouchDB, Cassandra - Distributed databases 6. Ceph, GlusterFS - Distributed file systems 7. RabbitMQ, NSQ, NATS - Distributed queue systems. 8. Manage VM's in a Data center ?? - I don't know any projects in this space. What's missing is an interface to manage all these together. Maybe this is the direction OpenStack should be heading?
- athrun 11y ago
  You are comparing open-source Software against Services. It's as if you were saying that to compete with Amazon.com (retail) you simply have to deploy Magento or Open Cart...
- pjc50 11y ago
  Many projects do not make a solution in the same way that many grains of sand do not, of themselves, make a sandcastle.
- kogepathic 11y ago
  > 8. Manage VM's in a Data center ?? - I don't know any projects in this space. 1. Apache CloudStack ( https://cloudstack.apache.org/ https://cloudstack.apache.org/ ) 2. oVirt + Foreman ( http://www.ovirt.org/Home http://www.ovirt.org/Home , http://theforeman.org/ http://theforeman.org/ ) 3. OpenNebula ( http://opennebula.org/ http://opennebula.org/ ) 4. Roll your own with clusterlabs + PXE + puppet
meekins 11y ago
I'd be really interested in the actual technical problems The Guardian had with OpenStack and what's the current situation with the OpenStack project. The company I work for has some interest on the platform but I am not aware what to expect (problems, new features etc) from it in its current state. A couple of years ago I experimented with and compared some open source cloud platforms. Admittedly my knowledge is old, but alredy then I was wondering the hype and visibility of the OpenStack project. While it has many big-name supporters that guarantee its lucrativeness for the enterprise, its feature set and flexibility was seriously behind other open source alternatives. Back then I fell for OpenNebula because even though it had its warts it already delivered many features (especially concerning hybrid clouds and heterogenous virutalization environments) that OpenStack still had on its future roadmap.
- rkeene2 11y ago
  I also tried OpenStack for a self-assembling cloud product I built. It was a huge pain and I also switched to OpenNebula about a year ago. It's been significantly better !
Angostura 11y ago
Stephen Gran, the senior integrator charged with getting OpenStack to work and who wrote fulsomely about it left the Guardian in March 2014 according to LinkedIn.
jackgavigan 11y ago
I've always been introgued by the Guardian's focus on building its own technology. It seems a "bold" strategy, given that tech is not its core business and the newspaper part of the group (which is pretty much all that remains now, since they've sold off their other assets) has been consistently losing money for some time now. I wonder if that strategy would still be in place if it wasn't owned by a not-for-profit trust. Other publishers who have gone down the "We're a tech company!" route have been forced to give up on that strategy[1]. I wonder whether the Guardian will be able to make that breakthrough, or whether they'll end up migrating to Wordpress or something similar. 1: http://digiday.com/publishers/gawkers-kinja-retreat-shows-false-hope-in-publishers-licensing-tech/ http://digiday.com/publishers/gawkers-kinja-retreat-shows-fa...
- gaius 11y ago
  They have a few very charismatic people in tech there, the kind of people who are very good showmen and salesmen (or women) but they've been "architects" or similar their whole careers, not hands-dirty engineers, and their grasp of what's feasible or not is driven mainly by which vendor buys them the nicest lunch or has the nicest typography in their whitepapers.
  - jackgavigan 11y ago
    Do you have direct experience of working there?
    - gaius 11y ago
      Back in the day as an expensive consultant, yes, and I know a few people there now. If you want to see a newspaper who are really doing something interesting with technology, check out the Daily Mail, last I heard they were going all-in on Scala. Newspapers that are still in print are fascinating IT places - because come hell or high water, the paper has to come out the next morning. They are the definition of mission critical. Websites are easy in comparison.
      - 11y ago
trolla 11y ago
I’ve used Openstack at my current $dayjob and I’d say that Openstack is probably one of the buggiest solutions that I’ve ever used. Even simple things like stopping or resizing VM-s sometimes failed completely and the VM came unusable after that. Feature wise it’s also lacking. Seems that Openstack spends more time/money on marketing and generating hype than creating a good solution. I’d even say that Docker has a somewhat similar problems, they spend more on hype/marketing than creating a stable and usable solution. However, at least Docker is more usable right now than a year ago, the same can’t be said about Openstack.
- mkulke 11y ago
  I guess those expectations are part of the problem, a talk in the recent openstack conf higlighted this: OpenStack was initially created for what these days is called "cloud-native" workloads, every VM was considered ephemeral. companies and users then try to mould it into a cheaper VMWare and are frustrated how bad it is at this. Resizing, stopping VMs, while admittedly being a rather trivial tasks, point to this usage of OpenStack. When I would mourn the loss of individual VMs on OpenStack (or public clouds for that matter), I would turn gray soon.
deleted 11y ago
[deleted]

79 points by robzyb 11y ago

Going off-topic here, but the AWS logo used in the article seriously looks like a cheap knock-off version. Edit: apparently they sourced it from Wikimedia Commons, which has a rather... shoddy replica.

melted 11y ago

Sounds like they'll be writing that AWS is a disaster 2 years down the road.

jdubs 11y ago

After developing a product on openstack and dealing with so many issues; I miss AWS so much.

MrTonyD 11y ago

As part of my job I teach some classes and give the occasional conference talk. As a result, I've spoken to dozens of people who have tried to adopt OpenStack and eventually gave up. There are a lot of reasons. But I also happen to know a very senior technical person with OpenStack - and he described a completely dysfunctional organization. I wish that there were some way to stop these organizations that are wasting people's time - maybe somebody could sue them out of business for the good of the industry?

johansch 11y ago
"More than 500 companies have joined the project, including AppFormix, Arista Networks, AT&T, AMD, Avaya, Brocade, Canonical, Cisco, Citrix, Comcast, Cray, Dell, Dreamhost, EMC, Ericsson, Fujitsu, Go Daddy, Google, Hewlett-Packard, Hitachi Data Systems, Huawei, IBM, Intel, Internap, Juniper Networks, Mellanox, Mirantis, NEC, NetApp, Nexenta, Oracle, PLUMgrid, Pure Storage, Qosmos, Red Hat, Solidfire, SUSE Linux, VMware, VMTurbo and Yahoo!" Yeah, I am sure it is quite easy to make them agree on the directions OS should take...

alrs 11y ago

Andrew Clay Shafer called it in 2013: https://stochasticresonance.wordpress.com/2013/11/04/openstack-a-plea/ https://stochasticresonance.wordpress.com/2013/11/04/opensta... In a former life, I was frustrated by the futility of trying to be involved with the Openstack project. My next phase was to warn people away. Now I wonder why I bothered. Years later it's apparent that I was right, but what benefit do I see from that? Better to aggressively ignore bad technology and spend effort on the stuff that looks promising.

rwmj 11y ago
He mentions Ceilometer which was recently rewritten (https://wiki.openstack.org/wiki/Gnocchi https://wiki.openstack.org/wiki/Gnocchi). Of course whether the replacement will work any better is an open question at this point. Never really understood why they don't just use rrdtool and have done. Networking which he also mentions in passing is also being rewritten (https://www.openstack.org/summit/vancouver-2015/summit-videos/presentation/ovn-native-virtual-networking-for-open-vswitch https://www.openstack.org/summit/vancouver-2015/summit-video...). I generally agree with his final points - OpenStack really needs to work for the end users, not for the many competing vendors working on it.
- alrs 11y ago
  They're not competing, they're cooperatively building a potemkin village so that they have a cloud story to tell Wall Street and the tech press. AWS is steamrollering everyone, but most of them incorrectly imagine AWS to be a mashup between VMware and a payday lender scam. They've been content to put people on stage every six months to peddle this-actually-works happy-talk. Dell tells the story. February 5, 2013: Dell announces plans to go private. May 20, 2013: Dell abandons Openstack.
  - sinxoveretothex 11y ago
    Dell abandoned Openstack in May 2013? Better tell ComputerWorldUK then, they seem out of date by 9 months because they linked me to this story from December 2013: http://www.computerworlduk.com/news/it-vendors/dell-and-red-hat-team-to-sell-enterprise-openstack-3493664/ http://www.computerworlduk.com/news/it-vendors/dell-and-red-...

dankohn1 11y ago

"we have had to host it on physical tin". Is this an English English thing? In America, we would say "host on our own iron". Do other countries use different metals?

drrob 11y ago
I'm English, and I have never used or even heard the phrase "host it on physical tin" before. Then again, I just tend to say "hosted on our own hardware" or just simply "self-hosted".
petewailes 11y ago
British person: I don't think this is one I've come across.
- f3y67j 11y ago
  Another brit: just once, three years ago by a Programme Director determined to eradicate our IT dept "We are not going to feed and water our own tin!". I was equally bemused. However the outcome has been very detrimental to our org.
techterrier 11y ago
probably cornish :D
- jinst8gmi 11y ago
  They do make great pies :)
  - techterrier 11y ago
    don't forget the ice cream!
gaius 11y ago
Yep pretty common in the UK. The chargeback the IT department makes to other groups or projects for kit is called the "tin tax" too.

pastyboy 11y ago

money no object then? Of course not. Safe harbour is a huge problem in EU AWS breaking the rules. The G could find itself in a sticky regulation mess when people start suing for data breeches.

vidarh 11y ago
Money probably no object given that they were using Cisco UCS and Netapp storage to begin with. With that as a starting point, a perceived disaster brewing and what's likely to be fairly steep discounts from AWS to get them to move given their size I'm sure it looked attractive (if you're paying list prices for AWS at any kind of size, you're leaving money on the table; if you're first going to use AWS at least make sure to negotiate).

jakozaur 11y ago

Related read: https://www.packet.net/blog/how-we-failed-at-openstack/ https://www.packet.net/blog/how-we-failed-at-openstack/

vidarh 11y ago

It's a bizarre article. On one hand I understand their frustration with OpenStack. On the other hand, if they expected to pretty much build their own AWS without any warning bells going off all over the place I find it very hard to be sympathetic - they're not remotely large enough for that. If they'd set their goals lower they could have had a very capable system fairly easily. That they're having problems getting load balancing working properly is a real sign that their ops team just didn't know what they're doing.

rrmm 11y ago
if the choice is "let's deploy software on AWS," vs "let's build an AWS work-a-like and then deploy software on it," it'll always be easier to use AWS. Managing physical servers is a pain. I've been in a similar situation in a small research group, but AWS doesn't fly for privacy and regulatory reasons. Managing hardware and infrastructure is a huge amount of overhead that doesn't directly relate to the core mission.
- vidarh 11y ago
  > if the choice is "let's deploy software on AWS," vs "let's build an AWS work-a-like and then deploy software on it," it'll always be easier to use AWS Yes, but that's a false equivalence. You don't need an AWS work-a-like unless you plan on competing with AWS. So much of AWS is dedicated to doing things you don't need to do in most single tenant systems. But if managing hardware and infrastructure is a huge amount of overhead, what in the world are you doing? If it's more than sliding a server into a rack, hooking up network and power, and leaving it to itself until/unless there are hardware failures or SMART alerts, then you've missed out on automation steps somewhere. For server environments I set up, I spend perhaps on average 30m per server in a datacentre per year, including travel. On top of that we estimate on average perhaps 10m of remote hands per server per year.
_Codemonkeyism 11y ago
"they're not remotely large enough for that." Second that. Maybe there are only ten companies in the world large enough to have positive ROI to build an internal AWS, like Google, eBay, Facebook, ... who don't want to depend on Amazon.

hoodoof 11y ago

"That software we've got causes us so many problems, it's disaster! When we get the new software everything will be fixed." X to Y Y to X A to B B to C MongoDB to this MySQL to that Why we switched Why we switched back blah blah It's one of the oldest themes in computers - man, this software is crap - that new software will fix all our problems! Until you find the problems with the new. Precisely the same age is the new software vendor harnessing all that negative energy about the competitor and megaphoning the "man, the new will fix the old, we're so excited!" Having said that, I have worked with the compute servers of all the major cloud vendors and Amazon must be credited for the quality and consistency of its AWS systems, and Google and Microsoft's cloud computing is in almost all respects equally good - there is as far as I can tell absolutely no reason to choose Amazon over Microsoft Azure (yes, even for Linux systems) or Google Compute Engine. In some areas of functionality GCE and Azure I found to work much more easily and smoothly than Amazon. When I went to work with OpenStack I immediately found the lack of completeness, inconsistency and lack of polish characteristic of many open source projects - trying to do simple things instantly became hard problems.

robzyb 11y ago
> "That software we've got causes us so many problems, it's disaster! When we get the new software everything will be fixed." This way of thinking isn't exclusive to software.
- hoodoof 11y ago
  Do you mean life partners? Anyhow its especially prevalent in software. Just search Google for "why we switched from". If anyone is thinking of writing yet another "why we switched from X to Y" blog post, I can save you the effort of a long blog post explaining the reasons. It's because "the old software was shit and the new is fricking awesome!" You don't need to explain the details.
  - robzyb 11y ago
    > Do you mean life partners? I mean everything. Almost literally everything. The most recent thing I can think of is seating arrangements. At my current workplace seating arrangements are mostly random. Someone here had the idea that we should group people by seniority so that people with related issues will be in close proximity. However, at my previous workplace we grouped by seniority, and someone had the idea that we should mix it up so that project teams would be in close proximity. [edit]: And lets not forget outsourcing vs. insourcing, grouping departments by geography vs. grouping departments by business type, etc., etc.

DanielBMarkham 11y ago

You know, there's probably a sweet spot between "AWS all the things!" and "We will build our own cloud" I mean geesh, people have been building small clouds since there were servers. That's the way mom and dad did it, and by gummity it oughta be good enough for you. The "new" stuff was autoscaling, PaaS, and so forth. So build out a few servers for content management and publishing, then write very small amount of code to push what you have out to a CDN. If you want realtime data capture, capture it using AWS (or whatnot) and pull it back locally. I'm not saying that's optimum for every solution, just that the all-or-nothing kind of thinking is probably what lured them into building their own cloud in the first place. You need a cloud for some stuff, so use a cloud. But you don't need a cloud for every freaking thing the company does. Its assets in the form of text content, internal docs, and branding are probably extremely small in modern terms.

smcl 11y ago

There's little substance to this other than "OpenStack is hard, let's go AWS!" - I would have liked to hear where exactly they ran into difficulties and what the problem was in detail. This is for two reasons: 1. "We have switched some components in our software stack" without much info is not an interesting story to read. 2. I have friends who work on OpenStack in RH who would I'm sure be very interested to know the sort of troubles that The Guardian had so if there is a usability or functionality issue it can be addressed.

deleted 11y ago
[deleted]

danieltillett 11y ago

I know cloud everything is the modern way to go, but isn't the Guardian just a glorified blog? Why could they not just push out their content to a series of load balanced static iron severs? Call a cron job every 5 minutes and be done with it.

illumen 11y ago
They are a dating website amongst other things. It's a large billion pound media organisation now with many different units.
- danieltillett 11y ago
  I am not sure why they are running a dating site, but even if they consider this important why one system to rule them all. Hardware is cheap compared to developers.
  - DanBC 11y ago
    > I am not sure why they are running a dating site Many newspapers (at least in UK) did. Guardian is one that stuck around to digital age. It makes them money.
72deluxe 11y ago
Perhaps they wanted a more "exciting" way of doing it?
jsmeaton 11y ago
First thing off the top of my head is tracking engagement etc. Also search? Not many things are simple anymore, especially when you're selling advertising space.

saintfiends 11y ago

It's a bit scary that Amazon AWS is the only complete solution in this space now. There are so many open projects in the cloud space that one would think it's a solved problem. Here's a list off the top of my head: 1. Docker - Container implementation. 2. Kubernets - Manage a cluster of Linux containers. 3. Mesos - Manage a cluster of resources (not just containers, I'm guessing there are some feature overlap with Kubernets). 4. CoreOS - An OS specialized in running containers. 5. RethinkDB, CouchDB, Cassandra - Distributed databases 6. Ceph, GlusterFS - Distributed file systems 7. RabbitMQ, NSQ, NATS - Distributed queue systems. 8. Manage VM's in a Data center ?? - I don't know any projects in this space. What's missing is an interface to manage all these together. Maybe this is the direction OpenStack should be heading?

athrun 11y ago
You are comparing open-source Software against Services. It's as if you were saying that to compete with Amazon.com (retail) you simply have to deploy Magento or Open Cart...
pjc50 11y ago
Many projects do not make a solution in the same way that many grains of sand do not, of themselves, make a sandcastle.
kogepathic 11y ago
> 8. Manage VM's in a Data center ?? - I don't know any projects in this space. 1. Apache CloudStack ( https://cloudstack.apache.org/ https://cloudstack.apache.org/ ) 2. oVirt + Foreman ( http://www.ovirt.org/Home http://www.ovirt.org/Home , http://theforeman.org/ http://theforeman.org/ ) 3. OpenNebula ( http://opennebula.org/ http://opennebula.org/ ) 4. Roll your own with clusterlabs + PXE + puppet

meekins 11y ago

I'd be really interested in the actual technical problems The Guardian had with OpenStack and what's the current situation with the OpenStack project. The company I work for has some interest on the platform but I am not aware what to expect (problems, new features etc) from it in its current state. A couple of years ago I experimented with and compared some open source cloud platforms. Admittedly my knowledge is old, but alredy then I was wondering the hype and visibility of the OpenStack project. While it has many big-name supporters that guarantee its lucrativeness for the enterprise, its feature set and flexibility was seriously behind other open source alternatives. Back then I fell for OpenNebula because even though it had its warts it already delivered many features (especially concerning hybrid clouds and heterogenous virutalization environments) that OpenStack still had on its future roadmap.

rkeene2 11y ago
I also tried OpenStack for a self-assembling cloud product I built. It was a huge pain and I also switched to OpenNebula about a year ago. It's been significantly better !

Angostura 11y ago

Stephen Gran, the senior integrator charged with getting OpenStack to work and who wrote fulsomely about it left the Guardian in March 2014 according to LinkedIn.

jackgavigan 11y ago

I've always been introgued by the Guardian's focus on building its own technology. It seems a "bold" strategy, given that tech is not its core business and the newspaper part of the group (which is pretty much all that remains now, since they've sold off their other assets) has been consistently losing money for some time now. I wonder if that strategy would still be in place if it wasn't owned by a not-for-profit trust. Other publishers who have gone down the "We're a tech company!" route have been forced to give up on that strategy[1]. I wonder whether the Guardian will be able to make that breakthrough, or whether they'll end up migrating to Wordpress or something similar. 1: http://digiday.com/publishers/gawkers-kinja-retreat-shows-false-hope-in-publishers-licensing-tech/ http://digiday.com/publishers/gawkers-kinja-retreat-shows-fa...

gaius 11y ago
They have a few very charismatic people in tech there, the kind of people who are very good showmen and salesmen (or women) but they've been "architects" or similar their whole careers, not hands-dirty engineers, and their grasp of what's feasible or not is driven mainly by which vendor buys them the nicest lunch or has the nicest typography in their whitepapers.
- jackgavigan 11y ago
  Do you have direct experience of working there?
  - gaius 11y ago
    Back in the day as an expensive consultant, yes, and I know a few people there now. If you want to see a newspaper who are really doing something interesting with technology, check out the Daily Mail, last I heard they were going all-in on Scala. Newspapers that are still in print are fascinating IT places - because come hell or high water, the paper has to come out the next morning. They are the definition of mission critical. Websites are easy in comparison.
    - 11y ago

trolla 11y ago

I’ve used Openstack at my current $dayjob and I’d say that Openstack is probably one of the buggiest solutions that I’ve ever used. Even simple things like stopping or resizing VM-s sometimes failed completely and the VM came unusable after that. Feature wise it’s also lacking. Seems that Openstack spends more time/money on marketing and generating hype than creating a good solution. I’d even say that Docker has a somewhat similar problems, they spend more on hype/marketing than creating a stable and usable solution. However, at least Docker is more usable right now than a year ago, the same can’t be said about Openstack.

mkulke 11y ago
I guess those expectations are part of the problem, a talk in the recent openstack conf higlighted this: OpenStack was initially created for what these days is called "cloud-native" workloads, every VM was considered ephemeral. companies and users then try to mould it into a cheaper VMWare and are frustrated how bad it is at this. Resizing, stopping VMs, while admittedly being a rather trivial tasks, point to this usage of OpenStack. When I would mourn the loss of individual VMs on OpenStack (or public clouds for that matter), I would turn gray soon.

deleted 11y ago

[deleted]