5 ms·

Disclaimer: I worked at CERN in IT for 5 years as staff. CERN has never been a Microsoft organisation excepted in the head of management that was pushing for M

Disclaimer: I worked at CERN in IT for 5 years as staff. CERN has never been a Microsoft organisation excepted in the head of management that was pushing for Microsoft solutions everywhere without success. - Scientific computation are done at CERN under Linux with the Root framework. - Most (all?) scientists uses OSx or Linux. - All computing clusters runs SC Linux or centos. - Most internal softwares: indico, EDH, landb, and other are running under Linux and are web based. - All DBs are OSS or oracle - All storages system are home made (EOS, Castor) or ceph based and run under Linux. - Data distribution is home made and based on a framework named xrootd under Linux. - Software distribution is also Linux based and run as a fuse module (cernVMFS) - Most systems services are UNIX C++ and Java for the control part. - CERN uses Openstack for virtualisation After the management pushed for Hyper-V and failed miserably. - Management pushed for SharePoint for years before the entire website switched to PHP and Drupal. There si not real "Microsoft" at CERN excepted AD and phones. It is however a study case of bad management decisions in IT.

chipperyman573 7y ago
Why was management pushing for it so hard? Better (existent) salespeople for microsoft vs OSS? Or were there actual reasons they thought microsoft would be better
- cernguy 7y ago
  > Why was management pushing for it so hard? Better (existent) salespeople for microsoft vs OSS? It's not clear to me. But it seems to have been part of a vision to "servicify" everything. Where everything should be a service "maintained" / "paid" but never "developed" to reduce costs. Something that backfired beautifully
  - kfk 7y ago
    It amuses me IT management is ok to pay millions in terrible software license fees (i.e. most of Oracle products), but not ok to budget a couple of millions to have a team develop some of that internally. Also another amusing fact is that most IT managers never wrote a line of code in their lives. IT should not just be about expensive vendors and outsourcing for labor cost arbitrage.
    - SXX 7y ago
      If terrible software fail it's fault of vendor and blame goes to said software and vendor. If internal development team fail to deliver needed software it's always fail of IT manager who can be fired. It's all that simple.
      - kakwa_ 7y ago
        Also it's the classical buy vs build. Ok, given the size and budget of CERN, it's probably cheaper for them to develop things in house. But maintaining a team of people not at the heart of your activities is always a bit challenging from an organizational point of view. For example, as it's not core for your activities, in case of a budget reduction, this team will likely be reduced, re-purposed or removed, leaving a lot of their services unmaintained with no bug fixes and difficult to operate by IT. It's also less battle tested than off the shelf software, so you will hit all the common traps before having a stable solution. And open-sourcing the code is not a magic bullet. Even if the code is open-sourced, the likelihood of the project building a community of users and external contributors is quite small. That being said, sometimes the offering for off the shelf software is just so bad that it's preferable to implement it in house. Or you have an idea that, even if not core to your business/activities, could provide a competitive advantage/huge gain in efficiency. But to pull that off, you need to be well ahead of the curve, have smart people in-house and significant resources to dedicated to that (CERN actually crosses these boxes). But it's also likely the world will catch-up and you will then be stuck with your in-house solution lacking features and reliability.
        speedplane 7y ago
        > given the size and budget of CERN, it's probably cheaper for them to develop things in house It's not just their size, it's that they're doing something that no one has done before. When you're building a me too product or even a version 2 of something, maybe buying makes sense. But when you're doing something for the first time, and you're not even sure what will work (e.g., CERN), by definition it's impossible to buy. If you could buy it, someone's already built it. Generally, the more uncertainty around the business model and potential solutions, the stronger the case for building. Conversely, sometimes the larger and more bureaucratic the institution, leads to slower iterations and experiments. Because of these two seemingly contradictory points, when a large company wants to do something unknown and relatively uncertain, they buy an entire company, provide them resources, but try no to interfere too much (see, e.g., GM buying Cruise).
  - tinus_hn 7y ago
    Consultants often come in with papers that prove that these solutions save money. It just never happens and you end up with expensive, barely working services like SharePoint.
deleted 7y ago
[deleted]
l0b0 7y ago
I'm not proud of it, but I worked on SharePoint at CERN IT. Some examples of how dysfunctional it was: - When asking about version control I was told that if I really wanted to use it I could have Visual SourceSafe 6.0 or something from way back in the nineties. It was basically CVS 0.1 alpha with a GUI, and I ended up learning Git instead. - I made sure all my changes worked on Firefox, because that's what I knew the physicists were using. My boss wanted me to support IE only, and after a heated discussion the quote which always haunted me was "We don't care about the physicists!" - A colleague lamented about having spent about a year developing some uber-flexible interface in SharePoint which would've been an order of magnitude simpler in other content management systems. The rest of CERN is completely different, as indicated by cernguy. I left a year into a two-year contract, and I'm happy to say I've not worked on a Microsoft platform ever since!
- cernguy 7y ago
  > The rest of CERN is completely different, as indicated by cernguy Yes, that is also a thing at CERN. Many other teams re-develop things internally (Linux based) because they could not trust or rely on IT management to do the right things. That had a very perveted effect of internal duplication across teams and experiments.
- wilsonpie 7y ago
  Well, now they are conCERNed
anthony_barker 7y ago
How widespread is oracle? What features are they using that Postgresql(or commercial derivative) couldn't handle?
rkeene2 7y ago
cernVMFS sounds like AppFS [0] with fewer features. Some of those features, like writability or setuid files, might be useful for CERN users. [0] http://AppFS.net/ http://AppFS.net/

chipperyman573 7y ago

Why was management pushing for it so hard? Better (existent) salespeople for microsoft vs OSS? Or were there actual reasons they thought microsoft would be better

cernguy 7y ago
> Why was management pushing for it so hard? Better (existent) salespeople for microsoft vs OSS? It's not clear to me. But it seems to have been part of a vision to "servicify" everything. Where everything should be a service "maintained" / "paid" but never "developed" to reduce costs. Something that backfired beautifully
- kfk 7y ago
  It amuses me IT management is ok to pay millions in terrible software license fees (i.e. most of Oracle products), but not ok to budget a couple of millions to have a team develop some of that internally. Also another amusing fact is that most IT managers never wrote a line of code in their lives. IT should not just be about expensive vendors and outsourcing for labor cost arbitrage.
  - SXX 7y ago
    If terrible software fail it's fault of vendor and blame goes to said software and vendor. If internal development team fail to deliver needed software it's always fail of IT manager who can be fired. It's all that simple.
    - kakwa_ 7y ago
      Also it's the classical buy vs build. Ok, given the size and budget of CERN, it's probably cheaper for them to develop things in house. But maintaining a team of people not at the heart of your activities is always a bit challenging from an organizational point of view. For example, as it's not core for your activities, in case of a budget reduction, this team will likely be reduced, re-purposed or removed, leaving a lot of their services unmaintained with no bug fixes and difficult to operate by IT. It's also less battle tested than off the shelf software, so you will hit all the common traps before having a stable solution. And open-sourcing the code is not a magic bullet. Even if the code is open-sourced, the likelihood of the project building a community of users and external contributors is quite small. That being said, sometimes the offering for off the shelf software is just so bad that it's preferable to implement it in house. Or you have an idea that, even if not core to your business/activities, could provide a competitive advantage/huge gain in efficiency. But to pull that off, you need to be well ahead of the curve, have smart people in-house and significant resources to dedicated to that (CERN actually crosses these boxes). But it's also likely the world will catch-up and you will then be stuck with your in-house solution lacking features and reliability.
      - speedplane 7y ago
        > given the size and budget of CERN, it's probably cheaper for them to develop things in house It's not just their size, it's that they're doing something that no one has done before. When you're building a me too product or even a version 2 of something, maybe buying makes sense. But when you're doing something for the first time, and you're not even sure what will work (e.g., CERN), by definition it's impossible to buy. If you could buy it, someone's already built it. Generally, the more uncertainty around the business model and potential solutions, the stronger the case for building. Conversely, sometimes the larger and more bureaucratic the institution, leads to slower iterations and experiments. Because of these two seemingly contradictory points, when a large company wants to do something unknown and relatively uncertain, they buy an entire company, provide them resources, but try no to interfere too much (see, e.g., GM buying Cruise).
- tinus_hn 7y ago
  Consultants often come in with papers that prove that these solutions save money. It just never happens and you end up with expensive, barely working services like SharePoint.

deleted 7y ago

[deleted]

l0b0 7y ago

I'm not proud of it, but I worked on SharePoint at CERN IT. Some examples of how dysfunctional it was: - When asking about version control I was told that if I really wanted to use it I could have Visual SourceSafe 6.0 or something from way back in the nineties. It was basically CVS 0.1 alpha with a GUI, and I ended up learning Git instead. - I made sure all my changes worked on Firefox, because that's what I knew the physicists were using. My boss wanted me to support IE only, and after a heated discussion the quote which always haunted me was "We don't care about the physicists!" - A colleague lamented about having spent about a year developing some uber-flexible interface in SharePoint which would've been an order of magnitude simpler in other content management systems. The rest of CERN is completely different, as indicated by cernguy. I left a year into a two-year contract, and I'm happy to say I've not worked on a Microsoft platform ever since!

cernguy 7y ago
> The rest of CERN is completely different, as indicated by cernguy Yes, that is also a thing at CERN. Many other teams re-develop things internally (Linux based) because they could not trust or rely on IT management to do the right things. That had a very perveted effect of internal duplication across teams and experiments.
wilsonpie 7y ago
Well, now they are conCERNed

anthony_barker 7y ago

How widespread is oracle? What features are they using that Postgresql(or commercial derivative) couldn't handle?

rkeene2 7y ago

cernVMFS sounds like AppFS [0] with fewer features. Some of those features, like writability or setuid files, might be useful for CERN users. [0] http://AppFS.net/ http://AppFS.net/