7 ms·
When I consulted at a health insurance company (a Blue Cross Blue Shield licensee with 1000 employees, 200 IT staff) about 10 years ago, I was absolutely horrif
by GiorgioG 9y ago
When I consulted at a health insurance company (a Blue Cross Blue Shield licensee with 1000 employees, 200 IT staff) about 10 years ago, I was absolutely horrified that the system of record for health care plan data was Excel spreadsheets. They had staff who would update the claims processing system based on thousands of spreadsheets. When you called customer service, the CSR app would auto-open the member's plan information XLS file to the CSR rep. It got the job done - but it was the wrong tool for the job.
We recommended they implement a PDM (product data management) system - it would have saved them millions of dollars (considering they had a staff of 20 whose sole job was to update the claims processing system by hand based on these spreadsheets.) Unfortunately when the 2008 financial crisis happened, management balked at the initial outlay. It's quite possible they're still using Excel to this day.
- dalbasal 9y agoThere are upsides to excel in cases like that… The health insurance people of various sorts who use this setup are the ones with domain knowledge. With an excel program, they can (1) understand how it works (2) add to it, modify it (3) create a copy and break it (4) debug it (5) have ideas about how it could work differently… I’m not saying that excel is the tool for the job (sounds like it is definitely not), and those advantages erode as the setup gets more complex and/or has more people using it. But, there are reasons excel gets used this way. The alternative is often rough. Software projects like this have high failure rates. Getting “requirements” out of users’/clients’ heads and into programmer’s is difficult. It means the standard (natural?) gap between domain knowledge and programming knowledge is restored. That has costs. I’m going to repeat again that I am not advocating health insurance companies run policies and claims in excel. It is a bad idea, almost certainly. I am saying that we should be curious at why they are doing it this way. I think it’s a “good” question in that it leads to interesting places. If the excel team was blessed with a plague of inspiration tomorrow morning, Is there a way they could make it so that excel is the right way to manage an insurance business?
- GiorgioG 9y ago> The alternative is often rough. Software projects like this have high failure rates. Getting “requirements” out of users’/clients’ heads and into programmer’s is difficult. It means the standard (natural?) gap between domain knowledge and programming knowledge is restored. That has costs. The alternative doesn't have to have a high failure rate. I was brought in to help reboot their in-house implementation of a product data management system - the core issue was they didn't have any real competency in software development (though they believed they did.) Ultimately I recommended they abandon the in-house effort and instead proposed using an off-the-shelf PDM system, we evaluated quite a few decent options over the course of six months and found several that met the business's requirements. It would have been the right thing to do and I believe it would have been successful. Culturally the IT department staff was plagued with not-invented-here syndrome. It was a generally dysfunctional IT group and it's quite likely that Excel got it's start as the system of record because the business needed to get the job done, and the IT department was unable or unwilling to deliver.
- gjjrfcbugxbhf 9y agoAlternatively you misunderstood the business requirements and would have cost then a fortune on a failed project. Alternatively you were brought in by management to make cost cutting recommendations - and your solution wasn't cheap enough. Alternatively the company accounted for the enormous risk of the project succeeding but actually being worse than what they had in place. You really don't know the politics of this. Edit: Another alternative is that your plan would have resulted in sufficient job losses that the company was worried that it would demotivate other employees and damage the company more than the cost savings.
- rattray 9y agoI upvoted your post because I think those alternatives are interesting to consider, but _you_ know far less about the politics of the situation than the OP. It's entirely possible that the OP was exactly correct, or close to correct. It's also quite likely that at least one of the factors you mentioned (or others) was also at play.
- oblio 9y agoOn top of what you're saying, the alternative is frequently a heap of under specified, under tested (and only manually tested!), copy-pasted and shoddy quality Cobol/Java/Oracle/C#/SQLServer/PHP/MySQL code. Yes, the performance will generally be better (1 minute to run a report, not 1 hour), but I'm not sure things will generally be better. As you said, the business people can at least modify the Excel. These systems? Not so much, without an usually massive support contract (Yaaaay! for us IT people, I guess). People sometimes make a false equivalent and imagine that the Excel jungle will be replaced by a Google-developed, 90%+ unit tested, refactored and cleaned periodically bunch of code.
- deleted 9y ago[deleted]
- fzumstein 9y agoI agree, the issue is not Excel as such but how it's being handled: I once wrote a blog post on how you can unit test Excel spreadsheets, too: https://www.zoomeranalytics.com/blog/unittests-for-microsoft-excel https://www.zoomeranalytics.com/blog/unittests-for-microsoft...
- Sean1708 9y ago> the business people can at least modify the Excel I'm not convinced that's a good thing. If they can change the Excel then they can also fuck up the system, and if you fuck up a system like this then it's horrendously difficult to unfuck. I think systems like this look great when they're working, but the second they go tits up you realise why people are horrified by them.
- EamonnMR 9y agoMake it easy to interoperate with databases and/or revision control painlessly for the end user and the developer. The users are fine with excel, it's the attendant issues like centralizing data, running automated services off of the data, backing it up, and keeping an audit trail that created pain when I worked in healthcare.
- dalbasal 9y agoIf excel online was better, a lot of that would be pretty doable.good points. But..I do think that if we're thinking in terms of "users" and "developers" then we're no longer talking about excel. The whole reason excel is used this way is the continuous spectrum of sophistication.
- osullivj 9y agoAlso because Excel is both an IDE for developing spreadsheets, and a GUI container for operating them. Splitting dev time and run time environments would help with control. When a dev supplies a user with a traditional .Net GUI app, they don't enable the user to hack the app by supplying a copy of VisualStudio alongside. Effectively, this is what Excel does.
- dalbasal 9y agoYes, but. It's no good if you fix excel by making it not excel. The whole reason excel is so popular is the intuitive, unintimidating but terribly wrong way it mixes data, code and GUI. Functions in the data, constants repeated in every row, hard coded variables all over the place. If you're just supplying a GUI and hiding the code, the user can't change, add or debug anymore. The point is that they can code, even if they don't know they are coding. The trick is to think up a way of shipping VisualStudio, but making it less terrifying to engineers, managers and such.
- osullivj 9y agoAgreed. But a different way of working the same trick is to have an alternate run time that can execute .xls[x|m] as a headless server process.
- otakucode 9y ago>If the excel team was blessed with a plague of inspiration tomorrow morning, Is there a way they could make it so that excel is the right way to manage an insurance business? No... but I think the converse might be possible. If those with the programming knowledge sought to replace the Excel system but in a way which actually preserved the useful features of the Excel solution. Dirty, filthy things that programmers don't like. No referential integrity, data duplication, deferring decisions to human beings rather than lists of rigid criteria, etc. If they could swallow their bile for just a bit, they could build a system just as useful and flexible and introduce tools to add in the things like data consistency and referential integrity - tools that the user would initiate, thus accepting responsibility for the restrictions such things impose on themselves and the data. That is not, unfortunately, how it is usually done. (I am currently literally working on a program to replace a 40K row Excel sheet that crashes multiple times a day which has a workflow centered around it that involves slicing and dicing the sheet, passing it off to multiple different groups for separate editing, etc... not sure I will be able to sell the team on this approach, however I do believe it is sound.)
- jimbokun 9y ago"If the excel team was blessed with a plague of inspiration tomorrow morning, Is there a way they could make it so that excel is the right way to manage an insurance business?" What a professional software developer's workflow has and Excel lacks: * principled way of sharing code across a team with distributed source control (or similarly powerful tool), * automated regression tests, * input validation, * continuous deployment tools to automatically deploy working code (default mode for Excel I guess), and a mechanism to roll back to a previous version (not so much), * build and dependency management tools for bringing in libraries from a vast open source ecosystem, * systems for reporting bugs and feature requests and tracking their progress, * networked database that is the shared "source of truth" for all users, * security and auditing, * hardware capacity planning (run locally or in AWS? what are the cost trade offs?) So sure, get the health insurance people doing all of those things, and have Microsoft figure out how to do all of them well in Excel, and congratulations, they are now software developers. None of these require using a "real programming language", at least not directly.
- anigbrowl 9y agoThat's great (though I'd quibble about a few of them, eg I was doing input validation in Excel 25 years ago, and boy do I feel old now), but while you're learning to do all that great stuff you're not learning anything about the domain you want to work in. OK, I don't know who wants to work in health insurance but the basic point is Excel is a good tool for people who have specialist knowledge in a particular domain and want to automate their work. Programmers know the best way of programming but frequently overestimate their understanding of other people's problems.
- Yhippa 9y agoIntegrated version control is one of the biggest things I wish they included in Excel rather than relying on an external system to do it.
- watwut 9y agoYou underestimate how difficult it is to work with at least some software companies, how expensive it is, how difficult negotiation it takes to convince quite a few nerds that user actually has business case for that requirement. And many software companies produce poor quality and are very slower then necessary despite using all buzzwords and all nighters. Managing even outsourced projects takes learning and expensive failures are nor an exception. All in all, the software we all collectively produce is still very expensive and unreliable. Even absent that, it is difficult to explain what users need to coders who have zero idea about the domain. Unless it turns out they are the kind that just know user is idiot the moment she has different opinion on her own job, in which case expect needed time to tripple. The homegrown excel is awesome analysis once the company decides to move on. It is concrete, you can generate test cases from it, it has features.
- Yhippa 9y agoOne other point is let's say you do create some fancy new system to handle the task. It's likely going to erode and be surpassed by new computing technology at which point you're looking at an expensive legacy migration which could prove to be costly and you have the same downsides you mentioned. Excel upgrades don't seem to break previous files as severely and have the backing of Microsoft. I can't believe I'm defending Excel.
- nickbauman 9y ago| I am saying that we should be curious at why they are doing it this way. If you hand a caveman a machine gun, he will club you to death with it. IOW, the casual ability for domain experts to screw up a spreadsheet is enough of a reason to decide that excel is a terrible solution to this problem.
- dalbasal 9y agoI would make a different caveman analogy. Hand a caveman a gun. He probably won't be a better rifleman than you (or someone). Really bad form and technique. But, he'll probably be a way better hunter than you, shoddy rifle technique or not. Caveman were better than you at a lot of things. They knew how to find squirrels
- jandrese 9y agoThe problem is switching would mean transitioning to a generalized system that they would have to work around. The Excel system looks like a nightmare to the HN crowd, but it was built around their business processes. It's basically a boutique app designed specifically for their job, it would be very hard to find a general product that does the job as well. I've seen this a lot where a consultant will swoop in and tell them that they need to use Peoplesoft or something and the organization spends millions of dollars and years of retraining to get a system that has a lot more friction in their actual workflow and is never quite as good for getting the job done. Of course there are often side benefits that can't be overlooked, like a much more sane backup solution, interoperability with other providers, etc... But in the end most of the employees will tell you straight up that the old system was better. Too much of the domain knowledge was baked into the old system and doesn't make the transition to the new system.
- Brotkrumen 9y ago>When I consulted [...] We recommended they implement a PDM (product data management) system. >We recommended they get a system that manages their product data Yep, I believe you are a consultant.
- mabbo 9y agoThey should be careful though: excel allows a maximum of 1,000,000 rows. I have seen more than one major crisis occur when a spreadsheet ran out of space, yet all of it was needed together to do daily business.
- will_pseudonym 9y ago1,048,576 rows to be precise. ;) Though you'll run into performance issues in more complex spreadsheets once you get into the 100,000 row area, or even 10,000 or so. Then you'll see power user departments switch to the more opaque but more performant Access.
- CabSauce 9y agoThey switch to Access until they hit the data size limit of 2GB. Then they come back to IT who has to deal with the mess that you get when you allow domain experts to build databases.
- anigbrowl 9y agoDomain experts are not the best people to build databases (says a domain expert who's failing miserably at that task right now), but database experts are sort of terrible at understanding domain problems. It seems to me that many programmers don't appreciate that industry practices or regulatory requirements impose quirky requirements on domain specialists that everyone knows are far from optimal but which need to be complied with anyway if the business is to turn a profit or retain its licensure. Someone mentioned a health insurance example above: there's 50 states, with 50 different sets of insurance regulations, and then some additional territories, federal regulations, and then wird business practices of healthcare providers who are subject to their own 57 varieties of regulation, and there are powerful economic and political interests that get in the way of burning it all down and replacing it with a single system designed by a committee of one patient, one doctor, one nurse, one pharmacist, one lawyer, and one economist. And even that highly simplistic requirements committee went through 3 versions while I was writing it up. The domain experts already have lots of complex knowledge and experience they need to automate. They don't want to learn about databases. Unfortunately, the reverse is equally true.
- ataturk 9y agoI worked at a major PBM and saw the same thing--stupid mess of Excel spreadsheets and Access databases. Major systems still on AS/400 with file drop off as the mechanism for communications instead of queues. Oh and one major system still running on a DEC Alpha OpenVMS system that required replacement hardware purchases on the gray market used equipment side.
- sanderjd 9y agoYeah, updating these sorts of insurance systems was a huge line of work for Rails devs in the late '00s.