8 ms·

The reason that both Access and Excel use is so prevalent in corporate “shadow IT” land is because there are many parts of the business that have problems for w

The reason that both Access and Excel use is so prevalent in corporate “shadow IT” land is because there are many parts of the business that have problems for which only a negative or marginal business-case can be made for IT to solve it (given the “get out of bed” costs of most IT departments). It’s a barrier-to-entry problem. Excel and Access are cheap enough and fly under the corporate IT radar (no involvement needed), that these marginal problems can be addressed in a self-serve manner. I’ve worked in organisations that have tried to kill these tools off but unless you can lower the cost-to-play or offer a “better” self/serve alternative, you will fail.

asdfman123 7y ago
Access is the perfect tool for an intelligent, technically-minded person with limited programming experience to create an application to replace spreadsheets. There are a lot of those kinds of people out there, and they're extremely useful in introducing minor optimizations that other people wouldn't be able to find. Access is for that guy who says "I know there's a better way to do this," but doesn't have access (no pun intended) to a team of programmers and a project manager. I didn't major in programming in undergrad but I've taken classes here and there, so in my first job out of college I replaced a really awful system of spreadsheet-jockeying with an Access DB. I considered other options, but that it's self contained and NOT a web application is a feature, not a bug. I couldn't get access to the corporate database, so I just ran the Access DB on a network drive. It's still probably there, ten years later, running happily on its own. Honestly, it seems like the fix for Access... is a better version of Access. It fills a very useful niche, between spreadsheets and full-fledged applications designed by programmers. It's so much easier to make a quick app that works for your organization than get an external team of programmers involved, who will probably tell you "no" or remain unconvinced that you're worth helping. With Access, you don't need political clout, you don't need years of experience, you don't need a title. Just build an Access application and get kudos from everyone in your team for making their lives easier.
- dsaavy 7y ago
  These are excellent points, I think it’s underestimated how difficult it is to get approval and budget for a small project that doesn’t directly contribute to the bottom line or have huge savings. These small but impactful solutions using Access act as both a stepping stone to greater skill sets, a way to build POCs for non-tech or low-tech people, and to avoid months of politics in big organizations. I’ve had projects that were red-taped before getting off the ground due to resources being placed elsewhere. I then take those project ideas, and instead build them in Access as a POC using shared drives, splitting the database, etc. By the time I have 100 users saying how useful the application is in a couple months, other bigger projects haven’t even finished a project plan. Meanwhile we’re then ready to scale the solution using an appropriate stack and basically can just say “replicate our POC features but improve performance, security, accessibility, etc”. So far that’s worked pretty well for me. In summary: find people’s Excel files with a mess of VBA and formulas —> see if the use case should be expanded —> build POC in Access without permission/budget from a bunch of people —> see how it goes and then plan to scale with the evidence you’ve gathered from your POC.
- dbrgn 7y ago
  I taught myself Access 97 when I was about 10 years old and helped a teacher set up a small database to register library books. I couldn't have done that with a real SQL database, but Access was somewhat discoverable. Never underestimate the empowering power of tools that can be learned without any external help! I hate Access, but I love it too.
- ebiester 7y ago
  I know you say that being something other than a web application is a feature, but we have Microsoft's replacement for access in Office 365 and it's PowerApps combined with Flow. I haven't been able to dive too deeply, and it definitely has a learning curve, but so did access. Further, data is actually backed up and permissions can be applied. I also have fondness for Access, but I think that we have a chance for something like that again.
  - TeMPOraL 7y ago
    O365 still gives you desktop versions of the Office software. It does makes me a little worried, but I'm hoping Microsoft isn't suicidal and won't pull the plug on desktop software, leaving us only with the half-featured web versions. Flow is interesting, but AFAIR it requires extra clicking in the administrative interface to get running, so it might not work for shadow IT as well as Access does.
    - ebiester 7y ago
      I am hopeful that between MS's financial commitment to the suite, improved JS performance in the browsers, and a set of practices coming around, it will reach parity in the next few years. I run Linux and while I miss features, I can do my job in an O365 org rather than having to have a mac or run a windows client for office suite.
      - TeMPOraL 7y ago
        On the one hand, I'd like it too. I use Linux as a daily driver - but I recently replaced my sidearm machine, and I keep it on Windows, so that I have a computer on which I can run desktop Office. On the other hand, I'm convinced that once Office on-line reaches feature parity with the desktop, they'll quickly drop files, and turn Office into just another SaaS that holds my data for ransom. I don't want that.
- qubex 7y ago
  I couldn’t agree with you more. I’m currently involved in a small startup where loads of administrative, transactional data is contained and manipulated by hand in an ever-growing Excel spreadsheet. I’m a Mac user but there’s plenty of Windows machines with Office lying around, and my first gut instinct was to start throwing something together and importing the Excel file to populate the DB, and construct ‘robust’(er) queries & scripts for interacting with and getting reports from the data. Perfect? No way. But within a couple of days I was able to get something working more reliably and (albeit temporarily) “shored up” a major calamity waiting to happen. Am I proud of my work? Absolutely not. But it was quick, cheap, and cheerful. Eventually when urgency has lessened and there’s more budget (in terms of mind-space, time, money) we can start looking for developers that will take my kludge and make it into something more bullet-proof. For now, the leap from Excel to Access has been a huge leap forward (the kind that the physically illiterate offhandedly describe as a “quantum loop”, not realising that quantum leaps, though discrete in nature, are tiny).
dexwiz 7y ago
I think Developers also underestimate IT Setup Costs for a DB since its so easy to do it locally. Setting up a simple RDBMS like MySql or Postgres and skinning with with a basic, out of the box UI is an afternoon of work if you need to look up commands, or minutes if you have done it recently. But setting up something for a Team immediately opens questions like Access Rights (Can we limit this just to the team, or even have profiles?), Security (Needs to plan nice on the VPN/intranet), Scalability (in theory easy, but provisioning/managing VMs is hard), and Recoverability (where are the backups stored and who makes them). Those are all hard problems from not just a technical perspective, but also a business one. But that is just the technical hurdles, not the process ones. In a well (overly?) managed system it can take a handful of people multiple meetings and possibly hundred of man hours to add a new field to a form. Adding the field is trivial, but deciding all the business rules around it is hard. If you run your own system, you can ignore any established process. So its two fold. Its not just IT start up costs, its also getting everyone to agree on changes to an existing system, versus running your own. I would argue many of the costs associated with migrating from Access to a "real" DB, is not the actual migration script, but the reassessment of all the business decisions that went into the design of the Access schema. Usually a committee evaluates if they are valid, and what to do with existing data to make it conform to a new schema.
- asdfman123 7y ago
  I've worked in multiple organizations that run small Access apps here and there and for us the main cost is just sitting down and doing it. My last job was at an HR company and things were historically done through Access and spreadsheets, until an IT guy came along and started writing C# web apps, rose in prominence and now runs a small internal software team. I worked for that guy. It's not terribly politically difficult to convert existing applications, but it's mind numbing work and it's bottom priority. There's not a huge need to change something that already works, and the only reason you do it is you know it can't stay like that forever.
- benhurmarcel 7y ago
  Plus, you need a server (ie. obtain one, set it up, and maintain it). With Access you just host it on the team's shared drive. In a large organization, getting a small server when you don't work in IT is not simple at all.
mamcx 7y ago
My dream is to build a modern take of the fox/dbase family. I starting with the inner language http://tablam.org http://tablam.org Access, as concept, is great. Is exactly what many need. But it have a lot of small deficiencies that make it disliked by developers. I have used FoxPro, and it have almost all the right things instead (Fox is Access, but goooood!... except a bit less user friendly). I wish I could dedicate to build this. Is weird to me why this kind of tool have so little support?
- taffer 7y ago
  I really like the idea of developing a modern Access alternative, but why are you trying to create a new language? Aren't QBE and SQL sufficient? Why not something that is like a cloud SaaS version of Access, but can be extended with some procedures and later used as a GraphQL backend like Hasura?
  - codeulike 7y ago
    Salesforce is (among other things) basically a cloud SaaS version of Access. And people pay a lot of money for it. See my other comment https://news.ycombinator.com/item?id=21403393 https://news.ycombinator.com/item?id=21403393
    - taffer 7y ago
      True, when I think about what I've seen, what people are building with Access and what people are building with Salesforce, there's a lot of overlap.
  - mamcx 7y ago
    SQL/GraphQL are query languages. Good for that, but not for fully develop an app. FoxPro have his own language that allow mix SQL and imperative constructs: https://en.wikipedia.org/wiki/Visual_FoxPro https://en.wikipedia.org/wiki/Visual_FoxPro This mean: * You code login in foxpro * And the stored procedures * And the forms * And the reports * And the script glues * .... "Just" adding SQL/GraphQL to a RDBMS is absolutely not enough. Both are too limited. For example: - Can you build a btree with SQL? No - Work with the terminal? No - Make a visual grid? No - etc Only query and maybe crud.
radcon 7y ago
I don't disagree, but isn't it a sign of much deeper issues if this "Shadow IT" land exists at your company in the first place? Why is it that experienced programmers can't efficiently solve a problem that a business user can handle in Access? Are they under-staffed? Neck-deep in spaghetti code? Lacking business knowledge and the opportunity to acquire it? Bottle-necked by a lack of business analysts or testers? I hear about this too often and nobody seems to think it's a major problem. IT departments can have a queue that's 2 years deep and people don't even bat an eye, they just think "Oh, that's how IT works!" And that's not even taking into account all the requests people aren't submitting because they've given up hope of getting any dev time.
- flukus 7y ago
  Agreed, I think it's because developers like to over engineer everything and the managers like to over engineer the processes. So many of these access excel solutions should be a days worth of work, simple perl cgi scripts with a minimalist UI deployed by rsync. Instead we have to use our super "productive" modern frameworks, split everything into a thousand files (god forbid you embed an sql query in the only place it's called), add unit tests, etc. There's certainly times for the later approach, but most businesses need much more of the former. So because developers don't have a reasonable platform to pinch off random little projects others step in.
  - codeulike 7y ago
    god forbid you embed an sql query in the only place it's called -gasps-
  - perl4ever 7y ago
    "So many of these access excel solutions should be a days worth of work" I don't know why you think a lot of VBA would be replaced by a little perl. Of course, my perspective has a lot to do with the fact that it was essentially impossible to get a new perl module installed where I used to work. But developers often don't appreciate the importance of presentation (and other) details in reports for managers. Your reference to "a minimalist UI" is telling. Even though Access and Excel can be buggy, unstable, and annoying, it doesn't make much sense to use anything else if you are automating a report that was previously assembled by hand in Excel, and needs to match precisely. And often IT types like to exercise power by gatekeeping - if you aren't doing "real" programming, you don't need a Turing complete solution, so Office ends up being the only option. I've been told that if I can select a list of columns from a dataset, and some filters, by pointing and clicking, that's all I, or my managers, need for reports. Honestly, I think a lot of people find fulfillment in their work through being the person who can say "no" to people, particularly managers that are theoretically higher ranking. And also by expressing themselves through creative decisions when others fail to specify details. I think that using Office/Access/VBA may be correlated to rejecting the value system of most developers, rather than a technical judgment.
cryptonector 7y ago
Everything needs to be self-service as much as possible. Need a SQL Server / PostgreSQL DB for your project, got click some buttons on a page and get back a URI for your database. Automate, automate, automate.
- TeMPOraL 7y ago
  That goes against typical IT policy that if people don't use something they can't break it, so let's make sure they can't use shared IT resources.
codeulike 7y ago
Yeah exactly. Working in an IT Dept when someone said "ah they've built a spreadsheet to do it, but now they want us to add that functionality to the main DB" we'd all roll our eyes because it was another lashed-together thing that we would have to unpick and re-implement. But the fact is, if they'd come straight to us and asked us to do it instead of doing it in Excel first, they would have had nothing, for years and years.
- dageshi 7y ago
  Also you can consider the spreadsheet versions as a nearly feature complete & tested specification.
- Ididntdothis 7y ago
  “But the fact is, if they'd come straight to us and asked us to do it instead of doing it in Excel first, they would have had nothing, for years and years.” Very true. As soon as us professionals take over the dev speed drops drastically and costs go up.
  - Spooky23 7y ago
    I killed a project a few years ago that had spent about $8M on some project to implement a pretty straightforward business process. No end was in sight. The catalyst was Access 2003 going end of life, and this little app doesn't work. I don't know why one of a 3 BAs couldn't figure out WTF was happening and fixing it. We had an intern futz around with Access for a few weeks and it's been running fine to this day.
specialist 7y ago
I believe Bill Gates' assertion that "90% of software hasn't been written" is more true every day. There are so many "workgroup" size (1-10 users) apps that simply don't get written with today's tool stack.

asdfman123 7y ago

Access is the perfect tool for an intelligent, technically-minded person with limited programming experience to create an application to replace spreadsheets. There are a lot of those kinds of people out there, and they're extremely useful in introducing minor optimizations that other people wouldn't be able to find. Access is for that guy who says "I know there's a better way to do this," but doesn't have access (no pun intended) to a team of programmers and a project manager. I didn't major in programming in undergrad but I've taken classes here and there, so in my first job out of college I replaced a really awful system of spreadsheet-jockeying with an Access DB. I considered other options, but that it's self contained and NOT a web application is a feature, not a bug. I couldn't get access to the corporate database, so I just ran the Access DB on a network drive. It's still probably there, ten years later, running happily on its own. Honestly, it seems like the fix for Access... is a better version of Access. It fills a very useful niche, between spreadsheets and full-fledged applications designed by programmers. It's so much easier to make a quick app that works for your organization than get an external team of programmers involved, who will probably tell you "no" or remain unconvinced that you're worth helping. With Access, you don't need political clout, you don't need years of experience, you don't need a title. Just build an Access application and get kudos from everyone in your team for making their lives easier.

dsaavy 7y ago
These are excellent points, I think it’s underestimated how difficult it is to get approval and budget for a small project that doesn’t directly contribute to the bottom line or have huge savings. These small but impactful solutions using Access act as both a stepping stone to greater skill sets, a way to build POCs for non-tech or low-tech people, and to avoid months of politics in big organizations. I’ve had projects that were red-taped before getting off the ground due to resources being placed elsewhere. I then take those project ideas, and instead build them in Access as a POC using shared drives, splitting the database, etc. By the time I have 100 users saying how useful the application is in a couple months, other bigger projects haven’t even finished a project plan. Meanwhile we’re then ready to scale the solution using an appropriate stack and basically can just say “replicate our POC features but improve performance, security, accessibility, etc”. So far that’s worked pretty well for me. In summary: find people’s Excel files with a mess of VBA and formulas —> see if the use case should be expanded —> build POC in Access without permission/budget from a bunch of people —> see how it goes and then plan to scale with the evidence you’ve gathered from your POC.
dbrgn 7y ago
I taught myself Access 97 when I was about 10 years old and helped a teacher set up a small database to register library books. I couldn't have done that with a real SQL database, but Access was somewhat discoverable. Never underestimate the empowering power of tools that can be learned without any external help! I hate Access, but I love it too.
ebiester 7y ago
I know you say that being something other than a web application is a feature, but we have Microsoft's replacement for access in Office 365 and it's PowerApps combined with Flow. I haven't been able to dive too deeply, and it definitely has a learning curve, but so did access. Further, data is actually backed up and permissions can be applied. I also have fondness for Access, but I think that we have a chance for something like that again.
- TeMPOraL 7y ago
  O365 still gives you desktop versions of the Office software. It does makes me a little worried, but I'm hoping Microsoft isn't suicidal and won't pull the plug on desktop software, leaving us only with the half-featured web versions. Flow is interesting, but AFAIR it requires extra clicking in the administrative interface to get running, so it might not work for shadow IT as well as Access does.
  - ebiester 7y ago
    I am hopeful that between MS's financial commitment to the suite, improved JS performance in the browsers, and a set of practices coming around, it will reach parity in the next few years. I run Linux and while I miss features, I can do my job in an O365 org rather than having to have a mac or run a windows client for office suite.
    - TeMPOraL 7y ago
      On the one hand, I'd like it too. I use Linux as a daily driver - but I recently replaced my sidearm machine, and I keep it on Windows, so that I have a computer on which I can run desktop Office. On the other hand, I'm convinced that once Office on-line reaches feature parity with the desktop, they'll quickly drop files, and turn Office into just another SaaS that holds my data for ransom. I don't want that.
qubex 7y ago
I couldn’t agree with you more. I’m currently involved in a small startup where loads of administrative, transactional data is contained and manipulated by hand in an ever-growing Excel spreadsheet. I’m a Mac user but there’s plenty of Windows machines with Office lying around, and my first gut instinct was to start throwing something together and importing the Excel file to populate the DB, and construct ‘robust’(er) queries & scripts for interacting with and getting reports from the data. Perfect? No way. But within a couple of days I was able to get something working more reliably and (albeit temporarily) “shored up” a major calamity waiting to happen. Am I proud of my work? Absolutely not. But it was quick, cheap, and cheerful. Eventually when urgency has lessened and there’s more budget (in terms of mind-space, time, money) we can start looking for developers that will take my kludge and make it into something more bullet-proof. For now, the leap from Excel to Access has been a huge leap forward (the kind that the physically illiterate offhandedly describe as a “quantum loop”, not realising that quantum leaps, though discrete in nature, are tiny).

dexwiz 7y ago

I think Developers also underestimate IT Setup Costs for a DB since its so easy to do it locally. Setting up a simple RDBMS like MySql or Postgres and skinning with with a basic, out of the box UI is an afternoon of work if you need to look up commands, or minutes if you have done it recently. But setting up something for a Team immediately opens questions like Access Rights (Can we limit this just to the team, or even have profiles?), Security (Needs to plan nice on the VPN/intranet), Scalability (in theory easy, but provisioning/managing VMs is hard), and Recoverability (where are the backups stored and who makes them). Those are all hard problems from not just a technical perspective, but also a business one. But that is just the technical hurdles, not the process ones. In a well (overly?) managed system it can take a handful of people multiple meetings and possibly hundred of man hours to add a new field to a form. Adding the field is trivial, but deciding all the business rules around it is hard. If you run your own system, you can ignore any established process. So its two fold. Its not just IT start up costs, its also getting everyone to agree on changes to an existing system, versus running your own. I would argue many of the costs associated with migrating from Access to a "real" DB, is not the actual migration script, but the reassessment of all the business decisions that went into the design of the Access schema. Usually a committee evaluates if they are valid, and what to do with existing data to make it conform to a new schema.

asdfman123 7y ago
I've worked in multiple organizations that run small Access apps here and there and for us the main cost is just sitting down and doing it. My last job was at an HR company and things were historically done through Access and spreadsheets, until an IT guy came along and started writing C# web apps, rose in prominence and now runs a small internal software team. I worked for that guy. It's not terribly politically difficult to convert existing applications, but it's mind numbing work and it's bottom priority. There's not a huge need to change something that already works, and the only reason you do it is you know it can't stay like that forever.
benhurmarcel 7y ago
Plus, you need a server (ie. obtain one, set it up, and maintain it). With Access you just host it on the team's shared drive. In a large organization, getting a small server when you don't work in IT is not simple at all.

mamcx 7y ago

My dream is to build a modern take of the fox/dbase family. I starting with the inner language http://tablam.org http://tablam.org Access, as concept, is great. Is exactly what many need. But it have a lot of small deficiencies that make it disliked by developers. I have used FoxPro, and it have almost all the right things instead (Fox is Access, but goooood!... except a bit less user friendly). I wish I could dedicate to build this. Is weird to me why this kind of tool have so little support?

taffer 7y ago
I really like the idea of developing a modern Access alternative, but why are you trying to create a new language? Aren't QBE and SQL sufficient? Why not something that is like a cloud SaaS version of Access, but can be extended with some procedures and later used as a GraphQL backend like Hasura?
- codeulike 7y ago
  Salesforce is (among other things) basically a cloud SaaS version of Access. And people pay a lot of money for it. See my other comment https://news.ycombinator.com/item?id=21403393 https://news.ycombinator.com/item?id=21403393
  - taffer 7y ago
    True, when I think about what I've seen, what people are building with Access and what people are building with Salesforce, there's a lot of overlap.
- mamcx 7y ago
  SQL/GraphQL are query languages. Good for that, but not for fully develop an app. FoxPro have his own language that allow mix SQL and imperative constructs: https://en.wikipedia.org/wiki/Visual_FoxPro https://en.wikipedia.org/wiki/Visual_FoxPro This mean: * You code login in foxpro * And the stored procedures * And the forms * And the reports * And the script glues * .... "Just" adding SQL/GraphQL to a RDBMS is absolutely not enough. Both are too limited. For example: - Can you build a btree with SQL? No - Work with the terminal? No - Make a visual grid? No - etc Only query and maybe crud.

radcon 7y ago

I don't disagree, but isn't it a sign of much deeper issues if this "Shadow IT" land exists at your company in the first place? Why is it that experienced programmers can't efficiently solve a problem that a business user can handle in Access? Are they under-staffed? Neck-deep in spaghetti code? Lacking business knowledge and the opportunity to acquire it? Bottle-necked by a lack of business analysts or testers? I hear about this too often and nobody seems to think it's a major problem. IT departments can have a queue that's 2 years deep and people don't even bat an eye, they just think "Oh, that's how IT works!" And that's not even taking into account all the requests people aren't submitting because they've given up hope of getting any dev time.

flukus 7y ago
Agreed, I think it's because developers like to over engineer everything and the managers like to over engineer the processes. So many of these access excel solutions should be a days worth of work, simple perl cgi scripts with a minimalist UI deployed by rsync. Instead we have to use our super "productive" modern frameworks, split everything into a thousand files (god forbid you embed an sql query in the only place it's called), add unit tests, etc. There's certainly times for the later approach, but most businesses need much more of the former. So because developers don't have a reasonable platform to pinch off random little projects others step in.
- codeulike 7y ago
  god forbid you embed an sql query in the only place it's called -gasps-
- perl4ever 7y ago
  "So many of these access excel solutions should be a days worth of work" I don't know why you think a lot of VBA would be replaced by a little perl. Of course, my perspective has a lot to do with the fact that it was essentially impossible to get a new perl module installed where I used to work. But developers often don't appreciate the importance of presentation (and other) details in reports for managers. Your reference to "a minimalist UI" is telling. Even though Access and Excel can be buggy, unstable, and annoying, it doesn't make much sense to use anything else if you are automating a report that was previously assembled by hand in Excel, and needs to match precisely. And often IT types like to exercise power by gatekeeping - if you aren't doing "real" programming, you don't need a Turing complete solution, so Office ends up being the only option. I've been told that if I can select a list of columns from a dataset, and some filters, by pointing and clicking, that's all I, or my managers, need for reports. Honestly, I think a lot of people find fulfillment in their work through being the person who can say "no" to people, particularly managers that are theoretically higher ranking. And also by expressing themselves through creative decisions when others fail to specify details. I think that using Office/Access/VBA may be correlated to rejecting the value system of most developers, rather than a technical judgment.

cryptonector 7y ago

Everything needs to be self-service as much as possible. Need a SQL Server / PostgreSQL DB for your project, got click some buttons on a page and get back a URI for your database. Automate, automate, automate.

TeMPOraL 7y ago
That goes against typical IT policy that if people don't use something they can't break it, so let's make sure they can't use shared IT resources.

codeulike 7y ago

Yeah exactly. Working in an IT Dept when someone said "ah they've built a spreadsheet to do it, but now they want us to add that functionality to the main DB" we'd all roll our eyes because it was another lashed-together thing that we would have to unpick and re-implement. But the fact is, if they'd come straight to us and asked us to do it instead of doing it in Excel first, they would have had nothing, for years and years.

dageshi 7y ago
Also you can consider the spreadsheet versions as a nearly feature complete & tested specification.
Ididntdothis 7y ago
“But the fact is, if they'd come straight to us and asked us to do it instead of doing it in Excel first, they would have had nothing, for years and years.” Very true. As soon as us professionals take over the dev speed drops drastically and costs go up.
- Spooky23 7y ago
  I killed a project a few years ago that had spent about $8M on some project to implement a pretty straightforward business process. No end was in sight. The catalyst was Access 2003 going end of life, and this little app doesn't work. I don't know why one of a 3 BAs couldn't figure out WTF was happening and fixing it. We had an intern futz around with Access for a few weeks and it's been running fine to this day.

specialist 7y ago

I believe Bill Gates' assertion that "90% of software hasn't been written" is more true every day. There are so many "workgroup" size (1-10 users) apps that simply don't get written with today's tool stack.