4 ms·
I have a thousand similar stories (my comment history will turn up several of them). For context, in the late 90s, I started at "the first iteration of" Fronti
by mdip 5y ago
I have a thousand similar stories (my comment history will turn up several of them).
For context, in the late 90s, I started at "the first iteration of" Frontier Communications, not long after it transformed from Rochester Telephone and Telegraph[0], before its acquisition by both Global Crossing and Citizens (I went with the Global Crossing side).
We were a ~20,000, no 5,000, no 15,000, no 10,000, well, somewhere in there depending on what part of any given year over the 2 decades that I worked for the variety of companies that started, for me, there.
On one occasion, we had a process for collecting tariffs. That term isn't exactly right, but it describes it best. Basically, when a phone call is placed in the US, if it does not terminate with your carrier, your carrier is paying a fee to another carrier, or to a long-distance provider[1]. For call termination purposes, Global Crossing was almost always the long-distance provider, not the local provider[2] and we were paid an amount by one of the larger players that doesn't go unnoticed when it's missing from the balance sheet.
The finer details are in a previous comment of mine, but the end result was my discovery that the process for getting paid was tied to a decade old desktop PC sitting on a shelf in the main data-center with a USR modem attached to it which was recently moved from Green Bay, WI to Detroit, MI (and none of the original staff came along with it). It was sitting at a boot screen asking for hard disk parameters. I reported to a VP at the time who came to know me as the guy to go to for the strange stuff, and having spent my teens building PCs and inputting those parameters I recognized label with the odd acronyms/numbers on it as the parameters it was complaining about -- clearly the last guy noticed the dead battery and the label maker, not a replacement battery, was available.
Up until a few years before I left, the "trouble-ticketing" system used for all customer and IT support was purchased, source code and all from a bankrupt ISV[3]. It used a Java fat client that talked SOAP, I think, to the worlds most normalized, awful performing database I've ever encountered. I saw queries coming from this app where the UI indicated date ranges like "1/1/2003 - 1/1/2005" and the app would spit out a query with "BETWEEN" statements for each day, with nested parenthesis. It was clearly the worlds ugliest ORM, and trying to make sense out of the over 2,000 tables that made up the application was, barely possible.
To resolve this grief, our internal development team undertook writing a web UI for it. At a certain point, I want to say we had twice as many physical hosts operating our internal ticketing system than we did hosting everything we exposed externally, and not just "more boxes" but the most powerful hardware in the DC.
There seems to be a lot of sentiment, here, that if this were a "sign-up" process, it'd be far faster/better.
Anecdotally, my experience leads me to think this is more "laziness/enterprise inertia/large-company edge-cases" than it is "it's not bringing in money, so we'll do it the dumbest way imaginable". At least, I know for a period of time large pieces of the on-boarding had the appearance to the customer of being highly automated, but at the end of the day, it was a Clarify ticket passed between several hands who made phone calls/sent faxes. They'd certainly allocate larger staff to something that brings in revenue, but anything regulatory-related, at least where I was (and probably similar in all Telecom -- almost certainly more obnoxious in anything banking-related) was taken very seriously.
The fact that they have contract staff in India handling the process manually says two things to me: (1) somewhere, there's a fine/other clear risk if they don't and (2) someone decided this was the cheapest way to do it. I'd challenge whether that last part is actually true, but I've seen so many strange things where the reason turned out to be "there's some lawyer/some rule somewhere that says we have to also do this odd thing when we unsubscribe a user and writing a tool to access the system that does that one thing will cost us $150,000". Usually that "odd thing" is some form of tracking for auditing purposes and the system that would be used for tracking whatever that odd thing is ends up being either ancient/impossible to interface with or comes with some insane Enterprise-y license model that results in costs doubling whenever you want to do anything useful.
[0] Name escapes me.
[1] I have been out of Telecom since just before Level 3 became CenturyLink (now Lumen). This was the case when I left, it is probably still the case, today, but someone might have fixed that by now, too.
[2] This was GC days, still operating a few actual termination points, most of the business was IP or traditional long-distance; Technically, though, GC was a CLEC.
[3] My memory is foggy, I may have details incorrect here but the product was called Clarify. To make matters worse, there was at least three different vendors out of ... eight or so? ... that called their product "Clarify" or used that in its name. I'm not even sure if ours was just "Clarify" because by the time we were done with it, its UI was entirely internal and it had an (ugly) internal logo.