26 ms·
I flew over to see a customer (only a 2h domestic flight) to solve an 'unsolvable' problem. They'd been going through it with support for weeks with no luck. I
by alexkus 14y ago
I flew over to see a customer (only a 2h domestic flight) to solve an 'unsolvable' problem. They'd been going through it with support for weeks with no luck. I got to the customer, took over his chair, edited a single file, added a missing ; character and it worked. Done. I got the next day and a half 'off' because my hotel/flights couldn't be rearranged for an earlier return.
The root cause was a piece of paper on his desk with a diagram of the systems and their IP addresses.
Direct shell access to the machine wasn't possible. To get a shell on the machine that ran the processes he had to go via a separate gateway host, and the connection from the gateway host to the destination machine was done by hostname (as the gateway machine had access to the internal DNS).
To get a copy of any config files he was able to FTP to the server directly from his laptop, but he wasn't able to use the same DNS servers as the gateway host so he relied upon the piece of paper on his desk which had the names and IP address of the machines. Unfortunately the IP address for the system containing this config file was the one for the corresponding dev system, not the production machine.
Every time he ran the process he was using the config file that was missing a ; character and so it spat out an error.
Every time he got a copy of the config file (via FTP) he was actually getting it from the dev system which wasn't missing the ; character.
Much back and forth from support on "are you sure that's the right config file?" and he always responded with "yes".
When I got there I looked at the config file via the shell (on the right machine obviously) and spotted the problem straight away.
I didn't make any money out of it, but I did get a day and a half to look around a new city on expenses.
- nthj 14y ago> I didn't make any money out of it, but I did get a day and a half to look around a new city on expenses. This strikes me as curious. Were you billing hourly?
- alexkus 14y agoNo, salaried. [EDIT] I didn't quite walk out straight after fixing the problem. I stuck around to help out with any other queries they had with our software but they ran out of questions by lunch time. [EDIT2] Customer wasn't being billed at all for my visit. They pay annual maintenance for the software though. If it's a serious (or weird) enough problem (or they shout loud enough) then someone is sent on site.