5 ms·

>#1 assumption I always make sure to check now is "is the code that's running actually from the source code I'm looking at". Once I got so far as stepping thro

>#1 assumption I always make sure to check now is "is the code that's running actually from the source code I'm looking at". Once I got so far as stepping through some malfunctioning code at ASM level on a production machine, only to discover that the ASM had a jump if equal instruction while the source code prescribed exactly the opposite. I compared to the executable binary that was supposed to be there, and it was different from what was actually there. There had been a bit flip somewhere along the line. I now spend more time than I probably should comparing hashes of any executable binary that's acting strangely enough.

felipemnoa 5y ago
I've been there a couple of times. Relevant: https://xkcd.com/1316/ https://xkcd.com/1316/
13of40 5y ago
I worked at Microsoft back when they were looking at the Itanium as a Win64 platform. We were making a .Net console application and one day I was in the lab while our daily test automation was running, and the Itanium machines had a bunch of crazy random chars and colors on the console windows. It wasn't causing tests to fail, but I rolled up my sleeves and debugged the issue, thinking it was probably just some buggy test thing. It turned out we were doing something nobody else was doing in .Net at the time, which was to return a structure by value while we were populating the console buffer. Apparently the JIT compiler lost track of which registers it was using to store the struct and overwrote them. In the disassembly it looked like "move some stuff into a register, then move more stuff into the same register". Thought I was losing it for a second. I reckon the moral is don't get into the mindset that the test is always broken. Sometimes it's actually the product.
lmm 5y ago
We had a fun one at my first job where the same Java webapp was behaving differently on a couple of servers (this is back when servers were administered manually by sysadmins). We did a recursive md5sum of the homedir on each server and got the same results. Turns out that by default tomcat will use filesystem order when loading jars from its library directory, and some of the servers used different filesystems, so even though each one had the same set of jars in the library directory, they would be loaded in a different order which lead to different behaviour.

felipemnoa 5y ago

I've been there a couple of times. Relevant: https://xkcd.com/1316/ https://xkcd.com/1316/

13of40 5y ago

I worked at Microsoft back when they were looking at the Itanium as a Win64 platform. We were making a .Net console application and one day I was in the lab while our daily test automation was running, and the Itanium machines had a bunch of crazy random chars and colors on the console windows. It wasn't causing tests to fail, but I rolled up my sleeves and debugged the issue, thinking it was probably just some buggy test thing. It turned out we were doing something nobody else was doing in .Net at the time, which was to return a structure by value while we were populating the console buffer. Apparently the JIT compiler lost track of which registers it was using to store the struct and overwrote them. In the disassembly it looked like "move some stuff into a register, then move more stuff into the same register". Thought I was losing it for a second. I reckon the moral is don't get into the mindset that the test is always broken. Sometimes it's actually the product.

lmm 5y ago

We had a fun one at my first job where the same Java webapp was behaving differently on a couple of servers (this is back when servers were administered manually by sysadmins). We did a recursive md5sum of the homedir on each server and got the same results. Turns out that by default tomcat will use filesystem order when loading jars from its library directory, and some of the servers used different filesystems, so even though each one had the same set of jars in the library directory, they would be loaded in a different order which lead to different behaviour.