3 ms·
When I worked at LSI Logic, we had a dual core processor that didn't have any cache consistency at all between cores. You actually had to pad all memory allocat
by drmpeg 8y ago
When I worked at LSI Logic, we had a dual core processor that didn't have any cache consistency at all between cores. You actually had to pad all memory allocations to a cache line so that the two cores didn't interfere with each other.
Unfortunately, we also let our customers develop their own code. One of our best customers (Motorola) starting having problems with one of their projects. Small changes in their code would cause all kinds of bizarre crashes.
I was asked to help out and after reviewing their code, I realized that nobody had told them about the cache coherency issue. After making a few changes (just moving .bss and .data variables into cache line protected malloc's), all the problems disappeared.
The funny part was when their engineering manager found out. He calls me on the phone and starts to chew me out. He was furious, so I put him on speaker and called my managers into the lab. We stood around and just let him rant for a few minutes until he ran out of steam. A legendary episode that we chuckled about many times thereafter.
- monocasa 8y agoThe PortalPlayer 5020 in my iPod mini was designed similarly. Two ARM7TDMIs with non coherent caches. I tried to hack up iPodLinux to support both cores in a general way, but never really got it working well. The official iPod software used one for the OS and dedicated the other to codecs IIRC.
- drmpeg 8y agoSame situation on the LSI Logic (actually, it came from C-Cube Microsystems) processor. One SPARC core running VxWorks and applications and the other SPARC core running bare metal and MPEG-2 codecs. The first version of the chip was introduced at CES 2001. Pretty early for a multi-core CPU. Only 150 MHz clock rate though.
- bogomipz 8y ago>"You actually had to pad all memory allocations to a cache line so that the two cores didn't interfere with each other." Can you elaborate? Specifically ow did you "pad all memory allocations to a cache line" and how exactly did that prevent interference? >"After making a few changes (just moving .bss and .data variables into cache line protected malloc's), all the problems disappeared." What is a "cache line protected malloc" exactly?
- drmpeg 8y agoJust an aligned malloc. The cache line size was eight 64-bit words, so aligning on 64 byte boundaries did the trick. The failure mechanism is when one processor writes a memory variable, the entire cache line is written. If the variable is within 64 bytes of a variable on the other processor (and in the same cache line), the variable on the other processor gets (possibly) overwritten. I forget exactly how the aligned malloc was coded. This was 18 years ago and gcc 2.7.2 on SPARC. For the application programmers, we just had an overloaded new operator that was aligned.
- nsb1 8y agoI'm envisioning how this played out... <Motorola Eng Manager ranting away in the background> First LSI engineer: "He seems pretty upset." Second LSI engineer: "Yeah." First LSI engineer: "Think we should do anything?" Second LSI engineer: "Meh, he'll get over it." First LSI engineer: "Still, maybe we should throw him a bone." Second LSI engineer: "<sigh> Fine. I'll add an overloaded new operator after I finish my beer." ... something like that? :)
- drmpeg 8y agoWe had figured out the issue years before that episode. We did write him a nice application note / errata describing the issue. They were just declaring static/global variables that they didn't know were forbidden.