One thing I've really wanted to try is a FPGA on a socket. I've seen FPGAs that fit in CPU sockets before and used the QPI link to communicate to the other proc

One thing I've really wanted to try is a FPGA on a socket. I've seen FPGAs that fit in CPU sockets before and used the QPI link to communicate to the other processor and anything else on the ring. Since you don't need to pay the PCI bus latency costs you could potentially do some pretty wild things such as talk to the NIC directly and keep your book on the FPGA and pushed into the LL cache. You can even speak to the LL cache directly, make DMA calls. Talk to the memory controller. Etc. I've tried some GPU stuff for options and looking down futures curve, but the cost of going off core is prohibitive in most cases.