This looks fantastic. Two ideas are - An automated test suite verifying the (near) equivalences. - Might be fun for someone to provide parallel implementation

This looks fantastic. Two ideas are - An automated test suite verifying the (near) equivalences. - Might be fun for someone to provide parallel implementations in other linear modeling frameworks (scikit-learn, julia, etc).