Reading the data is hardly the problem. Most R libraries seem to be inherently single threaded. Unlike sklearn where just about everything has a n_jobs paramete

Reading the data is hardly the problem. Most R libraries seem to be inherently single threaded. Unlike sklearn where just about everything has a n_jobs parameter. Even stuff like xgboost is insanely parallelizable. Try fitting a really big mixed effects model with LMER, you will cry (I have submitted multiple fixes and performance improvements to LMER and Mertools). Cool I have a model. Now I want to serve predictions in real time to downstream APIs. Hello single threaded R runtime my old friend.