This seems like the key research to me if we want any shot at preventing the technology from being locked away behind big corp API walls interfacing giant data

This seems like the key research to me if we want any shot at preventing the technology from being locked away behind big corp API walls interfacing giant data centers. Anything that removes the bloat and mysticism from the models so they can be scaled down and run on the little guy's computer is orders of magnitude more progress in my opinion than, e.g., increasing the token window by some epsilon.