Yes, great question! The system started without paged attention, and recreated its own paged attention implementation automatically once it realized it was a b

Yes, great question! The system started without paged attention, and recreated its own paged attention implementation automatically once it realized it was a bottleneck. Pretty cool!