5 ms·
do 4x512gb builds actually make sense now to run a big model (beyond a cool demo)? or are 200-500B moe models the sweet spot? In which case clustering 256gb bui
by traversajulian 1mo ago
do 4x512gb builds actually make sense now to run a big model (beyond a cool demo)? or are 200-500B moe models the sweet spot? In which case clustering 256gb builds might make more sense.