3 ms·

Does One Large Model Rule Them All? (Eric Schmidt and Matei Zaharia)

23 points by tim_sw 4y ago

pfannkuchen 4y ago
This feels a lot like polytheism vs monotheism.
ftxbro 4y ago
The large models are going to split into three classes: two server classes and one 'edge' class. The 'guru' server class of models will be gigantic and mind bogglingly expensive, costing billions of dollars to train and created only by a small number of the largest corporate-government collaborations. But they will be relatively slow and have high latency for inference. They will be highly regulated and treated at the same secrecy level as the nuclear science source codes in government labs. They will be used for two use cases: the first use case is to be used directly for high level corporate, government, and military solutions. The second and arguably more important use case is to act as the 'teacher' or 'distillation source' or 'high fidelity master' to be refined to create the lower tier 'sage' class of models. The 'sage' class of models is hundreds of times smaller, faster, and cheaper for inference than the 'guru' class from which it is derived. Even though it is orders of magnitude smaller it is still very large and expensive and can be run only in large data centers. These models are also highly guarded because they contain the (lossily) distilled wisdom of the 'guru' class models. These are the ones that will be used in most applications that have internet access and for which some amount of latency is acceptable. Access to these models is gatekeeped by the ones who control access to the larger 'guru' class models; attempts to bypass the gatekeeping to create an instance of this class of model directly without distillation will result in a worse model in terms of cost per inference. Presumably they will exist in a spectrum that allows faster and less thoughtful responses vs. slower and smarter responses. The third class will be the 'inference processing unit' (IPU) models which are highly distilled models that are written into the hardware of tamper proof 'trusted computing' modules in every IOT computing device. Even though they are local, they are 'trusted' to not say or render undesirable things. They cannot be replaced or modified by the users. They are highly quantized all the way to 1-bit quantization: they are tamper proof trusted sparse binary XNOR networks, and they are used as black boxes by the code that interfaces with the cyber-physical systems.
virtualritz 4y ago
Great example of what combination of font and color scheme not to choose. An ultra light typeface, gray on white. I have to squint to read this on my mobile phone -- despite the font size being large.
villgax 4y ago
Am I the only one perplexed about how is Eric relevant to any AI related discussion.
- asfarley 4y ago
  After learning some details about Lex, I lost whatever small faith that I had in Eric Schmidt’s technical aptitude in general.
ccvannorman 4y ago
This feels inherently unprovable / uncomputable. How would you prove that one model would be superior in every way to any other arbitrary model? For that you would need a model that was complete and consistent - which we've proven beyond a doubt is impossible (Godel) ..

23 points by tim_sw 4y ago

This feels a lot like polytheism vs monotheism.

The large models are going to split into three classes: two server classes and one 'edge' class. The 'guru' server class of models will be gigantic and mind bogglingly expensive, costing billions of dollars to train and created only by a small number of the largest corporate-government collaborations. But they will be relatively slow and have high latency for inference. They will be highly regulated and treated at the same secrecy level as the nuclear science source codes in government labs. They will be used for two use cases: the first use case is to be used directly for high level corporate, government, and military solutions. The second and arguably more important use case is to act as the 'teacher' or 'distillation source' or 'high fidelity master' to be refined to create the lower tier 'sage' class of models. The 'sage' class of models is hundreds of times smaller, faster, and cheaper for inference than the 'guru' class from which it is derived. Even though it is orders of magnitude smaller it is still very large and expensive and can be run only in large data centers. These models are also highly guarded because they contain the (lossily) distilled wisdom of the 'guru' class models. These are the ones that will be used in most applications that have internet access and for which some amount of latency is acceptable. Access to these models is gatekeeped by the ones who control access to the larger 'guru' class models; attempts to bypass the gatekeeping to create an instance of this class of model directly without distillation will result in a worse model in terms of cost per inference. Presumably they will exist in a spectrum that allows faster and less thoughtful responses vs. slower and smarter responses. The third class will be the 'inference processing unit' (IPU) models which are highly distilled models that are written into the hardware of tamper proof 'trusted computing' modules in every IOT computing device. Even though they are local, they are 'trusted' to not say or render undesirable things. They cannot be replaced or modified by the users. They are highly quantized all the way to 1-bit quantization: they are tamper proof trusted sparse binary XNOR networks, and they are used as black boxes by the code that interfaces with the cyber-physical systems.

virtualritz 4y ago

Great example of what combination of font and color scheme not to choose. An ultra light typeface, gray on white. I have to squint to read this on my mobile phone -- despite the font size being large.

villgax 4y ago

Am I the only one perplexed about how is Eric relevant to any AI related discussion.

asfarley 4y ago
After learning some details about Lex, I lost whatever small faith that I had in Eric Schmidt’s technical aptitude in general.

ccvannorman 4y ago

This feels inherently unprovable / uncomputable. How would you prove that one model would be superior in every way to any other arbitrary model? For that you would need a model that was complete and consistent - which we've proven beyond a doubt is impossible (Godel) ..