2 ms·
This point of view will be wrong in the long term. There are about 10 billion obvious things you can do with transformers as they are today that just haven't be
by tracerbulletx 2y ago
This point of view will be wrong in the long term. There are about 10 billion obvious things you can do with transformers as they are today that just haven't been done yet, and it's pretty likely one of these labs will iterate on transformers and make them much more powerful.
- boppo1 2y agoCan you elaborate?
- tracerbulletx 2y agoThe power for transformers is the generalizability of being able to learn a function to transform an arbitrary input signal in any modality into an arbitrary output signal in any other modality. Chat is fine, image generators fine, universal translation was more than fine. But any thing that can have a symbolic representation is a target for creating a new product using transformers. I think the big value will come from some random obscure places like CAD or real time graphics pipelines. I just think it will be a victory of a million paper cuts where it gets more cost effective than the alternatives in a million various industries. It will just be a ubiquitous technique when solving computing problems and people are going to create more specific datasets for specific problems and continue to get each industrial usage to a level where it's good enough at each task to change each market.
- benterix 2y agoPeople are already trying that with CAD and so on. Still, the main challenges are still there: enormous power needed to train the model (within reasonable time) and relatively powerful hardware to actually use it, inconsistency in output, bugs that you can't meaningfully debug. You might be right about many things above but so far traditional algorithms remain a far better option for many of these use cases (deterministic and efficient).