thntk

5 ms·

1.
by thntk 1y ago
The model architecture only uses and cites pre-2023 techniques from the GPT-2 and GPT-3 era. Probably they intentionally tried to use the most bare transformers architecture possible. Kudo to them to have found a clever way to play the open
2.
by thntk 2y ago
Anyone know if it can run training/fine-tuning and not just 4-bit inference? Does it support mixed precision training with either BF16 or FP16?
3.
by thntk 2y ago
It sounds like your startup is doing great, good luck.
4.
by thntk 2y ago
Have you tried hiring or rotating internal people just for solving specific (management) tasks? It is like the "just-in-time" style in Japanese corps.
5.
by thntk 2y ago
It's not founder mode or manager mode. I think it is just about effective management. When starting up, the founder needs to (1) know what should be done, (2) be able to do it themself, and (3) do it and confirmed it's done. When
6.
by thntk 2y ago
Isn't it the technological cycle in IT? We have seen PC softwares in the 90s, then web apps, then mobile apps, and now AI services. Except that currently AI is immature, so companies do not know exactly what to do with it yet. They are
7.
by thntk 2y ago
Anyone know what caused the very big performance jump from Large1 to Large2 in just a few months? Besides, parameter redundancy seems evidenced. Front-tier models used to be 1.8T, then 405B, and now 123B. Would front-tier models in the futu
8.
by thntk 2y ago
When Zuck said spy can easily steal models, I wonder how much of it comes from experiences. I remember they struggled to train OPT not long ago. On a more serious note, I don't really buy his arguments about safety. First, widespread A
9.
by thntk 2y ago
Correct me if I'm wrong, my impression is that 3.1 is a better fine-tuned variant of base 3.0 with extensive use of synthetic data.
10.
by thntk 2y ago
I've seen such articles more and more recently. In the past, when people had a vague idea, they had to do research before writing. During this process, they often realized some flaws and thoroughly revised the idea or gave up writing.
11.
by thntk 2y ago
We knew high quality data can help as evidenced by the \Phi models. However, this alone can never eliminate hallucination because data can never be both consistent and complete. Moreover, hallucination is an inherent flaw of intelligence in
12.
by thntk 2y ago
Besides the burden of knowledge and the flaws of academic funding, another factor to explain the slowdown in scientific progress is: the world has more things to be entertained with , including books, comics, games, movies, music, p@rn, so
13.
by thntk 9y ago
Oh, that is so ridiculous that I have to come back and LOL.
14.
by thntk 9y ago
What does porn have to do with free speech and privacy?