novaRom

11 ms·

91.
by novaRom 2y ago
Most articles today are written with LLMs. I found even entire web sites devoted to certain niche hobbies are LLM generated and ranked high on duck/google.
92.
by novaRom 2y ago
TL;DR we've trained n-gram model implemented in a form of suffix array on very large data set, it shows good perplexity
93.
by novaRom 2y ago
I did something similar many years ago in VHDL. There was a site called opencores for different open source HDL projects. I wonder if is there any good HPC level large scale distributed HDL simulator exists today? It makes sense to utilize
94.
by novaRom 2y ago
Large fast FPGAs are great but very expensive, small size slow FPGAs are not practical for most solutions, where ARM controllers are used, significantly cheaper.
95.
by novaRom 2y ago
What is technology behind this? Is it continuation of OpenAI's JukeBox or something diffusion or transformer based? Any influential papers?
96.
by novaRom 2y ago
Why someone still would use GPT-3.5 in 2024? There are tens of fully open models available which beat GPT-3.5 in every possible skill and you can run them locally.
97.
by novaRom 2y ago
Cohere’s Command R+ is unimpressive model, because it agrees with me every time I try to argue with smth like: "But are you sure? ..."; also it has: "last update in January 2023". Mixtral 8x22B is interesting because 8x7
98.
by novaRom 3y ago
Delaware is a popular state for incorporation in the United States due to its favorable corporate laws and established legal precedents. A C-Corporation (C-Corp) is a type of business structure that is a separate legal entity from its owner
99.
by novaRom 3y ago
Why relocation is needed? Can a EU citizen open a business in Estonia?
100.
by novaRom 3y ago
Discreteness, finitenes, causality.
101.
by novaRom 3y ago
TL;DR: This model is based on two assumptions: 1) The forces of nature decrease over cosmic time 2) Light is losing energy when it travels a long distance
102.
by novaRom 3y ago
https://archive.is/raAce
103.
by novaRom 3y ago
> largest/simplest productivity improvement in my life, so far many productivity improvements in the last years: Internet Search, Internet Forums, Wikipedia, etc. LLMs and other AI models is continuation of the improvement of infor
104.
by novaRom 3y ago
https://archive.is/Ou5Sb
105.
by novaRom 3y ago
It might be even quite an advantageous trait in our fast changing modern world of LLMs, constant distractions, and uncertainties.
106.
by novaRom 3y ago
I think with Mixtral Medium they mean MoE 2x13B which is on top on huggingface leaderboard? It is still not close to 8x175B, but size alone is not most important factor. With smarter training methods and data it is possible we will see perf
107.
by novaRom 3y ago
We have to add LLMs and MMMs (multi modal models) into all standard Linux distributions. A service will index all local files creating embedding connectors, this will be used to augment user prompts, and voila we can search for anything wit
108.
by novaRom 3y ago
I just tried Mistral MoE 8x7B model and it works a bit faster than llama-2-70B but it looks it has almost the same skills. In fact, all latest models of 13B-70B size are quite similar. Could it be large part of their training data is the sa
109.
by novaRom 3y ago
It's a disadvantage of current SOTA models: they are easy to train, but they must be large wasting lots of weights in order to generalize well. Maybe another architecture, transformer's successor will be more economical - having l
110.
by novaRom 3y ago
OpenAI do probably realize they will not win long term vs Open Source (see AI Alliance). Their way of centralized cloud models is simply too risky and not sustainable. What we see instead is more liberation, open source, cooperation, down-s
111.
by novaRom 3y ago
That's awesome, here is original paper: https://arxiv.org/abs/2303.10798
112.
Hexagonal Grids (2013)(redblobgames.com)
250 points by novaRom 3y ago|56 comments
113.
by novaRom 3y ago
Drinking water. But you can buy 500mg of water for 5 Euros of course, right? Traveling as a family with with children, and when your departure is delayed, you can spend quite a bit just to stay hydrated.
114.
by novaRom 3y ago
> They were loud! Also not hugely comfortable either. We didn't really sleep at all We tried Munich-Rome once and it was not really good experience, especially because windows were dirty, beds are small and not comfortable. Narrow s
115.
by novaRom 3y ago
AMD is close to release a new data center GPU, I wonder if price will be lower than H100. It seems PyTorch is fully supported, so it should be a good option for AI training.
116.
by novaRom 3y ago
In modern Deep Learning , very few high-impact papers contain any scientifical theoretical explanation. It's more like trial and error, "we found A works and it improves B". Tech reports.
117.
by novaRom 3y ago
16Gb is minimum to run 7B model with float16 weights; out of the box, with no further efforts.
118.
by novaRom 3y ago
Clear skies and drier weather should result in reduced cloud cover, which allows more direct sunlight to reach the surface during the day, leading to warmer temperatures. During the night, the lack of cloud cover also means there is less in
119.
by novaRom 3y ago
Current stable pattern changes into another stable one, probably quickly. Or it may split into two independent stable patterns. It will not disappear. It may result in long term climatic changes in different geographies, especially during t
120.
by novaRom 3y ago
I did use a tweaked nanoGPT to pretrain a 12M model on TinyStories (2Gbytes produced by GPT4), and results are pretty amazing. I've adapted it a bit on Wikipedia then, and it looks like a solid bullshit generator, much smarter than any

More ›