lossolo

64 ms·

871.
by lossolo 4y ago
> if you map every single neuron, at what point is it no longer just a 'model' of reality, and is reality If you map position of every fish in the sea will you have a model of a living sea with living fish and all their interac
872.
by lossolo 4y ago
To simulate something you need to first know how it works and know all the relevant interactions, this is something that we currently do not understand. Neurons are not equivalent of neural networks we use in computers, they are A LOT more
873.
by lossolo 4y ago
There is a second side of that coin. Great quote from 1930s Germany: First they came for the socialists, and I did not speak out—because I was not a socialist. Then they came for the trade unionists, and I did not speak out—because I was n
874.
by lossolo 4y ago
It's because BLOOM is undertrained, you can prune a lot of weights in BLOOM and it doesn't impact performance. Look at Chinchilla paper[1], 70B model outperforms 175B GPT-3 model. https://arxiv.org/abs/2203.15
875.
by lossolo 4y ago
I don't think that's a requirement for consciousness. For instance, I could sedate you, and you would be unconscious until I wake you up and give you a prompt. Then, a snapshot of your brain would react to inputs and output the an
876.
by lossolo 4y ago
Essentially, in all the books that were included in the training data and in almost all science fiction articles/stories on the internet, AI is portrayed as a sentient thinking machine. Bing AI reflects what was in that dataset. The pr
877.
by lossolo 4y ago
Indeed but all the papers I've read about this were only for simple transformers, for now at least.
878.
by lossolo 4y ago
Intelligence exists without language, language is only a way to describe the world around us and transfer information. We personally can't experience intelligence without language because you know language all your life and can't
879.
by lossolo 4y ago
> "E3, D3, C4 ..." and NOTHING else. It knows nothing of the board. It doesnt event know that there are two players. Yeah, like language have gramma rules games also have rules, in both cases LLM can learn rules, it's the
880.
by lossolo 4y ago
> THEN last night I got stuck on a funny syntax issue that googling could never find the answer. I got into a conversation with ChatGPT about it, and after it first gave me the wrong answer, I told it that I need this solution for the la
881.
by lossolo 4y ago
History repeats itself. > In January 1982, President Ronald Reagan approved a CIA plan to sabotage the economy of the Soviet Union through covert transfers of technology that contained hidden malfunctions, including software that later t
882.
by lossolo 4y ago
ChatGPT struggles with out-of-distribution problems. However, it excels at solving problems that have already been solved on the internet/GitHub. By connecting different contexts, ChatGPT can provide a ready solution in just a few seco
883.
by lossolo 4y ago
> 30-50 years ago One of the reasons that is not mentioned is that in the 1960s, the US's share of global GDP was 40%, today it's around 24%.
884.
by lossolo 4y ago
You meant 700 GB? 32 bit precision in 175B model is around ~700GB + overhead or around 350 GB if they use half-precision.
885.
by lossolo 4y ago
> 24x 64 GB nodes is 1.5 TB Looking at your calculations indicates that you mean RAM but it's 1.5 TB GPU VRAM (but this is assuming they use 64 bit precision, which is likely wrong so it's ~750 GB), not RAM.
886.
by lossolo 4y ago
OpenAI whole infra is magnitudes less complex compared to Google cloud and their infra, OpenAI is not creating cloud for thousands/millions of customers with different use cases, they use Azure to host their software, that's why I
887.
by lossolo 4y ago
If you have time to wait for results then sure, it could work in theory but in practice they are so slow and power inefficient (compared to newer nodes) that no one uses them for LLMs, that's why they cost ~200$ used on ebay.
888.
by lossolo 4y ago
We don't know anything about these outages and their nature. Maybe OpenAI outage was caused by a typo somewhere in code and Google had rolled out system wide monitoring + SQL change that had cascading issues and required many not obvio
889.
by lossolo 4y ago
> but imagine if you controlled all of the clients and could always trust them This will eliminate a few checks like teleporting/wall passing etc but you still can't trust the client, rest will shift to ML based cheating method
890.
by lossolo 4y ago
RETRO has external source of data but is something entirely different to what is discussed in the paper. In simplistic terms, in RETRO architecture you store embeddings in an external database and then use kNN search. It is not an end-to-en
891.
by lossolo 4y ago
It's in the article I linked. They deployed models similar to Codex, like LLM, and saw a small boost in productivity. They have also deployed many other models internally.
892.
by lossolo 4y ago
Open access doesn't mean open source. You have access to models through their APIs, you don't have any access to similar Google models.
893.
by lossolo 4y ago
> exceeding OpenAI may have an advantage in terms of publicity and open access to their models, but technologically, Google has better models in-house. Many of the models used by OpenAI are based on work at Google and their papers, and G
894.
by lossolo 4y ago
There are currently unsolved issues such as hallucination, query cost, scalability, fast model update with newly indexed data and toxicity that would need to be addressed for ChatGPT (or LaMDA - Google equivalent of OpenAI ChatGPT) to repla
895.
by lossolo 4y ago
> Since it's impossible to identify such functions by simply looking at them, it'd be a great way of basically performing a DoS-attack on the model. Correct that it's impossible because you would need to solve halting prob
896.
by lossolo 4y ago
> Researchers will flock to the organization with the biggest wins. And right now, that's OpenAI. It depends on the definition of a "win" in this context. Google has developed notable AI technologies such as AlphaGO, Alpha
897.
by lossolo 4y ago
No, this is another straw man from you. You skip over specific words I've written and then project meaning that fits your thesis.
898.
by lossolo 4y ago
> It's a machine designed to pick up on cues about what you want it to do, and then do that - it's practically engineered to confirm your biases. Mainly, biases it picked up from the training data. That's why most language
899.
by lossolo 4y ago
> The greatest threat is that we don't know its motivations, but then you proceed to explain its motivations as evidence of why it is the greatest threat. I'm not sure why you are dismissing what I wrote. I gave specific featur
900.
by lossolo 4y ago
> If full AGI is ever created then there's no telling what its thoughts and motivations will be. That is why it poses the greatest threat to humankind in history. If we develop a fully self-aware mechanical being, commonly known as

More ›