I’m near term AI-skeptical, but I think most of the capability & cost-related problems will be solved by improvements to current model architectures. My persona

I’m near term AI-skeptical, but I think most of the capability & cost-related problems will be solved by improvements to current model architectures. My personal canary for the start of this will be when we start seeing successful architectures beyond (roughly) “tokens -> transformer blocks -> linear transformation to probability distribution”. I wanted to engage with a few of the author’s specific points of skepticism though (hope you don’t mind, if you’re reading this) since I’m not sure if I agree with them. > Energy Usage This has always seemed like a fake problem to me, and mostly just a way to rally environmentalists around protesting genai (and bitcoin and…). The costs (time & capital) associated with nuclear power are mostly regulatory; in the past decade China has routinely built numerous modern reactors while the west has been actively scaling nuclear power down. I have yet to see a good case for not simply scaling up power generation to meet demand. This is assuming that AI power demands remain constant and even require gigawatt-scale power sources to train & operate. For all we know, one paper from one researcher anywhere in the world could show us that current model architectures are orders of magnitude inefficient and make current-standard frontier model training doable in a basement. > The Educational Impact A few years ago I came across some cool computer vision videos on youtube and became inspired to try reading the papers and understanding more. The papers were surprisingly short (only a few dozen pages usually), but contained jargon and special symbols that I found impossible to google. I eventually read more papers and even some books, and eventually discovered that if I had simply read e.g. Hartley & Zisserman’s Multiple View Geometry (commonly just referenced as “MVG”, another mystery I had to figure out) I would have had the hidden knowledge necessary to understand much of the math. If I had simply read earlier papers, I’d be familiar with the fact that much of the content is just slight additions on previous works rather than fully greenfield ideas. I’m not rich enough or successful enough to access the formal education that would have probably provided me the resources to learn this. Youtube videos from practitioners in the field felt more like insecure nerds gatekeeping their field with obtuse lessons. It took me about a year to produce anything useful without just using opencv. When I tried asking the same questions I had as a beginner, chatgpt was able to answer everything I was confused on, answer my dumb followup questions, and even produce demo code so I could explore the concepts. That experience has made it an irreplaceable educational tool for me. Im sure more people will cheat, but college is already an overvalued competence signal anyways. > The Invasion of Privacy Temporary problem, someone will produce a usable hardware equivalent that makes it similarly easy to use genai w/o the privacy implications. This is a segment begging for someone to provide a solution, I think most people with the resources currently to solve it just want to be the AI gatekeeer and thus the service model for this tech. > The Stealing We won’t find common ground, I dont believe in ownership of information and think copyright is largely immoral. > The Fatigue This section seems to largely revolve around no clear obvious tooling route to simply using the tech (complete empathy there, Im currently building my own tooling around it), along with not having a clear mental model of how these models work. I had the same anxiety when chatgpt first blew my mind and just decided to learn how they work. They’re dirt fucking simple, shitty non-learned computer vision systems are orders of magnitude more complex. You seem like a cool person, hmu my profile name @ gmail if you want to talk about it! I like 3b1b’s videos if you want a shorter version, karpathy is overhyped imo but also cool if you want to build a toy model line by line,