tarruda

29 ms·

451.
by tarruda 3y ago
3 billion parameters. Does that mean I will be able to run on a 8gb consumer GPU?
452.
by tarruda 3y ago
> I suffer a lot with context switching all the time, I'm responsible for many serious things that could lead the startup to a completely failure. I believe I'm in a similar situation. Currently on a small startup where I do ba
453.
by tarruda 3y ago
Maybe it uses a higher quality dataset
454.
by tarruda 3y ago
I didn't measure, but IIRC it was lower than 1 token/sec
455.
by tarruda 3y ago
3 tokens/sec is a lot faster than what I experienced. Even though your CPU has a lot more cores, I think llama.cpp was not being able to make good use of more than 8 threads. When did you test this? Maybe llama.cpp had some improvement
456.
by tarruda 3y ago
I ran the 30b and 65b Q4 on a laptop with 64 gb of RAM (8/16 CPU). It worked but token/s was very low for it to be practically useful.
457.
by tarruda 3y ago
Awesome. In my case I took the second option. When I was younger I used to get stressed about not being able to implement what I thought was best in terms of code/design. Now I just do what I'm told (at a very moderate pace) and f
458.
by tarruda 3y ago
Is your manager or anyone else in the team harassing you? If not, then your anxiety might come from an expectation that you must love every part of your job. Use something like pomodoro (this is simple and works: https://brainpls
459.
by tarruda 3y ago
The prompt contains rules of the game and a few examples. Does anyone know if GPT really learns 2048 only from this prompt or most of its knowledge came training data?
460.
by tarruda 3y ago
I was thinking more of a situation where you have a good understanding of the subject, just using GPT to generate a solution for you.
461.
by tarruda 3y ago
> I say this as somebody that probably uses it every 10 minutes Do you feel like your ability to produce correct output with GPT improved with practice? If so, then eventually you might reach a point where you get 99.99% accuracy with it
462.
by tarruda 3y ago
> profit generating employees Good professionals are being hit by these layoffs, but I suspect the vast majority create little to no value for their employers.
463.
by tarruda 4y ago
> Sony let North Korea censor what US audiences saw, for a while Source?
464.
by tarruda 4y ago
> The hype that LLMs are going to replace SWEs is clearly just that, hype LLMs cannot replace anyone, but it is clear that engineers which master LLMs usage might multiply their productivity by a lot. The question is: If one LLM assist
465.
by tarruda 4y ago
Looking forward to the next generation of cheap GPUs with enough VRAM to run models like Vicuna-13 locally.
466.
by tarruda 4y ago
Also worth noting that Zig embeds C stdlib source code (musl if I'm not mistaken). That means it is easier to cross compile C projects using zig since you don't need to install a cross toolchain. This is why some golang/rust
467.
by tarruda 4y ago
I did a bit of Zig exploration a few months ago, here's a few things that caught my attention: - You don't have implicit allocations when using Zig stdlib. For example, when you instantiate an ArrayList or HashMap, you need to pas
468.
by tarruda 4y ago
> * Privacy Considering it is using GPT-4 API, you can take for granted that at least OpenAI is collecting your data. Not sure how Microsoft deal with OpenAI works, but it is possible they also have access to it.
469.
by tarruda 4y ago
Did you use GPU inference? If so, how much memory is required?
470.
by tarruda 4y ago
> This notion isn't backed by anything Maybe by the fact that big LLMs have sophisticated behaviors that no one understands and/or can explain yet?
471.
by tarruda 4y ago
I did try llama 30b Q4 on textsynth/CPU. I did not measure, but TextSynth seemed faster than llama.cpp
472.
by tarruda 4y ago
A Monday morning unknown pleasure. Thank you.
473.
by tarruda 4y ago
> File system semantics need a rethink Normally when writing software that needs fopen/fsync, I would just use SQLite instead
474.
by tarruda 4y ago
You should be able to run Llama 30B q4 on a RTX 3090. Check the table here: https://bellard.org/ts_server/ 30B/q4 requires 20GB of RAM while 3090 has 24GB.
475.
by tarruda 4y ago
Somehow not surprised Bellard is also going to make an impact in AI/LLMs
476.
by tarruda 4y ago
> turning 10x programmers into 50x programmer-project-managers, hence just ever more scarce I don't understand the math here: Why would it make programmers more scarce?
477.
by tarruda 4y ago
Maybe they are experimenting with GPT automating devops
478.
by tarruda 4y ago
Didn't explore much, but it seems alpaca-lora has better results for coding tasks. One example I've used was: "Implement quicksort in python.". This is the result with Code alpaca: def quicksort(arr): if le
479.
by tarruda 4y ago
Maybe the OP is also contaminated a bias of wanting to find errors in GPT-4? Finding flaws in GPT-4 and ignoring the fact that we are at the dawn of AI, that seems like a good remedy to calm the AI existential crisis anxiety nowadays.
480.
by tarruda 4y ago
> Yesterday, I watched someone upload a video file to a chat app, ask a language model “Can you extract the first 5 s of the video?”, and then wait as the language model wrote a few lines of code and then actually executed that code Have

More ›