popinman322

6 ms·

31.
by popinman322 3y ago
It's valuable data sometimes. Those types of comments show you how other people perceived something. In this case my lens is colored by my interests; I've been thinking a lot on RL for LLMs and it slipped out here. For me I rely o
32.
by popinman322 3y ago
Among arxiv publications there are 217 results that contain "large language model" in the full text and "from scratch" in the title or abstract. There are 2873 results that contain "large language model" in the
33.
by popinman322 3y ago
When I saw "from scratch" instead of "pretrained" I assumed this was trained using some novel RL setup-- there's value to using the same verbiage as everyone else.
34.
by popinman322 3y ago
Half an hour of time recovered using a system not specialized to the work at hand. With better tooling and integrations that could scale to several hours a day spent doing more valuable work.
35.
by popinman322 3y ago
YouTube is interesting because most large creators, who take a majority of views, also have ad space inside their videos
36.
by popinman322 3y ago
AWS offers optimized inference and custom infrastructure ( https://aws.amazon.com/machine-learning/inferentia/ ).
37.
by popinman322 3y ago
Everyone is different and has different motivations. We don't all think the same way, either. Some workers just want a stable, sane work environment and are willing to compromise on pay for other benefits. These same people are typical
38.
by popinman322 3y ago
Does this work in tandem with beam search or does it do greedy sampling?
39.
by popinman322 3y ago
It should hopefully be a quick change to llama.cpp to add a mask per grammar state to bring it in line with your generation method; I don't think the two are incompatible, thankfully. I do wonder how much you win here by masking the to
40.
by popinman322 3y ago
Was this with or without fine-tuning?
41.
by popinman322 3y ago
I've done exactly this for another project. I'd recommend grabbing an open source model and fine-tuning on some augmented data in your domain. For example: I grabbed tech blog posts, turned each post into a collection of phonemes,
42.
by popinman322 3y ago
The correlation between IQ and job performance is typically weak[0] (weaker than the correlation between conscientiousness+agreeableness on job performance in some studies[1]) with a more modest correlation for "high complexity jobs&qu
43.
by popinman322 3y ago
Active Directory is actually pretty good I think. Having worked with both Azure and AWS, AzureAD makes AWS IAM permissions feel half baked. Granted my experience was inside Azure so we had the benefit of vertical integration.
44.
by popinman322 3y ago
1xH100 is faster than 8xA100 for a work-in-progress architecture I'm iterating on. Meanwhile the code doesn't work on TPUs right now because it hangs before initialization. (This is with PyTorch for what it's worth) All that
45.
by popinman322 3y ago
> That rising inequality is leading to poverty is a failure of government, not corporations or capitalism. Why introduce a layer of indirection by blaming the rich or corporations, when the rules are made by government? Blame the governm
46.
by popinman322 3y ago
There was a discussion about the paper[0] this was based on 7 days ago: https://news.ycombinator.com/item?id=35768824 [0] https://arxiv.org/abs/2304.15004
47.
by popinman322 3y ago
Tangentially related: A flat tech co-op model could be good. Build tools to make starting tech companies much easier, then let members start companies with those tools for either some regular payment in either labor (eg. allocate headcount
48.
by popinman322 4y ago
I also liked the graph that showed only training loss vs Adam. Would have been nice to see validation loss as well; are we just helping Adam overfit the data?
49.
by popinman322 4y ago
Both violate user privacy and imply a need for better data privacy protections. One was targeted and the other was not, but they are in the same class of issues even if the severity is different.
50.
by popinman322 4y ago
As someone who's applied before, you really need to know C. The first phase of the process is essentially a code comprehension exercise followed by a simple programming exercise. I think they're looking primarily for candidates wi
51.
by popinman322 4y ago
> I googled "fair use copyright US" and have now decided that US copyright law is stupid. Fair use helps artists, journalists, and (indirectly) the general public. It prevents censorship of critical or opposing views, among a l
52.
by popinman322 4y ago
I didn't read it as a move towards centralizing data, but instead as working with companies to federate over their userbase or between a collection of companies.
53.
by popinman322 4y ago
NeRFs are a form of inverse renderer; this paper uses Score Jacobian Chaining[0] instead. Model reconstruction from NeRFs is also an active area of research. Check out the "Model Reconstruction" section of Awesome NeRF[1]. From th
54.
by popinman322 4y ago
Is there something I'm missing? ChatLlama doesn't reference any human feedback datasets. > You can of course also cross train it using actual ChatGPT. You mean train it on ChatGPT's output? That's against OpenAI'
55.
by popinman322 4y ago
You're likely right that applying RLHF (+ fine-tuning with instructions) to LLaMA 7b would produce results similar to ChatGPT, but I think you're implying that that would be feasible today. RLHF requires a large amount of human fe
56.
by popinman322 4y ago
> It's a huge minus. The fact that all shell tools produce text is a basic tenet of shell programming which underlies the composability and interoperability of them all. PowerShell is essentially a cut down .NET environment with som
57.
by popinman322 4y ago
> Once that dataset gets replicated, we can start talking about size. Hopefully LAION will deliver. Is LAION starting a community project to rate model outputs? I didn't see anything on their site.
58.
by popinman322 4y ago
Seems like e/client and e/server are for when you'd like to force sections of code to run on one side or the other. Otherwise the compiler decides what lives where?
59.
by popinman322 4y ago
IIRC we still don't have very competitive neural network models for time series forecasting. It's a very active area of research.
60.
by popinman322 4y ago
> I think for full stops, it's like the difference between familiar language and polite language. If you are too polite with a close friend you give off the impression that you're not close enough for casual language, which can

More ›