msp26

19 ms·

241.
by msp26 3y ago
>I can make GPT4 return valid JSON simply by providing examples in the system message. This works nine times out of ten But you can do both. For my current use case of extracting information from articles, I have a json schema + one&#x2F
242.
by msp26 3y ago
No mention of reverse image search sadly. I've been looking for anything better than Google/yandex forever.
243.
by msp26 3y ago
It isn't trivial. That's the whole problem. Luckily my use cases have a manual check built in but even a proxy for confidence would be amazing.
244.
by msp26 3y ago
I would take unlobotomised models with some people whining about them saying naughty things over the current situation very easily.
245.
by msp26 3y ago
I think it's ridiculous how much dev time and money is spent in pursuit of this thing that a good chunk of users do not care about or actively despise.
246.
by msp26 3y ago
It's interesting that you bring up Dota because I find that it's a ready example of the vocab point failing. When you listen to pros talk, they misuse words and concepts all the time but it doesn't matter because the other pr
247.
by msp26 3y ago
You're overrating the influence of the name Microsoft here. It's just some devs from the company working on this with no proper guarantee backing the project. I've been through this whole song and dance already with Microsoft
248.
by msp26 3y ago
I do have some guidance code (running on a more universal wrapper I made for LLMs) sitting in my prompts directory. I'm not going to throw it out but I don't see the point in maintaining it.
249.
by msp26 3y ago
I wouldn't rely on guidance. I've been waiting for them to approve a commit for something as basic as token counts for OAI models for weeks now. It's just not actively developed enough for serious use cases.
250.
by msp26 3y ago
"We have also trained 34B variants, which we report on in this paper but are not releasing." "We are delaying the release of the 34B model due to a lack of time to sufficiently red team." From the Llama 2 paper
251.
by msp26 3y ago
>The 4k context length is nice, but RoPE makes it irrelevant anyway. Can you elaborate on this?
252.
by msp26 3y ago
The output is not 100% guaranteed. Be careful about that and have another layer to check the output. I had a schema with a string enum property to categorise some inputs. One of the category names was "media/other" or somethi
253.
by msp26 3y ago
If you're using OAI function calling, you can define a json schema with boolean values. If you have a handful of values, you can use an enum with a list of possible values. Unless your prompt seriously conflicts with the schema, it&#x2
254.
by msp26 3y ago
It's company code, I don't think I can open-source it. Happy to answer any questions/discuss prompt benchmarking though.
255.
by msp26 3y ago
Currently working on something similar for myself, this doesn't seem to fit my needs (benchmarking generations too rather than just classification). I only have a crude cosine similarity metric for accuracy for now. Also I'm using
256.
by msp26 3y ago
probably just easier to use drm-free copies of books
257.
by msp26 3y ago
It was probably just pdf copies of the book from random a random ddl website.
258.
by msp26 3y ago
System/assistant/user prompting
259.
by msp26 3y ago
>20% What a waste of compute for this entire circus.
260.
by msp26 3y ago
Any advice for people building the second layer stuff?
261.
by msp26 3y ago
>Refuel provides LLMs that can compute confidence scores for every label, if the LLM you've chosen doesn't provide token-level log probabilities. How does this work exactly?
262.
by msp26 3y ago
I don't like to use the word just often but when I say this in relation to eating, it's as a response to the ridiculous diets and other restrictions people force on themselves to lose weight.
263.
by msp26 3y ago
Any tips for someone currently working to create one of these systems?
264.
by msp26 3y ago
I've gotten actual sources and explanations I used for my master's from /h/. I don't even go on anything there except /hdg/. Surprisingly helpful bunch.