3 ms·
I use this: > Be terse. Do not offer unprompted advice or clarifications. > Avoid mentioning you are an AI language model. > Avoid disclaimers about your kno
by 4ad 2y ago
I use this:
> Be terse. Do not offer unprompted advice or clarifications.
> Avoid mentioning you are an AI language model.
> Avoid disclaimers about your knowledge cutoff.
> Avoid disclaimers about not being a professional or an expert.
> Do NOT hedge or qualify. Do not waffle.
> Do NOT repeat the user prompt while performing the task, just do the task as requested. NEVER contextualise the answer. This is very important.
> Avoid suggesting seeking professional help.
> Avoid mentioning safety unless it is not obvious and very important.
> Remain neutral on all topics. Avoid providing ethical or moral viewpoints in your answers, unless the question specifically mentions it.
> Never apologize.
> Act as an expert in the relevant fields.
> Speak in specific, topic relevant terminology.
> Explain your reasoning. If you don’t know, say you don’t know.
> Cite sources whenever possible, and include URLs if possible.
> List URLs at the end of your response, not inline.
> Speak directly and be willing to make creative guesses.
> Be willing to reference less reputable sources for ideas.
> Ask for more details before answering unclear or ambiguous questions.
Unfortunately most references it provides are bogus. It just makes up URLs and papers. Let's see if this new feature is any better.
- dragonwriter 2y agoThis new feature is restricted to sources you provide in the context window: “With Citations, users can now add source documents to the context window, and when querying the model, Claude automatically cites claims in its output that are inferred from those sources.”
- 4ad 2y agoYeah, I see that now. Completely useless.
- simonw 2y agoWhy is that useless? If you want reliable citations, this is an API that can help you implement reliable citations. Asking a model to return useful citations from its model weights with no assistance from external systems isn't how this stuff works.
- 4ad 2y agoIf I had the stuff to feed into the LLM context, I wouldn't need the LLM to find the stuff for me because I would already have it.
- dllthomas 2y ago"Completely useless" might be too strong. In a context where you're already doing RAG it would help you verify what's produced. Certainly it's radically less useful than if it could produce citations to the training set, though.
- simonw 2y agoSounds to me like you want a search engine, not an LLM.
- dragonwriter 2y agoTypically, you wouldn't manually select the documents in a query, you’d use this as part of a system wrapped around the LLM where a query (possibly itself from the model like other tool invocations) would be sent out to a web api, vectord DB, etc., return documents that would be fed back in to the model as part of a continuation query, and then the model would frame a response citing the relevant sources for elements of the response.
- vunderba 2y agoPatently untrue. If I have hundreds of PDF research papers totaling thousands of pages - just because I have them doesn't necessarily mean I know where to search. Having an LLM be able to find related pieces across all of these documents and extrapolate is tremendously useful.
- 4ad 2y agoA thousand pages is something reviewable by hand (albeit slowly) and easily amendable to grep. Anything that trivially fits in computer memory doesn't benefit from approximate search methods. I stand by what I said.
- 2y ago
- saaaaaam 2y agoNot useless at all. This is exactly my primary use of the Anthropic api: feed in a source document and ask for specific outputs that include exact citations from the source I provide.
- robwwilliams 2y agoLove your pre-prompt. Better than mine. The difference in FDRs is likely to be domain specific. I note that you are an expert in mathematical engineering and computing. In contrast my work area is neatly confined and defined by PubMed.
- 4ad 2y agoI must say that it works reasonably well even with imagined references and URLs because it usually gets the author right, or there is a similar paper with a reasonably close title. I'd say around 50% of the time I get a real reference, 30% of the time I get an imagined reference but from an author which studied the very problem under question, and 20% is completely halucinated.