6 ms·
AWS Bedrock is fairly easy. You can do it in 5 or 6 clicks. You have to upload your documents to S3, create a “Knowledge Base” then sync your documents into a
by benjaminwootton 3y ago
AWS Bedrock is fairly easy. You can do it in 5 or 6 clicks.
You have to upload your documents to S3, create a “Knowledge Base” then sync your documents into a vector database like OpenSearch or PineCone. You are then good to go via their playground or the AWS API.
I made a video here describing the process, check around 14 minutes in:
https://ensembleanalytics.io/blog/introducing-bedrock-knowledge-bases https://ensembleanalytics.io/blog/introducing-bedrock-knowle...
Bedrock is a decent product I think. All of the models in one place (apart from the big dogs from OpenAI) and a common API across them.
- hrdwdmrbl 3y agoIs there a limit? Could I create a knowledge base with 10,000 documents? 100k? 1M?
- 8organicbits 3y agohttps://docs.aws.amazon.com/bedrock/latest/userguide/quotas.html https://docs.aws.amazon.com/bedrock/latest/userguide/quotas....
- hrdwdmrbl 3y agoI’m sorry, I don’t understand those limits. It uses a lot of unfamiliar terms like “batch inference” and “modality”. I just want a nice UI that I can give my hard-drive to and then ask it questions.
- deleted 3y ago[deleted]
- nighthawk454 3y agoThat’s probably unrealistic at this time
- xuancanh 3y agoIt's doable with Amazon Q, but it's in Preview phase now. https://aws.amazon.com/q/ https://aws.amazon.com/q/
- hrdwdmrbl 3y agoIs Q the answer to this whole thread? People are talking about AWS Bedrock but that seems like something a startup would build upon and offer something like Q, eh?
- AndrewKemendo 3y agoI’m going to try it this week so maybe we have a follow up thread??
- lopkeny12ko 3y ago[flagged]
- ZephyrBlu 3y agoThis attitude puzzles me. "If you wish to make an apple pie from scratch, you must first invent the universe" energy. We are not always makers. Oftentimes we're consumers as well. I don't want to read documentation and experiment with my phone, I just want it to work out of the box and do what I expect. This is standard consumer behaviour and you're lying to yourself if you don't think you act like this with some things.
- benjaminwootton 3y agoThe documents are encoded as vectors and stored in a database, so I suspect it would be effectively unlimited. You would just pay for storage and compute. AWS OpenSearch has fairly good integration so you could look up costs for that. It’s not the cheapest AWS service to run and not exactly serverless as you pay by the hour.
- nashadelic 3y agoEven if you could, the problem is that these documents are first chunked into smaller pages, and then embeddings are created. When you ask a question, the algo searches for relevant chunks and passes them to the LLM's overall prompt. If there are too many chunks, or too many chunks with similar content, the search also coupled with the LLM's limited context window would mean only 1-3 chunks get passed. This isn't the same as the training data the LLM is trained on. As a result, it doesn't take advantage of the entire document set. So, if you have a billion documents, only 1-3 chunks will be picked for the final answer. When you know a question spans many many documents, the answer is never going to cover that. You could make a recursive also where you parse all the chunks, generate summaries of those and then pass them to the next chunk sequentially and so on. But you can imagine how expensive and slow that will be. It might still work for you but this is a very lossy approach.
- fennecbutt 3y agoEasy: if > n chunks returned query "too much information, ask user to clarify". If you have that much data in your vdb, and the user is querying a slice, it just won't do to have them ask something too generic, they need to be asked to be specific. Ie "what was that thing from last year?" "Sorry, could you be more specific?" "Oh uh the thing to do with the financial reports" "Financial reports, yes, what about them?" "It was something to do with the way we generate them" "Ah yes, the generation of financial reports; last year in December your team leader said on Slack...etc"
- tesdinger 3y agoHow does bedrock satisfy the non-hallucinating requirement?
- hhh 3y agoIt doesn't. You need to try and reduce hallucination as much as possible with your prompts, and then benchmark it.
- haolez 3y agoBedrock is cool, but I found it prohibitively expensive for hobbyists and small companies. On a first glance, it would cost me something like $ 5,000 per month for a simple trained model.
- benjaminwootton 3y agoI suspect most of the cost in the OpenSearch vector database? I agree it is high and it is not exactly serverless. You pay by the hour. Bedrock itself is charged based on tokens exchanged. I think Pinecone is a cheaper database for hobby and small business projects, though I haven't looked into it.
- rmbyrro 3y agoNo, if you use a custom model (trained with your data), you'll pay around $20 per hour, minimum. That equates to ~$15,000. You can reduce a lot by committing to a 6-month contract, but it won't get cheaper than about ~$5,000/mo. That's prohibitively expensive for small projects. Fine tuning GPT 3.5 is much cheaper.
- haolez 3y agoTo be fair, Bedrock is more flexible than OpenAI's fine tuning. It's nice to have at least the option to pay big bucks for this. But it's big bucks nonetheless.
- moltar 3y agoDon’t use OpenSearch it’s way overpriced. Use a cheap-o RDS with pgvector. Everything else is charged like any other LLM by token use.
- haolez 3y agoThis has nothing to do with OpenSearch. Using off the shelf models is cheap and pay as you go. Using custom models is what's expensive.