AWS Bedrock. Set it up to experiment with different models in one place. Turned to be fascinating; UI is far more friendly than AWS usually is. Cool stuff: Gua

AWS Bedrock. Set it up to experiment with different models in one place. Turned to be fascinating; UI is far more friendly than AWS usually is. Cool stuff: Guardrails: No longer do we have to use the AI to fight AI, AWS has presets for blocking inappropriate content going into or out of the AI block. Evaluation: Basically just give it a JSONL file of all the inputs and expected outputs, and compare two models or judge the quality of a model. This can be done by a human (it even manages the access of this team of humans). But more interestingly, it can be automated or done by AWS workers. Knowledge base and agents: Seems like it's only for Claude, but damn, Claude has been impressive for a good price. Claude Sonnet (the mid-range) has been about as good as GPT-4 for half the price. The weakness has been things like knowledge base and AWS lets us just quickly set up vector DBs and embeddings. Then the agents feature lets us connect with it and combine it with all of the above.