4 ms·
Thank you for your kind words! Valid point. I'm actually already working on testing better streaming using async-http-faraday, which configures the default ada
by earcar 2y ago
Thank you for your kind words!
Valid point. I'm actually already working on testing better streaming using async-http-faraday, which configures the default adapter to use async_http with falcon and async-job instead of thread-based approaches like puma and SolidQueue. This should significantly improve resource efficiency for AI workloads in Ruby - something I'm not aware is implemented by other major Ruby LLM libraries. The current approach with blocks is idiomatic Ruby, but the upcoming async support will make the library even better for production use cases. Stay tuned!