3 ms·
Note that if you don't have access to a server with a GPU, don't bother with Haystack since most of their USP functionality requires a GPU - and even that doesn
by axiosgunnar 5y ago
Note that if you don't have access to a server with a GPU, don't bother with Haystack since most of their USP functionality requires a GPU -
and even that doesn't mean the queries will execute fast - expect double digit seconds per query. YMMV
- antti909 5y agoFair enough. That's the inherent 'feature' of the Transformer models, though. We have a series of articles on optimization here https://medium.com/deepset-ai/accelerate-your-qa-system-with-gpus-6956d95f0954 https://medium.com/deepset-ai/accelerate-your-qa-system-with... - but also, no, double digits seconds is not necessarily the best result :) Our users have done much better <g>
- tholor 5y agoYep, you want a GPU for many of the models. However, then subsecond queries are easy to achieve. See benchmarks: https://haystack.deepset.ai/benchmarks/latest https://haystack.deepset.ai/benchmarks/latest