6 ms·

"In Python, most of the bottleneck comes from having to fetch and deserialize Redis data." This isn't a fair comparison. Of freaking course postgres would be f

by chaps 4y ago

"In Python, most of the bottleneck comes from having to fetch and deserialize Redis data." This isn't a fair comparison. Of freaking course postgres would be faster if it's not reaching out to another service.

montanalow 4y ago
As a contributor, I think it's interesting when comments focus on the language (Python vs Rust) vs the architecture (local vs remote). Inference is embarrassingly parallelizable, with Python Flask or Postgres replicas. I think the interesting thing is that data retrieval costs tend to dominate other costs, and yet are often ignored. ML algorithms get a lot focus and hype. Data retrieval, not as much.
- chaps 4y ago
  For anyone who skips the intro and just goes to the results, this is what they see: https://imgur.com/tEK73e8 https://imgur.com/tEK73e8 A suggestion: clean up the blog post's charts and headers to make it much, much more clear that what's being compared isn't python vs postgresml.
  - montanalow 4y ago
    Another suggestion: Don't build you identity around a language or platform. They come and go. Except SQL. It's been around for longer than either of us.
    - chaps 4y ago
      Agreed, which is why I use postgres for most of my work unless I can't avoid it.
- deepstack 4y ago
  that is reason many older developer tend to do everything biz logic etc all in db store procedure/functions/view, etc. The cost of getting the data is native, no connection pooling needed, and with V8/python integration in the PG, it is non trivial what language you use. If you are dealing with large amount of data in a db, why not just do everything there. DB like sql has cursor, merge, that makes manipulating large set of data much easier than moving it on to another language environment.
redhale 4y ago
Yes, that's essentially the point being made here. It's a fair comparison if your intent is to run this kind of job as quickly as possible.
- chaps 4y ago
  No it's not. It tells me exactly nothing useful about the postgresml performance because it's impossible for me to rule out redis and the http server when factoring in performance. It's two hops, with a guaranteed delay that the postgres setup won't have. If they wanted it to be a fair comparison they should have used FDWs to connect to the same redis and http server that the python benchmarks tested against.
  - vasco 4y ago
    The point is you don't need those hops if you use postgresML. It's like if I told you to move to a place where you can walk 5mins to work, and you tell me it's not a fair comparison because right now you have to drive to the station and then get on a train and you're interested in a comparison where you walk to the train instead. You don't need the train because you're already there! You don't need the network hops exactly because the data is already there in the right way.
    - chaps 4y ago
      I get the point of the post, but I still don't see how it's remotely useful to understand the performance in postgresml, as someone who's interested in using it for my own tooling. Maybe I don't spend enough time in ML space to know how often they use HTTP/redis in their flows as much as they do. Most of my stuff is just data on-disk, where adding two additional services would be embarrassingly overkill. Don't you think it would be incredibly useful as a baseline if they included a third test with FDWs against redis and the http server?
      - theamk 4y ago
        Are there any other FDWs that do ML inference? Remember, this is not plain file serving -- this is actually invoking XGBoost library which does complex mathematical operations. The user does not get data from disk, they get inference results. Unless you know of any other solution which can invoke XGBoost (or some other inference library), I don't see anything "embarrassingly overkill" there.
ta2234234242 4y ago
Further in their methodology they wrap a microservice around python and redis (which is doing an unmarshalling from redis and a marshalling to json) but they're not doing that with postgres. In fact, as far as I can tell, postgres is not running as a microservice here. The data still has to be marshalled into some output other services can use.

by chaps 4y ago

montanalow 4y ago

As a contributor, I think it's interesting when comments focus on the language (Python vs Rust) vs the architecture (local vs remote). Inference is embarrassingly parallelizable, with Python Flask or Postgres replicas. I think the interesting thing is that data retrieval costs tend to dominate other costs, and yet are often ignored. ML algorithms get a lot focus and hype. Data retrieval, not as much.

chaps 4y ago
For anyone who skips the intro and just goes to the results, this is what they see: https://imgur.com/tEK73e8 https://imgur.com/tEK73e8 A suggestion: clean up the blog post's charts and headers to make it much, much more clear that what's being compared isn't python vs postgresml.
- montanalow 4y ago
  Another suggestion: Don't build you identity around a language or platform. They come and go. Except SQL. It's been around for longer than either of us.
  - chaps 4y ago
    Agreed, which is why I use postgres for most of my work unless I can't avoid it.
deepstack 4y ago
that is reason many older developer tend to do everything biz logic etc all in db store procedure/functions/view, etc. The cost of getting the data is native, no connection pooling needed, and with V8/python integration in the PG, it is non trivial what language you use. If you are dealing with large amount of data in a db, why not just do everything there. DB like sql has cursor, merge, that makes manipulating large set of data much easier than moving it on to another language environment.

redhale 4y ago

Yes, that's essentially the point being made here. It's a fair comparison if your intent is to run this kind of job as quickly as possible.

chaps 4y ago
No it's not. It tells me exactly nothing useful about the postgresml performance because it's impossible for me to rule out redis and the http server when factoring in performance. It's two hops, with a guaranteed delay that the postgres setup won't have. If they wanted it to be a fair comparison they should have used FDWs to connect to the same redis and http server that the python benchmarks tested against.
- vasco 4y ago
  The point is you don't need those hops if you use postgresML. It's like if I told you to move to a place where you can walk 5mins to work, and you tell me it's not a fair comparison because right now you have to drive to the station and then get on a train and you're interested in a comparison where you walk to the train instead. You don't need the train because you're already there! You don't need the network hops exactly because the data is already there in the right way.
  - chaps 4y ago
    I get the point of the post, but I still don't see how it's remotely useful to understand the performance in postgresml, as someone who's interested in using it for my own tooling. Maybe I don't spend enough time in ML space to know how often they use HTTP/redis in their flows as much as they do. Most of my stuff is just data on-disk, where adding two additional services would be embarrassingly overkill. Don't you think it would be incredibly useful as a baseline if they included a third test with FDWs against redis and the http server?
    - theamk 4y ago
      Are there any other FDWs that do ML inference? Remember, this is not plain file serving -- this is actually invoking XGBoost library which does complex mathematical operations. The user does not get data from disk, they get inference results. Unless you know of any other solution which can invoke XGBoost (or some other inference library), I don't see anything "embarrassingly overkill" there.

ta2234234242 4y ago

Further in their methodology they wrap a microservice around python and redis (which is doing an unmarshalling from redis and a marshalling to json) but they're not doing that with postgres. In fact, as far as I can tell, postgres is not running as a microservice here. The data still has to be marshalled into some output other services can use.