3 ms·
That's very insightful. Quick question, since you are doing this in production - how are you serializing images into a message ? Base64 or something else.
by sandGorgon 10y ago
That's very insightful. Quick question, since you are doing this in production - how are you serializing images into a message ? Base64 or something else.
- asadjb 10y agoWe just do a str(file.read()). Not the most elegant solution. :)
- zo1 10y agoNot the person you're asking, but from the experiences I've had with Celery + other messaging queues: Don't pass around large binary blobs if you can avoid it. Whether that is an image or something else is irrelevant. You can do such things as passing a database unique key, GUID, or file-path to the raw data on disk. Obviously, you will also need to engineer around that if you've got a distributed system. The tangential benefit is that you're not using a "messaging" queue or system for persisting or semi-persisting your image data. That's a big no-no as such systems are transient in nature and that often doesn't align with binary or image processing. Base64 is for when you want to pass around binary in a text-based format. E.g. XML or JSON. But do keep in mind that because of the encoding format, converting binary data to base64 does increase the payload size by about 15% or so.
- brianwawok 10y agoYou really should not ram images in your task queue. Send them off to S3 or Cloud Storage with a GUID, and send along the GUID however you want (JSON is fine)