4 ms·
Machinery: An asynchronous task/job queue for Golang
- rw 11y agoBadly titled. This is not an official library.
- richardknop 11y agoI'm sorry. I changed the title.
- bsaul 11y agoGreat ! I was just starting today to think about how something like that could be done in golang. I think a full example should involve the client part as well ( in order to remain typed along the chain, and have the json format describing the task an implementation detail). Edit : i didn't read properly, the readme also describes how a task is created. I didn't expect it to use a low level format though. Why not relying directly on the type of the task ( "AddTask") rather than asking for a "add" string parameter ?
- richardknop 11y agoThere is a very basic client implementation of client in examples/send/send.go. This is a very early stage project though. I agree that some parts of the API might need to be rethought and other approaches considered.
- jzelinskie 11y agoLooks like a good start, but from a quick glance it is missing some Go idioms. Some cursory recommendations: - No Makefile please, this project's build process isn't yet complicated enough to warrant one. - Rename the App type something more descriptive like Server - Rename InitApp to NewApp (NewXXX is an idiomatic Go constructor). Likewise with InitWorker - Make NewWorker a method of an App - Isolate the AMQPConnection into another library and use the design pattern in database/sql to allow users to create their own implementations of brokers - Do whatever you can to avoid empty interfaces (aka interface{}). You may have to be clever, but really avoid these at all costs. Using them basically mentions you'll have to sprinkle type assertions everywhere in the code rather than maybe having one place where you unmarshal values.
- richardknop 11y agoThank you very much for advice. Yes, I agree empty interfaces{} are a bad way of approaching this. I am thinking of using reflection in order to achieve this. I will need to think of some clever way to describe the types in the message so I can then use reflection to unpack JSON into proper types and pass them to a task (without making the API too overcomplicated). I will try to do this over next couple of weekends. Hopefully I can come up with some clever and clean API. Isolating AMQPConnection is also a great idea, I am planning to support multiple brokers (Redis, database etc) in the future.
- richardknop 11y agoI did a quick implementation of reflection today so arguments are not empty interfaces now and tasks are just simple functions with properly typed arguments. I will think more about simplifying it further.
- armon 11y agoThis looks pretty interesting! We use a library we've built called relay (https://github.com/armon/relay https://github.com/armon/relay) to do something similar also in Go. I like the higher level task based approach of this library, since this is a pretty common pattern when using task brokers.
- richardknop 11y agoLooks very interesting. I will look over the code in next couple of days :)
- jaytaylor 11y agoHow is this different from a simple async processing queue with Kafka?