4 ms·

FastStream: Effortless Kafka/RabbitMQ Event Stream Integration for Microservices

10 points by kumaranvpl 3y ago

kumaranvpl 3y ago
FastStream simplifies the process of writing producers and consumers for message queues, handling all the parsing, networking and documentation generation automatically. Here's a look at the core features that make FastStream a go-to framework for modern, data-centric microservices: - Multiple Brokers: FastStream provides a unified API to work across multiple message brokers (Kafka, RabbitMQ support) - Pydantic Validation: Leverage Pydantic's validation capabilities to serialize and validates incoming messages - Automatic Docs: Stay ahead with automatic AsyncAPI documentation - Intuitive: Full-typed editor support makes your development experience smooth, catching errors before they reach runtime - Powerful Dependency Injection System: Manage your service dependencies efficiently with FastStream's built-in DI system - Testable: Supports in-memory tests, making your CI/CD pipeline faster and more reliable - Extendable: Use extensions for lifespans, custom serialization and middlewares - Integrations: FastStream is fully compatible with any HTTP framework you want (FastAPI especially) - Built for Automatic Code Generation: FastStream is optimized for automatic code generation using advanced models like GPT and Llama. Please check out the tutorial available here: https://faststream-gen.airt.ai/Tutorial/Cryptocurrency_Tutorial/ https://faststream-gen.airt.ai/Tutorial/Cryptocurrency_Tutor...
ivoras 3y ago
Does it support multiple brokers within the same projects? As in can I use both Kafka and RabbitMQ with it?
- Lancetnik 3y ago
  Not natively, but you have a tools to run multiple brokers manually (run them together at lifespan hooks). In the nearest future we will add this feature at the framework core.
- davorrunje 3y ago
  It is an interesting feature request, I create an issue that you can follow the progress: https://github.com/airtai/faststream/issues/758 https://github.com/airtai/faststream/issues/758
ainadan 3y ago
Looks great! Could FastStream be suitable choice for running ML inferences with improved memory management and congestion control compared to REST APIs?
- davorrunje 3y ago
  Actually, serving ML inferences was the initial motivation for building the library. We couldn’t find anything similar to FastAPI for Apache Kafka and then we decided to build one.
hacker00007 3y ago
does it support pydantic v1?
- Lancetnik 3y ago
  Sure, by default it installs PydanticV2, but you can downgrade it manually - all works fine. We are testing it with bith versions https://github.com/airtai/faststream/actions/runs/6312083287/job/17137378871 https://github.com/airtai/faststream/actions/runs/6312083287...
pythrick 3y ago
How FastStream benefits from the usage of asyncio? Is it able to concurrently process another message while a message awaits for I/O operation? Is this the default behavior, or does it requires some parameter tweak? Does it behaves differently when using FastStream as a FastAPI plugin/addon? I've being using Propan for a while, and planning to move to FastStream asap. Great tool btw, thank you for all your combined effort now. Congrats!

10 points by kumaranvpl 3y ago

FastStream simplifies the process of writing producers and consumers for message queues, handling all the parsing, networking and documentation generation automatically. Here's a look at the core features that make FastStream a go-to framework for modern, data-centric microservices: - Multiple Brokers: FastStream provides a unified API to work across multiple message brokers (Kafka, RabbitMQ support) - Pydantic Validation: Leverage Pydantic's validation capabilities to serialize and validates incoming messages - Automatic Docs: Stay ahead with automatic AsyncAPI documentation - Intuitive: Full-typed editor support makes your development experience smooth, catching errors before they reach runtime - Powerful Dependency Injection System: Manage your service dependencies efficiently with FastStream's built-in DI system - Testable: Supports in-memory tests, making your CI/CD pipeline faster and more reliable - Extendable: Use extensions for lifespans, custom serialization and middlewares - Integrations: FastStream is fully compatible with any HTTP framework you want (FastAPI especially) - Built for Automatic Code Generation: FastStream is optimized for automatic code generation using advanced models like GPT and Llama. Please check out the tutorial available here: https://faststream-gen.airt.ai/Tutorial/Cryptocurrency_Tutorial/ https://faststream-gen.airt.ai/Tutorial/Cryptocurrency_Tutor...

ivoras 3y ago

Does it support multiple brokers within the same projects? As in can I use both Kafka and RabbitMQ with it?

Lancetnik 3y ago
Not natively, but you have a tools to run multiple brokers manually (run them together at lifespan hooks). In the nearest future we will add this feature at the framework core.
davorrunje 3y ago
It is an interesting feature request, I create an issue that you can follow the progress: https://github.com/airtai/faststream/issues/758 https://github.com/airtai/faststream/issues/758

ainadan 3y ago

Looks great! Could FastStream be suitable choice for running ML inferences with improved memory management and congestion control compared to REST APIs?

davorrunje 3y ago
Actually, serving ML inferences was the initial motivation for building the library. We couldn’t find anything similar to FastAPI for Apache Kafka and then we decided to build one.

hacker00007 3y ago

does it support pydantic v1?

Lancetnik 3y ago
Sure, by default it installs PydanticV2, but you can downgrade it manually - all works fine. We are testing it with bith versions https://github.com/airtai/faststream/actions/runs/6312083287/job/17137378871 https://github.com/airtai/faststream/actions/runs/6312083287...

pythrick 3y ago

How FastStream benefits from the usage of asyncio? Is it able to concurrently process another message while a message awaits for I/O operation? Is this the default behavior, or does it requires some parameter tweak? Does it behaves differently when using FastStream as a FastAPI plugin/addon? I've being using Propan for a while, and planning to move to FastStream asap. Great tool btw, thank you for all your combined effort now. Congrats!