4 ms·

So much this. Same goes with all the Kafka usage, making development literally Kafkaesque. The data eventually flows through your microservice landscape, being

So much this. Same goes with all the Kafka usage, making development literally Kafkaesque. The data eventually flows through your microservice landscape, being eventually consistent and performance issues become a whole different nightmare. Yea, maybe I just didn't read enough of the documentation but maybe your 1500 users per month web-app doesn't need 7 microservices and Kafka as single source of truth, all hold together by 8 developers. It's astonishing how many teams want to use cool tech instead of create a great tool for users or how many teams think that technology will solve bad management.

holoduke 5y ago
I migrated a unmaintainable over engineered java spring application consisting of around 20 micro services using Kafka all over the place to one single Rust application. A relief for anyone in the company.
- leetrout 5y ago
  Just you alone? Do others in the company know Rust? I really like the idea of Rust I just worry about how things shake out In The Real World(tm). I have no doubt Rust is here to stay for a while so I think it’s pretty solid play when appropriate.
- sk5t 5y ago
  Are you saying you rewrote not just the Spring application but all of the micro services as well?
  - dopidopHN 5y ago
    I understood it as a distributed spring application.
- dominotw 5y ago
  What happened to the people who worked on those 20 microservices. What do they do now?
  - holoduke 5y ago
    A few walked away. They did not agree with the move. But the company was really in the need for a radical change since development was stuck in complexity and maintainance.
- BurningFrog 5y ago
  One reason this can be doable is that working in the 20 microservice environment is so unproductive that the original app probably did very little.
  - papito 5y ago
    Exactly. Making sure those services work in production, and the constant troubleshooting in local environments as well would easily be most of the developers' days.
Swinx43 5y ago
So much this! As someone that regularly deals with data platforms and data engineering issues this nails it. Today’s desire to make everything event driven and “real-time” boggles the mind. Solutions are contorted to fit the Kafka operating model. The inevitable result is blog posts about how “we solved our self inflicted problems that would have never existed if we chose a sane design for our real problem. “ Oh and never forget that no senior stake holder ever really checks their info/dashboards in real time. Never mind actually making decisions in real time based on it.
- dominotw 5y ago
  Same! My dataengineering team wants everything to be moved from batch oriented to 'stream oriented'. We are replacing our batch workflows to flink/spark streaming. But no one has been able to answer me what business use cases need streaming or what if any problem is being solved by this added level of complexity.
  - disgruntledphd2 5y ago
    Sometimes, streaming makes sense. When I was at a FAANG, we had a sampled streaming database with nice plots (kinda, ugly but functional). This was incredibly useful as it allowed teams to figure out what had happened in prod without needing to wait for the next day's batch jobs. OTOH, most companies don't need something this complicated, and I'd imagine you could do the same with MySQL/Postgres if you're not at FAANG scale.
    - Swinx43 5y ago
      I agree. When it is the right solution to the problem at hand there is nothing better than a good streaming solution. It goes for pretty much all technology, that you need to choose the right tool for the job. For some reason the marketing just gets most people all worked up and ready to fit problems into technology that was never designed for their problems.

holoduke 5y ago

I migrated a unmaintainable over engineered java spring application consisting of around 20 micro services using Kafka all over the place to one single Rust application. A relief for anyone in the company.

leetrout 5y ago
Just you alone? Do others in the company know Rust? I really like the idea of Rust I just worry about how things shake out In The Real World(tm). I have no doubt Rust is here to stay for a while so I think it’s pretty solid play when appropriate.
sk5t 5y ago
Are you saying you rewrote not just the Spring application but all of the micro services as well?
- dopidopHN 5y ago
  I understood it as a distributed spring application.
dominotw 5y ago
What happened to the people who worked on those 20 microservices. What do they do now?
- holoduke 5y ago
  A few walked away. They did not agree with the move. But the company was really in the need for a radical change since development was stuck in complexity and maintainance.
BurningFrog 5y ago
One reason this can be doable is that working in the 20 microservice environment is so unproductive that the original app probably did very little.
- papito 5y ago
  Exactly. Making sure those services work in production, and the constant troubleshooting in local environments as well would easily be most of the developers' days.

Swinx43 5y ago

So much this! As someone that regularly deals with data platforms and data engineering issues this nails it. Today’s desire to make everything event driven and “real-time” boggles the mind. Solutions are contorted to fit the Kafka operating model. The inevitable result is blog posts about how “we solved our self inflicted problems that would have never existed if we chose a sane design for our real problem. “ Oh and never forget that no senior stake holder ever really checks their info/dashboards in real time. Never mind actually making decisions in real time based on it.

dominotw 5y ago
Same! My dataengineering team wants everything to be moved from batch oriented to 'stream oriented'. We are replacing our batch workflows to flink/spark streaming. But no one has been able to answer me what business use cases need streaming or what if any problem is being solved by this added level of complexity.
- disgruntledphd2 5y ago
  Sometimes, streaming makes sense. When I was at a FAANG, we had a sampled streaming database with nice plots (kinda, ugly but functional). This was incredibly useful as it allowed teams to figure out what had happened in prod without needing to wait for the next day's batch jobs. OTOH, most companies don't need something this complicated, and I'd imagine you could do the same with MySQL/Postgres if you're not at FAANG scale.
  - Swinx43 5y ago
    I agree. When it is the right solution to the problem at hand there is nothing better than a good streaming solution. It goes for pretty much all technology, that you need to choose the right tool for the job. For some reason the marketing just gets most people all worked up and ready to fit problems into technology that was never designed for their problems.