3 ms·

I'd argue that this article could be read more like "start as a monolith, and then move towards microservices when sensible", except the penny hasn't fully drop

I'd argue that this article could be read more like "start as a monolith, and then move towards microservices when sensible", except the penny hasn't fully dropped for the author that the sensible time for their organisation is right now. The company appears dogmatically locked into their idling python code, their SQL making unpredictable commits, and their SQL framework making it difficult to make schema migrations. This is a financial services company describing data-integrity bugs in their production platforms.

mousetree 3y ago
These are all specific problems that can be individually solved. Personally I don’t see how changing their architecture or programming language would solve those
- nijave 3y ago
  In addition, they risk introducing new problems they already solved with their current setup
wavemode 3y ago
In what way does "moving towards microservices" relate to solving "data-integrity bugs"? I would argue the opposite - the more distributed you make the system, the more subtle consistency bugs tend to creep in.
- DonnyV 3y ago
  For a financial services company they should be using a compiled language. Something like C# or Java or Rust or Go with Postgres.
  - quaunaut 3y ago
    Why? What inherent advantage do those languages have with financial logic?
    - notpachet 3y ago
      Better static analysis, for one.
      - quaunaut 3y ago
        And that helps in financial analysis how?
        notpachet 3y ago
        I didn't say anything about financial analysis. But static typing is a good defense mechanism against type errors at runtime (I don't think anyone would argue against that). When you're running a financial services product, the cost of a type error in production can be dramatically higher than in non-financial code. Speaking from firsthand experience.
        IshKebab 3y ago
        I don't think it's specific to financial services. He just meant that financial services is an area where you really don't want bugs! And one of the easiest ways to eliminate entire classes of bugs is to use static typing.
      - jakewins 3y ago
        We have services in Python and C# at $dayjob, and my experience is that Python has mostly left C# behind here - what is something you can express in the C# type system that MyPy cannot statically check? Conversely, I find myself constrained by C#'s typesystem not having features now widely popular from Rust and Typescript and available in modern Python: const generics / literals, protocols/structural interfaces and matching over algebraic datatypes, to name a few examples.
        neonsunset 3y ago
        Was bewildered when reading this message. How come? Python and TypeScript by definition don't have a concept of const generics the way it applies to C++ templates or Rust. Nor "matching over algebraic datatypes" made any sense - sure sounds slightly below average fancy but comes down to just using C# pattern matching effectively, which the author may have never done. But then it struck me "structural interfaces sure sounds familiar", I go check the pinned repositories in the github profile and it all made sense immediately - the pinned ones are written in Go, which explains the author's woes with confidently navigating something more sophisticated.
  - artimaeis 3y ago
    He's been talking about the problems/solutions around Wave for a while. Every time I can't help but think if they'd just started with C# or Java, maybe Go or Rust, they'd be in a better position. Here's what I see them as providing: - Simple async, no wasted CPU on I/O - Strong typing defaults - Reliable static analysis Half of the "pros" he lists for using GraphQL are just provided out of the box using ASP.NET Core with NSwag to document the endpoints. If they want to keep the client-side composability, create some OData endpoints and you've got it. > Self-documentation of exact return type > Code generation of exact return type leads to safer clients > Our various apps (user app, support app, Wave agent app, etc.) can mostly share one API, reducing complexity > Composable query language allows clients to fetch exactly the data they need in a single packet roundtrip without needing to build a large number of special-purpose endpoints Bias flag: I'm primarily a C#/.NET developer. I came up on python and ruby but have had more success in my region getting paid to work on .NET code.
lstodd 3y ago
Where does the "idling python code" come from? If it blocks, it blocks, that's not "idling". And I doubt they are running a process per core.
- deleted 3y ago
  [deleted]
jakewins 3y ago
The author is ex Microsoft, Google and was a senior staff eng at Twitter; i don’t know them but my experience with their blogging over the last decade is that they seem generally very well informed. To me it seems unlikely “the penny hasn’t dropped”, like you say. On the actual criticism you’re raising: In what concrete way would moving to micro services help with reducing data integrity bugs? In my experience I’d expect the exact opposite - micro services systems generally being harder to test and generally having more possible interleavings and partial failure paths.
whiterknight 3y ago
Daily reminder that when one process is blocked others can be working so nothing is “idling” on a web server properly configured.

mousetree 3y ago

These are all specific problems that can be individually solved. Personally I don’t see how changing their architecture or programming language would solve those

nijave 3y ago
In addition, they risk introducing new problems they already solved with their current setup

wavemode 3y ago

In what way does "moving towards microservices" relate to solving "data-integrity bugs"? I would argue the opposite - the more distributed you make the system, the more subtle consistency bugs tend to creep in.

DonnyV 3y ago
For a financial services company they should be using a compiled language. Something like C# or Java or Rust or Go with Postgres.
- quaunaut 3y ago
  Why? What inherent advantage do those languages have with financial logic?
  - notpachet 3y ago
    Better static analysis, for one.
    - quaunaut 3y ago
      And that helps in financial analysis how?
      - notpachet 3y ago
        I didn't say anything about financial analysis. But static typing is a good defense mechanism against type errors at runtime (I don't think anyone would argue against that). When you're running a financial services product, the cost of a type error in production can be dramatically higher than in non-financial code. Speaking from firsthand experience.
      - IshKebab 3y ago
        I don't think it's specific to financial services. He just meant that financial services is an area where you really don't want bugs! And one of the easiest ways to eliminate entire classes of bugs is to use static typing.
    - jakewins 3y ago
      We have services in Python and C# at $dayjob, and my experience is that Python has mostly left C# behind here - what is something you can express in the C# type system that MyPy cannot statically check? Conversely, I find myself constrained by C#'s typesystem not having features now widely popular from Rust and Typescript and available in modern Python: const generics / literals, protocols/structural interfaces and matching over algebraic datatypes, to name a few examples.
      - neonsunset 3y ago
        Was bewildered when reading this message. How come? Python and TypeScript by definition don't have a concept of const generics the way it applies to C++ templates or Rust. Nor "matching over algebraic datatypes" made any sense - sure sounds slightly below average fancy but comes down to just using C# pattern matching effectively, which the author may have never done. But then it struck me "structural interfaces sure sounds familiar", I go check the pinned repositories in the github profile and it all made sense immediately - the pinned ones are written in Go, which explains the author's woes with confidently navigating something more sophisticated.
- artimaeis 3y ago
  He's been talking about the problems/solutions around Wave for a while. Every time I can't help but think if they'd just started with C# or Java, maybe Go or Rust, they'd be in a better position. Here's what I see them as providing: - Simple async, no wasted CPU on I/O - Strong typing defaults - Reliable static analysis Half of the "pros" he lists for using GraphQL are just provided out of the box using ASP.NET Core with NSwag to document the endpoints. If they want to keep the client-side composability, create some OData endpoints and you've got it. > Self-documentation of exact return type > Code generation of exact return type leads to safer clients > Our various apps (user app, support app, Wave agent app, etc.) can mostly share one API, reducing complexity > Composable query language allows clients to fetch exactly the data they need in a single packet roundtrip without needing to build a large number of special-purpose endpoints Bias flag: I'm primarily a C#/.NET developer. I came up on python and ruby but have had more success in my region getting paid to work on .NET code.

lstodd 3y ago

Where does the "idling python code" come from? If it blocks, it blocks, that's not "idling". And I doubt they are running a process per core.

deleted 3y ago
[deleted]

jakewins 3y ago

The author is ex Microsoft, Google and was a senior staff eng at Twitter; i don’t know them but my experience with their blogging over the last decade is that they seem generally very well informed. To me it seems unlikely “the penny hasn’t dropped”, like you say. On the actual criticism you’re raising: In what concrete way would moving to micro services help with reducing data integrity bugs? In my experience I’d expect the exact opposite - micro services systems generally being harder to test and generally having more possible interleavings and partial failure paths.

whiterknight 3y ago

Daily reminder that when one process is blocked others can be working so nothing is “idling” on a web server properly configured.