7 ms·

I initially directed (as organise stories, unblock people, do some devops & code review), but eventually I had to jump into the backend, because, as the most se

I initially directed (as organise stories, unblock people, do some devops & code review), but eventually I had to jump into the backend, because, as the most senior developer, I'm expected to jump into whatever part is having trouble. The backend had to receive events from a webhook and accumulate statistics on those events, then serve these statistics via REST API Frontend was plain old React reaching the REST API. Issues: * The database (MS SQL Server, as recommended by Microsoft) would randomly lock up for days. Bear in mind this was a 1GB of RAM (the minimum amount that Microsoft accepts as enough to run a SQL Server instance) RDS serving around 600 MB of data. * The introduction of the Repository Pattern (as recommended by the ASP.net documentation) introduced a lot of complexity (including incompleted/non-defined operations) that slowed down significantly the development. * LINQ supports interleaving DB operations with not-db operations, quickly becoming a O(n^m) footguns. If you make `row => row.x > 5` a method, LINQ will happily download all the rows and run the comparison in program instead of in the DB. Bear in mind I can't construct these footguns with django ORM, or make them typecheck with any Haskell data access framework. * The test runner UI would pretend no tests existed if the code didn't compile for any reason. Trying to debug something by running a specific failing test? If you miss a semicolon now you have to go find your test again. * The logger would log the message and the affected class in different lines. Do you want to grep for an error? You better know the obscure grep syntax for "previous line". Want to change the logging format? Good luck finding the code that adds the newline in order to find how to influence it. If you ask the IDE to find the implementation of the logging function it will show you the interface, nothing more. * You better not do some type of async stuff in LINQ, because the system is going to spawn a process for each entry, obliterating your production instance by creating thousands of processes, using a bunch of MB for each process. There were more issues, I didn't make an exhaustive list of them because, you know, I don't need to make exhaustive lists of malfunctions and bad design ideas for other programming languages.

hallway_monitor 5y ago
Based on this anecdote, it sounds like the team had little or no .net experience and your seniors probably don't deserve the title.
- javcasas 5y ago
  Curiously, I haven't had these problems in Python/cherrypy/django/flask, Haskell/snap/yesod, NodeJS/express/nextjs, so, maybe it's something beyond lack of .net experience.
  - tomrod 5y ago
    Your experience mirrors mine, both early (circa 2005) and more recently. Footguns are common. I am not sure where the .NET community coalesces (python and similar tend to hit up IRC).
    - topkai22 5y ago
      The ability to find and get answers on stack overflow is pretty impressive, although that might not be the “community” your looking for. I’ve had decent direct engagements with them framework dev tends on GitHub before as well.
      - tomrod 5y ago
        To add on to community -- it's more than that as well, it's the zeitgeist. In the scientific computing world, if something has hit NumFocus' portfolio then it's active, maintained, accessible, and (IMO) can consider adding into a tech stack. Similar for Apache for general FLOSS projects. I don't understand the dotnet's extension ecosystem -- how to find quality sourcing, etc. Going from an established project into dotnet SDK makes sense. But searching GitHub for useful dotnet frameworks -- stars are probably misleading here?
  - koide 5y ago
    There's been lots of bad decisions along the way for .net, but In its current state (.net Core 6) I can't really see any of your problems happening. Although what strikes me most is your first point. Running SQL server on 1gb and complaining that it didn't work well isn't really wise. Maybe this happened on the early years of .net Core or even on FF?
achandlerwhite 5y ago
For the benefit of others who might get the wrong idea, async/await definitely doesn’t spawn processes and in almost all cases not even extra threads.
- moomin 5y ago
  Pretty sure he was employing “Task.Run” which is… nearly always the wrong thing but does appear in a lot of the early examples.
Hawxy 5y ago
Was this the first time using .NET for most of your devs? > LINQ supports interleaving DB operations with not-db operations This hasn't been the case since EF Core 3.0, it will throw an exception if something can't be translated to prevent this problem. Prior to this (EF Core 2.2), a warning would be spat out if an in-memory operation occurs. Due to this, I assume this project was a number of years ago. > The logger would log the message and the affected class in different lines. The default logging format is unfortunate, although it can be made a single line in .NET 5 onwards. This issue would've been easily solved by using Serilog, swapping out the built-in logging with a few lines of code. I don't know the last time I encountered a .NET dev that wasn't at least aware of Serilog as an alternative.
- fomine3 5y ago
  Good to know for me who used EF5.
smaudet 5y ago
Keep in mind I've never done ASP.NET (only desktop development): The old C# stuff is indeed crusty, intermixed with hot nonsense from the Balmer days. Latest Framework version is I'd say OK, nothing to write home about, newer stuff is much better (.net 5, 6). I do think MS SQL Server is a bit, well, bad. But don't worry, irrelevant to the C# stuff - I used to use java servlets to query mssql, while it was better performing than mysql, I wouldn't pick it again. The anecdote about the React frontend bit is interesting, but ultimately depends on what the UI was doing. JS is certainly good to bang out some quick UI, but if it was shell for the backend, I'd say your app was either mis-designed or just naturally backend heavy. Again, haven't used (not in web dev right now), but Blazor exists for doing C# web UI, instead of complaining that the backend was bad/hard a better comparison would be between Blazor and React. Or you could switch architectures and do a non-component-based web UI - faster to put together a mashup of JS junk but it doesn't necessarily perform as well as doing a web client old-style - simple request response works for a lot of use-cases, its not as flashy, but it is fast.
Foobar8568 5y ago
The (Generic) Repository Patterns are some of the most anti-patterns created and goes fully agains't DRY and SOLID, it's like whole generations of developers are just on the same cargo and have never worked with a database( or datasource) having more than 1 table. So it looks great for a blog post, for a linkedin thought leader article, but when actually used, you end up duplicating the same shit over and over and over unless you use extension methods. But then why having the damn Repo patterns with the basic methods in the first place? These patterns are just awful and don't solve anything, abstraction of the DB? You don't need that pattern for that.
- manigandham 5y ago
  Agreed. Fortunately the main documentation now shows that this is an anti pattern since EF dbcontext is already a repository pattern and implements unit-of-work semantics.
  - NicoJuicy 5y ago
    I wanted to say exactly this.
  - DeathArrow 5y ago
    Well, please explain to me how can I switch the app from using Postgres to Redis or Cassandra with ease if I am not using something like a repository pattern? It's true, EF abstracts the database away by not requiring you to write SQL but EF is only usable with some SQL databases. So, having another abstraction layer might not be a bad idea.
    - javcasas 5y ago
      Yup, because we devs switch databases every week. That's the real reason why all projects are late: we are so busy switching databases all the time we don't have time for writing code!
    - kingmotley 5y ago
      dotnet add package EfCore.Cassandra. Then in startup.cs or program.cs, add this to your services: optionsBuilder.UseCassandra("Contact Points=127.0.0.1;", opt => { opt.MigrationsHistoryTable(HistoryRepository.DefaultTableName, "<schema name>"); }); See more here: https://efcorecassandra.readthedocs.io/en/latest/intro/getting-started-with-efcorecassandra.html https://efcorecassandra.readthedocs.io/en/latest/intro/getti...
manigandham 5y ago
Your last point is not how threading or async works in .NET. It’s a single process that has a thread pool that runs async Tasks. The rest of the comment indicates that the team didn’t have much or any experience with .NET. I’m sure there are issues with learning curve but the ecosystem is very capable (the corollary to being very productive) but you need to know how to properly use it.
- gregoryl 5y ago
  It also indicates the ego of the "best" senior dev who leaps in to save the day is quite large, and not very well correlated with ability and experience with the stack.
  - javcasas 5y ago
    We also tried hiring. After checking 100+ resumes, and interviewing the best 10 candidates, we hired the best we could find, who turned out to be average. My ego is not unfounded, I have had to do this successfully in other projects, but I can only do it if both the tools, the codebase and the framework allow me to do it
    - kingmotley 5y ago
      I would say it sounds unfounded to me. You "jumped" in to a different language, different database, different framework, and you landed up making junior level mistakes. Your explanations of what went wrong show clearly that you still don't have a grasp of how async/await works. So you don't understand the concept, which is now common across a number of languages including javascript (which you claim to know) and python (which you claim to know), dart, and scala. Based on your technical knowledge of languages you claim to know, I would say you are closer to a junior developer.
      - javcasas 5y ago
        You seriously want me to tell you how async, promises, callbacks, coroutines, closures+continuations, call/cc, and copies of the stack are actually points of view in the same concept? If I'm a junior, 99% of the professional programmers out there are simply not programmers at all.
DeathArrow 5y ago
All that sounds like poor app design. Microsoft doesn't force you to use SQL Server. I used Postgres and Redis for a quite big microservice based app. Repository isn't mandatory, however is nice to separate the business logic from the details of the data store or database. Using a repository shouldn't be a a problem unless you are doing it wrong. I've seen enough of poor repository pattern implementation. I would question the skills of the developers here. LINQ was not used properly if you had mixed IEnumerable and IQueryable based queries. It's not frameworks fault here, I'm afraid. The system is not spawning threads for async requests, it's just creating Tasks which cost very little since they aren't physical threads. Think of goroutines, not threads. All the things you mention seem to be due to lack of experience. I am not sure how using another platform would have yield better results, skill sets remaining the same. It's not a problem if you try a new language, platform or framework as long as you don't just churn out code assembling method calls based on name (. NET makes this very easy because of intellisense) and examples found on Stack Overflow. As long as you understand what the code does, you are fine.
- javcasas 5y ago
  So the problem is that I'm using Microsoft's recommendations (Sql Server, repository pattern, LINQ) for a Microsoft product, and the difference between an inexperienced team and an experienced team is that the experienced team knows what list of recommendations coming from Microsoft are actually the ones you should follow in order to bypass most of the footguns. I think I'm going to go with frameworks that have the actual best reccomendations baked in, instead of the frameworks that recommend footguns.
  - DeathArrow 5y ago
    >I think I'm going to go with frameworks that have the actual best reccomendations baked in, instead of the frameworks that recommend footguns. For every good framework, regardless of platform and language, there is going to be a learning curve. That's because flexibility and extensibility add complexity. Only a trivial framework with limited use cases would not allow you to make mistakes. I take it you never used C or C+=, because there's a footgun waiting for you at every keystroke. :)
moomin 5y ago
It’s interesting to look at this from the perspective of an experienced .NET developer, because you’re not wrong: * SQL Server can be incredibly fast with good access patterns, but the rest suggests you didn’t have those. * Yeah, the repository pattern is garbage. I prefer just define the operations you need and use Dapper. * I know some on here are wild about Entity Framework, but personally I’d rather write the SQL myself. You need a good mental model of what’s going on under the hood or what you’ve described happens. * Honestly never used the default test runner because it seems pretty bad. NCrunch, however, is incredible and doesn’t suffer from this. * I don’t think I’ve ever worked anywhere that used the default logging implementation and yes, that format is weird. (You can, however, step into the logging code if you want.) * Depends on what you’re doing, but yeah 10,000,000 Task.Runs is not going to perform well. But I’m not sure the equivalent code works well anywhere except possibly BEAM. So whilst none of these things cause me problems on a day-to-day basis, I can 100% see how inexperienced devs could drive straight into that tar pit.
- hackerfromthefu 5y ago
  Like you, I also wish HN would preserve newlines or keep asterisks as bullet points on newlines! :D
  - moomin 5y ago
    FML...
divega 5y ago
> LINQ supports interleaving DB operations with not-db operations, quickly becoming a O(n^m) footguns. This is no longer the case since EF Core 3.0. Only top level projection can contain expressions that aren’t translatable to SQL. > You better not do some type of async stuff in LINQ, because the system is going to spawn a process for each entry… I wonder exactly what you ran into. This is not an accurate description of how EF Core’s async support works. Not even if you replace the word “process” with the word “thread”.

hallway_monitor 5y ago

Based on this anecdote, it sounds like the team had little or no .net experience and your seniors probably don't deserve the title.

javcasas 5y ago
Curiously, I haven't had these problems in Python/cherrypy/django/flask, Haskell/snap/yesod, NodeJS/express/nextjs, so, maybe it's something beyond lack of .net experience.
- tomrod 5y ago
  Your experience mirrors mine, both early (circa 2005) and more recently. Footguns are common. I am not sure where the .NET community coalesces (python and similar tend to hit up IRC).
  - topkai22 5y ago
    The ability to find and get answers on stack overflow is pretty impressive, although that might not be the “community” your looking for. I’ve had decent direct engagements with them framework dev tends on GitHub before as well.
    - tomrod 5y ago
      To add on to community -- it's more than that as well, it's the zeitgeist. In the scientific computing world, if something has hit NumFocus' portfolio then it's active, maintained, accessible, and (IMO) can consider adding into a tech stack. Similar for Apache for general FLOSS projects. I don't understand the dotnet's extension ecosystem -- how to find quality sourcing, etc. Going from an established project into dotnet SDK makes sense. But searching GitHub for useful dotnet frameworks -- stars are probably misleading here?
- koide 5y ago
  There's been lots of bad decisions along the way for .net, but In its current state (.net Core 6) I can't really see any of your problems happening. Although what strikes me most is your first point. Running SQL server on 1gb and complaining that it didn't work well isn't really wise. Maybe this happened on the early years of .net Core or even on FF?

achandlerwhite 5y ago

For the benefit of others who might get the wrong idea, async/await definitely doesn’t spawn processes and in almost all cases not even extra threads.

moomin 5y ago
Pretty sure he was employing “Task.Run” which is… nearly always the wrong thing but does appear in a lot of the early examples.

Hawxy 5y ago

Was this the first time using .NET for most of your devs? > LINQ supports interleaving DB operations with not-db operations This hasn't been the case since EF Core 3.0, it will throw an exception if something can't be translated to prevent this problem. Prior to this (EF Core 2.2), a warning would be spat out if an in-memory operation occurs. Due to this, I assume this project was a number of years ago. > The logger would log the message and the affected class in different lines. The default logging format is unfortunate, although it can be made a single line in .NET 5 onwards. This issue would've been easily solved by using Serilog, swapping out the built-in logging with a few lines of code. I don't know the last time I encountered a .NET dev that wasn't at least aware of Serilog as an alternative.

fomine3 5y ago
Good to know for me who used EF5.

smaudet 5y ago

Keep in mind I've never done ASP.NET (only desktop development): The old C# stuff is indeed crusty, intermixed with hot nonsense from the Balmer days. Latest Framework version is I'd say OK, nothing to write home about, newer stuff is much better (.net 5, 6). I do think MS SQL Server is a bit, well, bad. But don't worry, irrelevant to the C# stuff - I used to use java servlets to query mssql, while it was better performing than mysql, I wouldn't pick it again. The anecdote about the React frontend bit is interesting, but ultimately depends on what the UI was doing. JS is certainly good to bang out some quick UI, but if it was shell for the backend, I'd say your app was either mis-designed or just naturally backend heavy. Again, haven't used (not in web dev right now), but Blazor exists for doing C# web UI, instead of complaining that the backend was bad/hard a better comparison would be between Blazor and React. Or you could switch architectures and do a non-component-based web UI - faster to put together a mashup of JS junk but it doesn't necessarily perform as well as doing a web client old-style - simple request response works for a lot of use-cases, its not as flashy, but it is fast.

Foobar8568 5y ago

The (Generic) Repository Patterns are some of the most anti-patterns created and goes fully agains't DRY and SOLID, it's like whole generations of developers are just on the same cargo and have never worked with a database( or datasource) having more than 1 table. So it looks great for a blog post, for a linkedin thought leader article, but when actually used, you end up duplicating the same shit over and over and over unless you use extension methods. But then why having the damn Repo patterns with the basic methods in the first place? These patterns are just awful and don't solve anything, abstraction of the DB? You don't need that pattern for that.

manigandham 5y ago
Agreed. Fortunately the main documentation now shows that this is an anti pattern since EF dbcontext is already a repository pattern and implements unit-of-work semantics.
- NicoJuicy 5y ago
  I wanted to say exactly this.
- DeathArrow 5y ago
  Well, please explain to me how can I switch the app from using Postgres to Redis or Cassandra with ease if I am not using something like a repository pattern? It's true, EF abstracts the database away by not requiring you to write SQL but EF is only usable with some SQL databases. So, having another abstraction layer might not be a bad idea.
  - javcasas 5y ago
    Yup, because we devs switch databases every week. That's the real reason why all projects are late: we are so busy switching databases all the time we don't have time for writing code!
  - kingmotley 5y ago
    dotnet add package EfCore.Cassandra. Then in startup.cs or program.cs, add this to your services: optionsBuilder.UseCassandra("Contact Points=127.0.0.1;", opt => { opt.MigrationsHistoryTable(HistoryRepository.DefaultTableName, "<schema name>"); }); See more here: https://efcorecassandra.readthedocs.io/en/latest/intro/getting-started-with-efcorecassandra.html https://efcorecassandra.readthedocs.io/en/latest/intro/getti...

manigandham 5y ago

Your last point is not how threading or async works in .NET. It’s a single process that has a thread pool that runs async Tasks. The rest of the comment indicates that the team didn’t have much or any experience with .NET. I’m sure there are issues with learning curve but the ecosystem is very capable (the corollary to being very productive) but you need to know how to properly use it.

gregoryl 5y ago
It also indicates the ego of the "best" senior dev who leaps in to save the day is quite large, and not very well correlated with ability and experience with the stack.
- javcasas 5y ago
  We also tried hiring. After checking 100+ resumes, and interviewing the best 10 candidates, we hired the best we could find, who turned out to be average. My ego is not unfounded, I have had to do this successfully in other projects, but I can only do it if both the tools, the codebase and the framework allow me to do it
  - kingmotley 5y ago
    I would say it sounds unfounded to me. You "jumped" in to a different language, different database, different framework, and you landed up making junior level mistakes. Your explanations of what went wrong show clearly that you still don't have a grasp of how async/await works. So you don't understand the concept, which is now common across a number of languages including javascript (which you claim to know) and python (which you claim to know), dart, and scala. Based on your technical knowledge of languages you claim to know, I would say you are closer to a junior developer.
    - javcasas 5y ago
      You seriously want me to tell you how async, promises, callbacks, coroutines, closures+continuations, call/cc, and copies of the stack are actually points of view in the same concept? If I'm a junior, 99% of the professional programmers out there are simply not programmers at all.

DeathArrow 5y ago

All that sounds like poor app design. Microsoft doesn't force you to use SQL Server. I used Postgres and Redis for a quite big microservice based app. Repository isn't mandatory, however is nice to separate the business logic from the details of the data store or database. Using a repository shouldn't be a a problem unless you are doing it wrong. I've seen enough of poor repository pattern implementation. I would question the skills of the developers here. LINQ was not used properly if you had mixed IEnumerable and IQueryable based queries. It's not frameworks fault here, I'm afraid. The system is not spawning threads for async requests, it's just creating Tasks which cost very little since they aren't physical threads. Think of goroutines, not threads. All the things you mention seem to be due to lack of experience. I am not sure how using another platform would have yield better results, skill sets remaining the same. It's not a problem if you try a new language, platform or framework as long as you don't just churn out code assembling method calls based on name (. NET makes this very easy because of intellisense) and examples found on Stack Overflow. As long as you understand what the code does, you are fine.

javcasas 5y ago
So the problem is that I'm using Microsoft's recommendations (Sql Server, repository pattern, LINQ) for a Microsoft product, and the difference between an inexperienced team and an experienced team is that the experienced team knows what list of recommendations coming from Microsoft are actually the ones you should follow in order to bypass most of the footguns. I think I'm going to go with frameworks that have the actual best reccomendations baked in, instead of the frameworks that recommend footguns.
- DeathArrow 5y ago
  >I think I'm going to go with frameworks that have the actual best reccomendations baked in, instead of the frameworks that recommend footguns. For every good framework, regardless of platform and language, there is going to be a learning curve. That's because flexibility and extensibility add complexity. Only a trivial framework with limited use cases would not allow you to make mistakes. I take it you never used C or C+=, because there's a footgun waiting for you at every keystroke. :)

moomin 5y ago

It’s interesting to look at this from the perspective of an experienced .NET developer, because you’re not wrong: * SQL Server can be incredibly fast with good access patterns, but the rest suggests you didn’t have those. * Yeah, the repository pattern is garbage. I prefer just define the operations you need and use Dapper. * I know some on here are wild about Entity Framework, but personally I’d rather write the SQL myself. You need a good mental model of what’s going on under the hood or what you’ve described happens. * Honestly never used the default test runner because it seems pretty bad. NCrunch, however, is incredible and doesn’t suffer from this. * I don’t think I’ve ever worked anywhere that used the default logging implementation and yes, that format is weird. (You can, however, step into the logging code if you want.) * Depends on what you’re doing, but yeah 10,000,000 Task.Runs is not going to perform well. But I’m not sure the equivalent code works well anywhere except possibly BEAM. So whilst none of these things cause me problems on a day-to-day basis, I can 100% see how inexperienced devs could drive straight into that tar pit.

hackerfromthefu 5y ago
Like you, I also wish HN would preserve newlines or keep asterisks as bullet points on newlines! :D
- moomin 5y ago
  FML...

divega 5y ago

> LINQ supports interleaving DB operations with not-db operations, quickly becoming a O(n^m) footguns. This is no longer the case since EF Core 3.0. Only top level projection can contain expressions that aren’t translatable to SQL. > You better not do some type of async stuff in LINQ, because the system is going to spawn a process for each entry… I wonder exactly what you ran into. This is not an accurate description of how EF Core’s async support works. Not even if you replace the word “process” with the word “thread”.