4 ms·

RudyMQ – A message queue in .NET

29 points by aashishkoirala 13y ago

csharpdude 13y ago
Interesting. I have to be honest I did not know about the whole reader/writer lock support. I was accustomed to just using the "lock" keyword.
- bananas 13y ago
  Watch out for ReaderWriterLockSlim. It's dangerous under certain circumstances particularly if: 1. You have one core or two cores. 2. Your CPU load is high. 3. Your thread pool is pretty heavily utilised so there is a queue building up (think IIS). It relies on spinlock so it enters a tight loop for a number of cycles to avoid lock contention. At this point it increases the CPU load so every thread ends up getting blocked on spinlocks. This means you end up with N runnable threads (one per core) which are utilising 100% CPU in a tight loop and your site goes down. I wouldn't rely on it for a concurrency mechanism in production! We've had this in production and it took us out for a few hours. Note: the root cause was a for loop that never terminated but all the other threads went batshit at 100% inside ReaderWriterLockSlim so it's impossible to identify what is going on. You can't even log into a box under that load so it's a case of waiting for it to blow and dropping a minidump out with process explorer and getting that off-site and firing it up in VS. Total nightmare.
  - josteink 13y ago
    > Watch out for ReaderWriterLockSlim. Yes. And if you register this in a using-statement thinking it will be disposed, you may encounter a edge case where the lock has been created, but not assigned and thus wont get disposed and you will have orphaned locks jamming up your system. Where I worked, we had complete system crashes which could be traced back to "incorrect" usage of this very class[1]. It only happened in production, it was hell to debug, and we had to use more low-level tools than I ever thought should be needed for .NET code. We replaced it with a traditional lock {} and never looked back. [1] "Incorrect" as in not having around 50 lines of boilerplate which has to be tediously and 100% correctly done, on a per lock-invocation basis, since generalizing the implementation will ruin its correctness.
    - Pxtl 13y ago
      > Where I worked, we had complete system crashes which could be traced back "incorrect" usage of this very class Ah, yes. The ultimate abstraction failure - a bad library is one that requires you not only understand its abstraction, but you understand its implementation and the underlying layer in such meticulous detail that you could re-write the entire thing from scratch. See also webforms. Worse than useless.
  - danesparza 13y ago
    Here is a gem on SO about ReaderWriterLockSlim: http://stackoverflow.com/a/17296055/19020 http://stackoverflow.com/a/17296055/19020 HC SVNT DRACONES
    - aashishkoirala 13y ago
      Hey, thanks for that. That is enlightening.
    - bananas 13y ago
      That's marvellous - thanks for posting :)
    - Pxtl 13y ago
      "I have to admit it has way too many bells and whistles" That just described half the objects in the .NET framework for me.
lafar6502 13y ago
nice, but this line var receiveOperation = queue.StartReceiving<MyMessage>(100, /* poll every 100 ms */ shows that you are polling the queue for new messages. Why such inefficient technique? And BTW it looks like there's no support for transactional operations. This is indeed a rudimentary queue ;)
- aashishkoirala 13y ago
  What it does there is call the WCF service to see if there is a new message. If there is, it will consume it and call again. The wait only comes into play if there are no messages. In terms of why I am polling, without going into duplex communication, I could not find a purely event-driven/subscription-based way to do this. I mean, I could use a queue- but then a queue that uses another queue to do its thing would be somewhat absurd.
- aashishkoirala 13y ago
  Also, there is somewhat of a support for transactional operations - you can declare a queue as transactional - which will return the message to the queue if the client fails. However, there is no concept of poison queue - or integration with TransactionScope.
  - Pxtl 13y ago
    I'm curious, have you ever got distributed transactions - particularly ones over WCF and the like - working? I've found that wile TransactionScope works like a dream in the trivial case, it becomes disproportionately hairy and frustrating as you start trying to use such elaborate features.
    - aashishkoirala 13y ago
      It generally works with anything that supports MSDTC; multiple SQL Server nodes being the most usual scenario. The one case where it starts to get hairy is when you bring MSMQ into the mix, not surprisingly. That has been my experience, anyway.
netriani 13y ago
What would be the disadvantage of using duplex WCF channel instead of polling?

Interesting. I have to be honest I did not know about the whole reader/writer lock support. I was accustomed to just using the "lock" keyword.

bananas 13y ago
Watch out for ReaderWriterLockSlim. It's dangerous under certain circumstances particularly if: 1. You have one core or two cores. 2. Your CPU load is high. 3. Your thread pool is pretty heavily utilised so there is a queue building up (think IIS). It relies on spinlock so it enters a tight loop for a number of cycles to avoid lock contention. At this point it increases the CPU load so every thread ends up getting blocked on spinlocks. This means you end up with N runnable threads (one per core) which are utilising 100% CPU in a tight loop and your site goes down. I wouldn't rely on it for a concurrency mechanism in production! We've had this in production and it took us out for a few hours. Note: the root cause was a for loop that never terminated but all the other threads went batshit at 100% inside ReaderWriterLockSlim so it's impossible to identify what is going on. You can't even log into a box under that load so it's a case of waiting for it to blow and dropping a minidump out with process explorer and getting that off-site and firing it up in VS. Total nightmare.
- josteink 13y ago
  > Watch out for ReaderWriterLockSlim. Yes. And if you register this in a using-statement thinking it will be disposed, you may encounter a edge case where the lock has been created, but not assigned and thus wont get disposed and you will have orphaned locks jamming up your system. Where I worked, we had complete system crashes which could be traced back to "incorrect" usage of this very class[1]. It only happened in production, it was hell to debug, and we had to use more low-level tools than I ever thought should be needed for .NET code. We replaced it with a traditional lock {} and never looked back. [1] "Incorrect" as in not having around 50 lines of boilerplate which has to be tediously and 100% correctly done, on a per lock-invocation basis, since generalizing the implementation will ruin its correctness.
  - Pxtl 13y ago
    > Where I worked, we had complete system crashes which could be traced back "incorrect" usage of this very class Ah, yes. The ultimate abstraction failure - a bad library is one that requires you not only understand its abstraction, but you understand its implementation and the underlying layer in such meticulous detail that you could re-write the entire thing from scratch. See also webforms. Worse than useless.
- danesparza 13y ago
  Here is a gem on SO about ReaderWriterLockSlim: http://stackoverflow.com/a/17296055/19020 http://stackoverflow.com/a/17296055/19020 HC SVNT DRACONES
  - aashishkoirala 13y ago
    Hey, thanks for that. That is enlightening.
  - bananas 13y ago
    That's marvellous - thanks for posting :)
  - Pxtl 13y ago
    "I have to admit it has way too many bells and whistles" That just described half the objects in the .NET framework for me.

lafar6502 13y ago

nice, but this line var receiveOperation = queue.StartReceiving<MyMessage>(100, /* poll every 100 ms */ shows that you are polling the queue for new messages. Why such inefficient technique? And BTW it looks like there's no support for transactional operations. This is indeed a rudimentary queue ;)

aashishkoirala 13y ago
What it does there is call the WCF service to see if there is a new message. If there is, it will consume it and call again. The wait only comes into play if there are no messages. In terms of why I am polling, without going into duplex communication, I could not find a purely event-driven/subscription-based way to do this. I mean, I could use a queue- but then a queue that uses another queue to do its thing would be somewhat absurd.
aashishkoirala 13y ago
Also, there is somewhat of a support for transactional operations - you can declare a queue as transactional - which will return the message to the queue if the client fails. However, there is no concept of poison queue - or integration with TransactionScope.
- Pxtl 13y ago
  I'm curious, have you ever got distributed transactions - particularly ones over WCF and the like - working? I've found that wile TransactionScope works like a dream in the trivial case, it becomes disproportionately hairy and frustrating as you start trying to use such elaborate features.
  - aashishkoirala 13y ago
    It generally works with anything that supports MSDTC; multiple SQL Server nodes being the most usual scenario. The one case where it starts to get hairy is when you bring MSMQ into the mix, not surprisingly. That has been my experience, anyway.

netriani 13y ago

What would be the disadvantage of using duplex WCF channel instead of polling?