3 ms·

I expect that what they've pointed to as the cause is only part of the problem. We'll never know the full picture unless they share it. Being upfront, my exper

by than3 5y ago

I expect that what they've pointed to as the cause is only part of the problem. We'll never know the full picture unless they share it. Being upfront, my experience of the people in charge of the organization there doesn't have much goodwill left. Nothing against Thibault personally, I think he's done some great things but seems to be busy with whatever he's interested in and management isn't it, and he has some unprofessional people with access that work for the project/charity. I had volunteered my services years ago as a System Administrator (no charge) with suggestions, but with limited modes of communication, multiple issues going stale, no response with an auto issue closed. They have issues that don't get addressed, and their process doesn't appare to be aimed to cultivate qualified volunteers or improve the bus factor of the project. To make things worse, when I expressed disagreement with constructive feedback regarding one of their process decisions, one of the other dev's with access apparently took offense and the next day I found my lichess account had been edited by an admin without notice or notification. I could not log in (wrong password), the password and email for password recovery were changed, and trying to access the profile URI directly showed the account as banned. It seemed this was done out of spite, and definitely without any kind of due process. Appeals by email went unanswered within the 90 day cutoff I gave them. As a result I submitted a complaint to the french charities regulatory body and moved on since the group wasn't worth wasting any more of my time. I haven't heard back so who knows if anything came of what I reported. In my opinion, they've got more internal problems than they let on, and to me this is just spillover. Its unfortunate because any failure like this impacts so many people, but I don't find it surprising given my limited experience of the people there.

Shadonototra 5y ago
- than3 5y ago
  Yes, that is the only place you can communicate with them. If they haven't tampered with the history it should still be there for anyone that wants to dig. I was simply describing my experience and yet someone always tries to minimize or discredit assuming some other motive; whatever, the world isn't all rainbows and sunshine. You seem to have missed the point, didn't read, both, or have some other agenda. To turn it around, what's your motive for being 'unusually' defensive? You know there are three lichess right, the organization, the service, and the OSS code. What are we talking about here? The former, not the latter two. At this point I could care less how they run their ship. I've learned that what they sell to the public, and the process by which they operate are different things, and through their actions that they don't value processes that are generally accepted and work. As a result, I won't have anything to do with them, but its annoying how many people blindly have to assert the assumption that Lichess (the organization) can do no wrong, and the marketed narrative is the whole picture, and implying that anyone saying differently is attacking or has some subversive plan to hurt the community. Honestly, that kind of propaganda strategy sounds just like something one might hear in the Nuremberg trials (Henry Goering interview with Goebbels). To be clear, I only posted today because this spoiled my newsfeed. If they want to be transparent they should publish a professional post-mortem.
- IggleSniggle 5y ago
  …wow. Did you even read what they wrote?
  - than3 5y ago
    Yes, I read what was published by Lichess and the other person. If by 'they' you are referring to Lichess (I'm assuming you were since my post speaks for itself), the published report has holes and doesn't account for the reported site-wide service interruption that occurred some time earlier before they were forced to suspend the tournament. Specific holes include, there isn't a timeline with any kind of useful supporting documentation or detail (metrics, logs between services) etc. The graph that was provided is worthless beyond showing a correlative spike in traffic when the issue became noticeable. It does not inform the cause in any practical way other than to say there was an issue at some point before the spike which culminated in an outage. They carefully don't mention any useful, supporting, or practical detail about the architecture, logging, load shedding, etc, that would support that what they say is what happened. They say it wasn't overloaded, when it was clearly resource constrained. In my opinion the communication is an assurance article, a table with no legs. We can only guess at what happened, unless they share more specific details. Also, annoyingly, they say load testing and simulation can't be done at that scale. It can, but to get usable results that accurately model and inform, requires competent engineering and resources. It is not simple, but neither is it beyond the pale either.
    - evanrelf 5y ago
      Hey than3, I think IggleSniggle's comment[1] about "not reading what they wrote" was replying to Shadonototra's comment[2], not yours. So "they" is you; IggleSniggle is questioning whether Shadonototra read your comment. [1]: https://news.ycombinator.com/item?id=29656928 https://news.ycombinator.com/item?id=29656928 [2]: https://news.ycombinator.com/item?id=29656251 https://news.ycombinator.com/item?id=29656251
      - than3 5y ago
        Ahh thanks, it had looked like a reply to me and not the other guy. Indirect pronouns can be troublesome. Dangerous at both ends and crafty in the middle.
schaefer 5y ago
Would you consider reading the excellent book "Working in Public: The Making and Maintenance of Open Source Software" by Nadia Eghbal? The data behind what "Open Source" projects look like differs from the popular culture narratives and assumptions about what they should look. I doubt there was justification for disabling your player account. and I'm sorry that happened to you. But it sounds like your expectations about code contributions, and onboarding new volunteers may have been far from that project's reality.
- than3 5y ago
  Thank you for the suggestion. I had not heard about this book and based on the amazon description this looks interesting. I've worked with a few OSS projects in the past that were much more open and transparent. My expectations were framed by the contribution documentation Lichess provided on their site and it did differ significantly from reality. I'll definitely be picking a copy up in the near future for reading, currently I have to first finish up with the holiday gift exchange books from colleagues. I'm not a management person but I'm currently making my way through HBR 10 Essential reads (book) which was given; I'd never have bought it myself but I've found some useful insight reading through it (unexpected).
- qchris 5y ago
  Just would like add a +1 for Working in Public. It's a really excellent look into a number of different community dynamics, including but frankly not limited to open source projects exclusively.
ninth_ant 5y ago
Most anyone who has built a high-demand service has encountered issues exactly like this. Unusual activity places stress on a small piece of poorly optimized code that was never tested for it, and never considered because it’s never been an issue before. A fleet of hardware goes idle while the single lock spins. Whatever personal issues you may have with the admins, it's extremely plausible that this issue was sufficient to cause an outage. And yes these things happen at big tech companies and mature software products even when they have teams and testing frameworks specifically looking for them. A smaller project like this, I cut them a break.
austhrow743 5y ago
What did you report them for?
- than3 5y ago
  At the time, I was looking for more detailed information on the charity in the hopes of finding a more direct contact than a non-responsive email box. In most countries, operating as a charity has certain annual reporting requirements. These reports often contain a direct contact (i.e. Agent of Service type contacts). I was able to find the charity registration info from sirene.fr by their address (there were 14 listings out of that address when I just rechecked). Siret# 83037871700029, Siren# is the first 9 digits. There was no phone number listed so I moved onto JOAFE and found that there were no annual reports attached to that Siren #. Backing up I went to RCS, the company & trade register which is required to obtain a kbis-extract (needed to open a bank account as a company in france) and there were no listings on the Lichess or Lichess.org trade names. RCS doesn't allow a search by Siren # and that's where my search died. The complaint was that they were operating, and had not filed the required reports. I don't speak french, so the complaint was in English via a webform and since it was difficult to pursue the matter further at the time, I chose to move on to better things. Edit: I've confirmed that the annual reporting is still a problem on JOAFE. Also, back when this originally happened I had noticed the budget they linked to on googledocs for what they spend on infrastructure was dated, not sure if that is still the case. If anyone with a greater knowledge of OSINT or a command of the french language wants to look into this further there are a number of administrative inconsistencies that might be worth following up on. Personally, I don't see how you can be registered as a french charity and accept donations/other assistance without the appropriate bank account or filing annual reports. Though I admit, I don't have much first-hand knowledge of doing business in France. Either way, its not worth my time taking this any further.

by than3 5y ago

Shadonototra 5y ago

than3 5y ago
Yes, that is the only place you can communicate with them. If they haven't tampered with the history it should still be there for anyone that wants to dig. I was simply describing my experience and yet someone always tries to minimize or discredit assuming some other motive; whatever, the world isn't all rainbows and sunshine. You seem to have missed the point, didn't read, both, or have some other agenda. To turn it around, what's your motive for being 'unusually' defensive? You know there are three lichess right, the organization, the service, and the OSS code. What are we talking about here? The former, not the latter two. At this point I could care less how they run their ship. I've learned that what they sell to the public, and the process by which they operate are different things, and through their actions that they don't value processes that are generally accepted and work. As a result, I won't have anything to do with them, but its annoying how many people blindly have to assert the assumption that Lichess (the organization) can do no wrong, and the marketed narrative is the whole picture, and implying that anyone saying differently is attacking or has some subversive plan to hurt the community. Honestly, that kind of propaganda strategy sounds just like something one might hear in the Nuremberg trials (Henry Goering interview with Goebbels). To be clear, I only posted today because this spoiled my newsfeed. If they want to be transparent they should publish a professional post-mortem.
IggleSniggle 5y ago
…wow. Did you even read what they wrote?
- than3 5y ago
  Yes, I read what was published by Lichess and the other person. If by 'they' you are referring to Lichess (I'm assuming you were since my post speaks for itself), the published report has holes and doesn't account for the reported site-wide service interruption that occurred some time earlier before they were forced to suspend the tournament. Specific holes include, there isn't a timeline with any kind of useful supporting documentation or detail (metrics, logs between services) etc. The graph that was provided is worthless beyond showing a correlative spike in traffic when the issue became noticeable. It does not inform the cause in any practical way other than to say there was an issue at some point before the spike which culminated in an outage. They carefully don't mention any useful, supporting, or practical detail about the architecture, logging, load shedding, etc, that would support that what they say is what happened. They say it wasn't overloaded, when it was clearly resource constrained. In my opinion the communication is an assurance article, a table with no legs. We can only guess at what happened, unless they share more specific details. Also, annoyingly, they say load testing and simulation can't be done at that scale. It can, but to get usable results that accurately model and inform, requires competent engineering and resources. It is not simple, but neither is it beyond the pale either.
  - evanrelf 5y ago
    Hey than3, I think IggleSniggle's comment[1] about "not reading what they wrote" was replying to Shadonototra's comment[2], not yours. So "they" is you; IggleSniggle is questioning whether Shadonototra read your comment. [1]: https://news.ycombinator.com/item?id=29656928 https://news.ycombinator.com/item?id=29656928 [2]: https://news.ycombinator.com/item?id=29656251 https://news.ycombinator.com/item?id=29656251
    - than3 5y ago
      Ahh thanks, it had looked like a reply to me and not the other guy. Indirect pronouns can be troublesome. Dangerous at both ends and crafty in the middle.

schaefer 5y ago

Would you consider reading the excellent book "Working in Public: The Making and Maintenance of Open Source Software" by Nadia Eghbal? The data behind what "Open Source" projects look like differs from the popular culture narratives and assumptions about what they should look. I doubt there was justification for disabling your player account. and I'm sorry that happened to you. But it sounds like your expectations about code contributions, and onboarding new volunteers may have been far from that project's reality.

than3 5y ago
Thank you for the suggestion. I had not heard about this book and based on the amazon description this looks interesting. I've worked with a few OSS projects in the past that were much more open and transparent. My expectations were framed by the contribution documentation Lichess provided on their site and it did differ significantly from reality. I'll definitely be picking a copy up in the near future for reading, currently I have to first finish up with the holiday gift exchange books from colleagues. I'm not a management person but I'm currently making my way through HBR 10 Essential reads (book) which was given; I'd never have bought it myself but I've found some useful insight reading through it (unexpected).
qchris 5y ago
Just would like add a +1 for Working in Public. It's a really excellent look into a number of different community dynamics, including but frankly not limited to open source projects exclusively.

ninth_ant 5y ago

Most anyone who has built a high-demand service has encountered issues exactly like this. Unusual activity places stress on a small piece of poorly optimized code that was never tested for it, and never considered because it’s never been an issue before. A fleet of hardware goes idle while the single lock spins. Whatever personal issues you may have with the admins, it's extremely plausible that this issue was sufficient to cause an outage. And yes these things happen at big tech companies and mature software products even when they have teams and testing frameworks specifically looking for them. A smaller project like this, I cut them a break.

austhrow743 5y ago

What did you report them for?

than3 5y ago
At the time, I was looking for more detailed information on the charity in the hopes of finding a more direct contact than a non-responsive email box. In most countries, operating as a charity has certain annual reporting requirements. These reports often contain a direct contact (i.e. Agent of Service type contacts). I was able to find the charity registration info from sirene.fr by their address (there were 14 listings out of that address when I just rechecked). Siret# 83037871700029, Siren# is the first 9 digits. There was no phone number listed so I moved onto JOAFE and found that there were no annual reports attached to that Siren #. Backing up I went to RCS, the company & trade register which is required to obtain a kbis-extract (needed to open a bank account as a company in france) and there were no listings on the Lichess or Lichess.org trade names. RCS doesn't allow a search by Siren # and that's where my search died. The complaint was that they were operating, and had not filed the required reports. I don't speak french, so the complaint was in English via a webform and since it was difficult to pursue the matter further at the time, I chose to move on to better things. Edit: I've confirmed that the annual reporting is still a problem on JOAFE. Also, back when this originally happened I had noticed the budget they linked to on googledocs for what they spend on infrastructure was dated, not sure if that is still the case. If anyone with a greater knowledge of OSINT or a command of the french language wants to look into this further there are a number of administrative inconsistencies that might be worth following up on. Personally, I don't see how you can be registered as a french charity and accept donations/other assistance without the appropriate bank account or filing annual reports. Though I admit, I don't have much first-hand knowledge of doing business in France. Either way, its not worth my time taking this any further.