9 ms·

Coding Cloud Heroku May Face Lawsuit Over Bait-and-Switch Claims

178 points by wwarnerandrew 14y ago

redthrowaway 14y ago
>Heroku declined our request for further comment. “Unfortunately, we can’t publicly comment on legal matters,” said spokeswoman Dana Oshiro in an email. “Heroku is committed to our customers’ success and focused on delivering the best possible product and experience. We’ll continue to update our customers via our blog.” I don't run a company, and I'm not responsible for maximizing return on my shareholders' investment. So maybe I lack the mental schemas that make these sorts of statements seem reasonable, but they always strike me as short-sighted. Yes, admitting wrongdoing may open you up to legal liability. But it seems to me that, in the long run, it's far better to take your licks and own up to your mistakes than to hide behind the shield of pending litigation. My gut instinct is to offer a sincere mea culpa and provide a transparent roadmap to improvement, and if you lose a court case then so be it. I'd like to think that the good will and respect you build from accountability and honesty will ultimately outweigh any short-term losses from court settlements or judgements. As I said, I don't live in that world. I would be saddened, though not necessarily surprised, to learn that I was wrong.
- biot 14y ago
  What makes this proposed class action suit possible is that Heroku did admit[0] that their service didn't match up with what their documentation and website claimed, that they messed up, and would be fixing it. And it wasn't something like "our documentation and site could have been misinterpreted", they actually admitted the website and documentation were inaccurate with metrics which were incorrect. It's like a class action lawyer's wet dream of culpability on the part of the defendant. [0] https://blog.heroku.com/archives/2013/2/16/routing_performance_update https://blog.heroku.com/archives/2013/2/16/routing_performan...
- kyrra 14y ago
  Seeing that there is a website/lawfirm[1] threatening legal action, I can bet Heroku/Salesforce lawyers jumped on that ASAP and let all their employees that are allowed to talk publicly not to comment on the issue anymore. The US legal system is setup that if a company admits fault in something like this, it makes defending yourself much more difficult. Meaning if a lawsuit is brought about, it's not about proving innocence anymore, it's about mitigating the damages you will have to pay. Your lawyer is going to want to have as many options available to him/her to deal with your case during it's lifespan, and admitting fault before the lawsuit even begins will limit the lawyers options when it comes to negotiations and what tactics they can use in the courtroom. [1] http://herokuclassaction.com/ http://herokuclassaction.com/
  - incongruity 14y ago
    In the grand scheme of things, this is why it's always better to defect in the simplified idea game of the prisoner's dilemma. Collaboration is rarely rewarded once lawyers get anywhere near an issue and opting to collaborate (i.e.: admit fault and try to fix it) more or less wins you no points in court, in the U.S. Given that each possible liability is its own case and stands on its own merits, it pretty much matches the simplest case of the prisoner's dilemma perfectly, IMHO... and so here we are. Sadly.
    - OGinparadise 14y ago
      Collaboration is rarely rewarded once lawyers get anywhere near an issue and opting to collaborate (i.e.: admit fault and try to fix it) more or less wins you no points in court, in the U.S. Might, in part, have to do with how lawyers get compensated. A short case or a quick settlement will result in very little compensation for them. Over the course of years and decades that matters so we see everything analyzed to death. At $500+ an hour. Per lawyer.
      - jacques_chester 14y ago
        The careful decomposition of problem domain into solution domain followed by recursive search of the solution domain is the lawyer's job. If they don't analyse to death, they are breaching their fiduciary duty. Don't like the legal method? Don't hire a lawyer. Try doing it yourself. See how far you get when the other guy's lawyers have done the exhaustive analysis and found the deciding element buried in a logical node on a distant subgraph you didn't even know was there to be explored.
        OGinparadise 14y ago
        Yeah, other developed countries are all going downhill for not being overlawyered up like in US. It's a system set by lawyers for lawyers so of course you need a lawyer more than in many other countries. "Everyone" sues over everything
- resu_nimda 14y ago
  They have made public statements on their blog, and hopefully that will continue. In this situation, I think it is the right move to avoid getting involved with this shameless law firm trying to incite drama for their own gain.
  - redthrowaway 14y ago
    This seems like precisely the type of situation class action lawsuits are intended to deal with: a company defrauded a lot of its customers in the same way. It makes more sense for all parties that the case be tried as one than to hear each claimant individually.
- Adaptive 14y ago
  Sadly, the US legal system does not reward heartfelt, honest statements, particularly in these situations. Which is why fine print almost always makes even the nicest company sound like a jerk.
- anigbrowl 14y ago
  There's a middle ground; it is better to fess up and try to square things with your customers in a situation where you've done wrong, but at the same time you don't want to be exploited by unscrupulous plaintiffs in turn. Most of those customers just want what they paid for or to recover their actual losses (what are called economic damages in law) but a few may see a lawsuit as an opportunity to enrich themselves. The court system is supposed to prevent that, and overall it works moderately well, but once papers are filed it's better for the arguments to take place in court rather than outside.
- danso 14y ago
  > Yes, admitting wrongdoing may open you up to legal liability. But it seems to me that, in the long run, it's far better to take your licks and own up to your mistakes than to hide behind the shield of pending litigation. This is an admirable, well-intentioned perspective, but it is one that won't net you much benefit, not even in the long run. Filing legal papers -- or, in this case, raising the banner for a class-action -- is serious business. By the time it happens, the time for defusing the situation and calling a truce is long past. The only opinion that matters is that of the judge and/or jury. And let's face it, being the loser in court is going to have far more impact on the public's opinion of the defendant than any honest-from-the-heart statement or outreach. So with that said, there is nothing of substance that the defendant can say that will help them in court. The opposing attorney is not going to say, "Well, I was going to grill you, but I've had a change of heart ever since reading that impassioned, honest, salt-of-the-earth statement you released pre-trial". The judge won't care either. And the jurors aren't supposed to even be aware of it. I think a civil trial is not much different than the situation portrayed in the famous "Don't talk to police" video: http://www.youtube.com/watch?v=6wXkI4t7nuc http://www.youtube.com/watch?v=6wXkI4t7nuc The people you're facing in court don't care that you've made an effort, post-lawsuit-filing, to be upfront and friendly. Anything you say will not help you, it can only hurt you. In this case, Heroku's reticence in speaking their mind outside of court is the right one.
- OGinparadise 14y ago
  This is normal, once legal threats are issued "we can't comment on legal issues...but customers are #1...blah blah" is the standard answer. A mea culpa right now might haunt you in the long term.
citricsquid 14y ago
If the decisions made by Heroku to misrepresent the product were made prior to the Salesforce acquisition (December 2010) do the founders of Heroku have any liability? Would this sort of thing be a consideration with an acquisition?
- chc 14y ago
  I don't believe any decision was made to misrepresent the product. The product just evolved and the product documentation didn't. The end result is the same — the product was advertised with inaccurate claims — but this appears to be more a case of neglect than a deliberate misrepresentation.
  - jacques_chester 14y ago
    It's not a criminal matter. Mens rea -- "guilty conscience", the intention to act unlawfully -- is a component of crime. But it's not necessary for lots of civil legal matters. For example, if we both enter into a contract, you can hold me to my end. If I fail to hold up my end without intending to fail, then you can still insist that I make right or seek remedies for my failure. (IANAL, TINLA).
    - chc 14y ago
      That all sounds right to me, but I think you've mistsken my meaning. The point wasn't that they lack mens rea, but that the decision to misrepresent the product can't have been made before the acquisition because that decision was never made at all. It was misrepresented by reality gradually changing out from under initially truthful statements.
      - jacques_chester 14y ago
        You don't have to decide to misrepresent in order to be misrepresentative. If your material is wrong, it's wrong, and depending on the jurisdiction you may be on the hook. Ridiculous? Well consider that it gives a strong incentive to ensure that claims made to customers are somewhat accurate. Reducing the cost of caveat emptor makes society richer by reducing transaction costs and so encouraging trade. Sellers have a strong informational advantage over buyers; in general the cost to them of making a correct statement is much lower than having each customer having to find out the truth independently. Not that this is the purpose of torts law, which evolved mostly to fill glaring, unjust gaps in the remedies of the older forms of common law. But that is the social and economic function that it has come to serve. (IANAL, TINLA).
eli 14y ago
IANAL, but my understanding is that threatening a class action lawsuit is easy, but actually bringing an action to court is extremely difficult. And (at least to me) this seems like an awfully weak case -- I don't see any obvious or intentional fraud. The whole thing seems strange to me. I don't really understand what Rap Genius hopes to gain here. If Heroku wasn't providing enough performance for their money (regardless of technical cause), then why did they stick around so long? Would Rap Genius still have sued if the documentation had been 100% correct and instead the problem was just plain old slow I/O on Heroku's side?
- niggler 14y ago
  Ianal stands for I am not a lawyer, so "a lawyer" in "IANAL a lawyer" is redundant.
  - eli 14y ago
    Hah, fixed. Sometimes my fingers get ahead of my brain when typing quickly.
- tomlemon 14y ago
  > If Heroku wasn't providing enough performance for their money (regardless of technical cause), then why did they stick around so long? We didn't know how bad the performance was because Heroku's tools (logs and New Relic) reported incorrect performance data (i.e., they said requests weren't queuing when they were actually spending a ton of time queuing) EDIT: > Would Rap Genius still have sued if the documentation had been 100% correct and instead the problem was just plain old slow I/O on Heroku's side? To be clear, we have no official association with http://herokuclassaction.com/ http://herokuclassaction.com/ or the lawyer behind it – I actually found out about the site from the article's author when he interviewed me for the story. (But I do think Heroku owes its customers a refund)
  - jaggederest 14y ago
    New Relic isn't Heroku's tool, man. It's like a car dealership selling you a Pioneer stereo.
LAMike 14y ago
On one hand the Rap Genius expose seemed like they got duped and mislead, but on the other hand something about this lawyer mentioned in the article comes off as ambulance-chase-y to me.
- lancefisher 14y ago
  It's not really ambulance chasing if all he's after is a partial refund. From the article: As for Lehman, he wants some of the $20,000 his company pays Heroku each month returned. “We deserve some sort of refund here,” he says. EDIT: I re-read the article, and it's not even clear that Rap Genius is part of this. Looks like a lawyer trying to drum up support.
- rayiner 14y ago
  Class action lawyering usually sound ambulance chasey, because any given person doesn't lose that much in a given instance and so the actual people harmed have little incentive to sue. At the same time, it's an edge case in the economy. If you harm a large number of people a small amount, they won't sue you to recover for their injury.
dubcanada 14y ago
http://www.wired.com/wiredenterprise/2013/03/hieroku/ http://www.wired.com/wiredenterprise/2013/03/hieroku/ HIEROKU!? Ignoring the funny url. I doubt this will go anywhere besides in the domain expiry list.
lancefisher 14y ago
From the article, Heroku says, "The problem affected an older version of Heroku’s service, he said, and those who use a new version of the service would not have these issues." As far as I understand that is not really the case. The newer version is the Cedar stack which supports concurrent apps where Bamboo only supports non-concurrent apps. This would reduce the problem with random routing, but not eliminate it.
- randall 14y ago
  The problem is not that they provide random routing, the problem is they said in their documentation that they provided intelligent routing, a design choice they backed off of when they switched stacks. Customers still have a problem, but Heroku doesn't from a legal / class action-y perspective.
bridgeyman 14y ago
In MIT's Mathematics for Computer Scientists course (6.042), I remember learning from Tom Leighton that in certain cases randomly distributing jobs actually IS the optimal solution to load balancing. This technique was somehow associated with Akamai, Professor Leighton's company. Maybe I am misremembering...
- w-ll 14y ago
  I dont know, but I would assume it has to deal with the fact you can't predict how long a job will take to complete, and a random selection would have more resources-more free-more often, verses FIFO where a group of really large tasks could halt everything.
  - kordless 14y ago
    That's exactly correct. Given jobs cleared out quick, that wouldn't necessarily be the case.
  - manmal 14y ago
    And what if they turned the whole thing into a request queue and workers get the "messages" as they become available? To ensure there are no timeouts, they could define a max time a request may stay in the queue, and then take it out and handle it with emergency workers or kill it?
- ww520 14y ago
  I think the matter is if random distribution is what is documented, no one will raise an eyebrows. I believe the Heroku claimed intelligent routing based on server loads to route traffic to idle servers. That threw people off.
  - jonknee 14y ago
    Not only that, but they charged a premium based on their false claims. I'm surprised it took this long for legal rumblings to come around.
- 14y ago
rdl 14y ago
Heh, nothing will make people go back to taking Heroku's side faster than a class action lawsuit on the other side. This would be an awesome form of submarine marketing if they were actually behind it :)
- chubot 14y ago
  Um, why? I was one of many plaintiffs in a class action lawsuit against EA (which was settled in our favor). That did not help EA's image at all :) It looks pretty bad when your customers complain about problems that are hidden by your own monitoring tools, and then you tell them to buy more crap to fix it! It's not a small amount either -- as mentioned the queuing time was larger than the request time. And on top of it you're paying out the nose for this crappy monitoring tool.
  - rdl 14y ago
    Even people who have been somewhat screwed by this issue with Heroku tend to love Heroku; Salesforce could make this right with generous payouts to affected people and 2-6mo of engineering to offer a better load balancer as a service (Actually, I think I could build this as a third party provider, even) EA is basically a lost cause; their business model and culture is the problem. Having better sacks at Dachau wouldn't have helped the Nazi image, either (to auto-Godwin)
    - chubot 14y ago
      Well that's what they should have done BEFORE the threat of class action lawsuit. It makes them look at a lot worse if it happens after. I'm pretty surprised they didn't offer refunds already. It was pretty clear they were asked for, and with no update by now, it's safe to say they were refused. That seems unreasonable to me. If you were in the position RapGenius, would you honestly not have expected a refund already?
      - rdl 14y ago
        I wonder what happened to Heroku. They used to be awesome. They still have elements of being awesome, but seem either resource constrained or self destructive. I find it hard to believe Salesforce wouldn't give them what they need to do things correctly -- ranging from "go multi-AWS Region or move off AWS" to building a decent load balancing system to handling this fiasco.
scott_meade 14y ago
"[The attorney who set up this site] has spoken with one customer, who has since backed off. “That being said, I am in the process of drafting a complaint, in case their feet warm up,” he told us."
vertis 14y ago
It really disgusts me how quick people are to bite the hand that feeds them. Heroku is not perfect, but they give a whole lot more value than they charge for. As unlikely as it is, I hope that Heroku terminates the account of anyone that joins the class action against them.
- jonknee 14y ago
  > As unlikely as it is, I hope that Heroku terminates the account of anyone that joins the class action against them. It would be hard to give worse legal advice than this.
  - jrockway 14y ago
    Oh come on, it's easy! "In my opinion as a lawyer, you should release radioactive bees into the refrigerators of your customers." Now that's bad legal advice.
- Karunamon 14y ago
  >It really disgusts me how quick people are to bite the hand that feeds them. Really? Because it really disgusts me when a company: * Misrepresents their product (saying it's A when it's actually B) * Privately acknowledges this misrepresentation under the radar * Makes large and ridiculous amounts of bank on this misrepresentation (since you have to throw more dynos at the app to mitigate the lag caused by the misrouting.. and Heroku may be many things, but economical it isn't) * And on that note, provides very expensive monitoring and performance analysis tools that further perpetuate the misrepresentation! * And then force you to resort to litigation to get some portion of this huge mis-spent investment refunded. (Looking at the suit, this is all that's being asked for) This wasn't a mistake on Heroku's part, this was calculated. And even if you accept that it was a mistake, the fact of the matter is that their negligence in this matter made them a hell of a lot of money. This is definitely grounds for a suit.
  - res0nat0r 14y ago
    By misrepresents you mean that the end users didn't exactly understand how underlying architecture worked, didn't like the performance, but kept using the service anyway. If every customer I dealt with while working on the EC2 platform could sue because they didn't understand how every technical detail of AWS worked and they couldn't optimize for it...well there would be more lawyers employed by Jeff Bezos than engineers. This lawsuit will go nowhere and it is sad that such a supposedly tech savvy crowd wants to bring out the pitchforks so fast. But I guess since Heroku has a lot of cash they are now the enemy since they've made it and they should be taken down.
jacques_chester 14y ago
I guess I saw this coming: http://news.ycombinator.com/item?id=5237920 http://news.ycombinator.com/item?id=5237920 http://news.ycombinator.com/item?id=5241686 http://news.ycombinator.com/item?id=5241686 Really, though, given how much money Heroku charge and how much money Salesforce.com would be able to stump up in a settlement or judgement ... it was inevitable that a fast-moving law firm would jump on this.
jrockway 14y ago
TLDR: RapGenius (YCS11), in a painful attempt to remain relevant, decides to sue their web host for running their slow code slowly.
- mikeryan 14y ago
  If you're going to tldr; an article you should really read it. Rap genius isn't suing. No one, in fact, is actually yet suing.
  - jrockway 14y ago
    The first line in the article "Rap Genius founder Tom Lehman says Heroku misled him and cost his company money." I have to imagine they're involved based on the number of articles I've read about them over the last week. Their own article, tech blogs interviewing them about the article, and now the founder's picture at the top of this article.
    - dandelany 14y ago
      Granted, the article is poorly written and seems to imply their involvement. But it explicitly states, near the end, that the law firm involved has yet to find any plaintiffs: He’s spoken with one customer, who has since backed off. “That being said, I am in the process of drafting a complaint, in case their feet warm up,” he told us. Furthermore, Tom (the dude pictured) commented in a thread above saying he didn't even hear about the lawsuit until Wired called him to interview him for the article. Consider putting away your "Jump to Conclusions" mat and joining us in the world of facts.
      - jrockway 14y ago
        You're right. I wish I could delete my comments.
- goronbjorn 14y ago
  RapGenius just recently raised $15m from Andreessen Horowitz; they hardly need to make 'painful attempt[s] to remain relevant.' And they're not even involved with this lawsuit.
piggity 14y ago
Only lawyers think it's a good idea to deploy lawyers into the routing mesh.
- raverbashing 14y ago
  But you can bet money will be intelligently routed to their pockets
raverbashing 14y ago
I think the money for this class action lawsuit would be better used if it would go for a sysadmin/devops person to actually deploy the systems involved. "Bait and switch"? If a couple of Dynos is not enough for your load, you should be moving to self-managed hosts
DigitalSea 14y ago
At the end of the day Heroku are in the wrong here. Performance issues which Heroku basically wouldn't admit until they were outed about them meant that companies would have invested tonnes of time and effort which in turn equals money on trying to fix issues that were caused by Heroku the whole time.
- bascule 14y ago
  I'm sorry, did I miss something since aphyr's analysis? http://aphyr.com/posts/278-timelike-2-everything-fails-all-the-time http://aphyr.com/posts/278-timelike-2-everything-fails-all-t... Why is Heroku "in the wrong" exactly?

178 points by wwarnerandrew 14y ago

>Heroku declined our request for further comment. “Unfortunately, we can’t publicly comment on legal matters,” said spokeswoman Dana Oshiro in an email. “Heroku is committed to our customers’ success and focused on delivering the best possible product and experience. We’ll continue to update our customers via our blog.” I don't run a company, and I'm not responsible for maximizing return on my shareholders' investment. So maybe I lack the mental schemas that make these sorts of statements seem reasonable, but they always strike me as short-sighted. Yes, admitting wrongdoing may open you up to legal liability. But it seems to me that, in the long run, it's far better to take your licks and own up to your mistakes than to hide behind the shield of pending litigation. My gut instinct is to offer a sincere mea culpa and provide a transparent roadmap to improvement, and if you lose a court case then so be it. I'd like to think that the good will and respect you build from accountability and honesty will ultimately outweigh any short-term losses from court settlements or judgements. As I said, I don't live in that world. I would be saddened, though not necessarily surprised, to learn that I was wrong.

biot 14y ago
What makes this proposed class action suit possible is that Heroku did admit[0] that their service didn't match up with what their documentation and website claimed, that they messed up, and would be fixing it. And it wasn't something like "our documentation and site could have been misinterpreted", they actually admitted the website and documentation were inaccurate with metrics which were incorrect. It's like a class action lawyer's wet dream of culpability on the part of the defendant. [0] https://blog.heroku.com/archives/2013/2/16/routing_performance_update https://blog.heroku.com/archives/2013/2/16/routing_performan...
kyrra 14y ago
Seeing that there is a website/lawfirm[1] threatening legal action, I can bet Heroku/Salesforce lawyers jumped on that ASAP and let all their employees that are allowed to talk publicly not to comment on the issue anymore. The US legal system is setup that if a company admits fault in something like this, it makes defending yourself much more difficult. Meaning if a lawsuit is brought about, it's not about proving innocence anymore, it's about mitigating the damages you will have to pay. Your lawyer is going to want to have as many options available to him/her to deal with your case during it's lifespan, and admitting fault before the lawsuit even begins will limit the lawyers options when it comes to negotiations and what tactics they can use in the courtroom. [1] http://herokuclassaction.com/ http://herokuclassaction.com/
- incongruity 14y ago
  In the grand scheme of things, this is why it's always better to defect in the simplified idea game of the prisoner's dilemma. Collaboration is rarely rewarded once lawyers get anywhere near an issue and opting to collaborate (i.e.: admit fault and try to fix it) more or less wins you no points in court, in the U.S. Given that each possible liability is its own case and stands on its own merits, it pretty much matches the simplest case of the prisoner's dilemma perfectly, IMHO... and so here we are. Sadly.
  - OGinparadise 14y ago
    Collaboration is rarely rewarded once lawyers get anywhere near an issue and opting to collaborate (i.e.: admit fault and try to fix it) more or less wins you no points in court, in the U.S. Might, in part, have to do with how lawyers get compensated. A short case or a quick settlement will result in very little compensation for them. Over the course of years and decades that matters so we see everything analyzed to death. At $500+ an hour. Per lawyer.
    - jacques_chester 14y ago
      The careful decomposition of problem domain into solution domain followed by recursive search of the solution domain is the lawyer's job. If they don't analyse to death, they are breaching their fiduciary duty. Don't like the legal method? Don't hire a lawyer. Try doing it yourself. See how far you get when the other guy's lawyers have done the exhaustive analysis and found the deciding element buried in a logical node on a distant subgraph you didn't even know was there to be explored.
      - OGinparadise 14y ago
        Yeah, other developed countries are all going downhill for not being overlawyered up like in US. It's a system set by lawyers for lawyers so of course you need a lawyer more than in many other countries. "Everyone" sues over everything
resu_nimda 14y ago
They have made public statements on their blog, and hopefully that will continue. In this situation, I think it is the right move to avoid getting involved with this shameless law firm trying to incite drama for their own gain.
- redthrowaway 14y ago
  This seems like precisely the type of situation class action lawsuits are intended to deal with: a company defrauded a lot of its customers in the same way. It makes more sense for all parties that the case be tried as one than to hear each claimant individually.
Adaptive 14y ago
Sadly, the US legal system does not reward heartfelt, honest statements, particularly in these situations. Which is why fine print almost always makes even the nicest company sound like a jerk.
anigbrowl 14y ago
There's a middle ground; it is better to fess up and try to square things with your customers in a situation where you've done wrong, but at the same time you don't want to be exploited by unscrupulous plaintiffs in turn. Most of those customers just want what they paid for or to recover their actual losses (what are called economic damages in law) but a few may see a lawsuit as an opportunity to enrich themselves. The court system is supposed to prevent that, and overall it works moderately well, but once papers are filed it's better for the arguments to take place in court rather than outside.
danso 14y ago
> Yes, admitting wrongdoing may open you up to legal liability. But it seems to me that, in the long run, it's far better to take your licks and own up to your mistakes than to hide behind the shield of pending litigation. This is an admirable, well-intentioned perspective, but it is one that won't net you much benefit, not even in the long run. Filing legal papers -- or, in this case, raising the banner for a class-action -- is serious business. By the time it happens, the time for defusing the situation and calling a truce is long past. The only opinion that matters is that of the judge and/or jury. And let's face it, being the loser in court is going to have far more impact on the public's opinion of the defendant than any honest-from-the-heart statement or outreach. So with that said, there is nothing of substance that the defendant can say that will help them in court. The opposing attorney is not going to say, "Well, I was going to grill you, but I've had a change of heart ever since reading that impassioned, honest, salt-of-the-earth statement you released pre-trial". The judge won't care either. And the jurors aren't supposed to even be aware of it. I think a civil trial is not much different than the situation portrayed in the famous "Don't talk to police" video: http://www.youtube.com/watch?v=6wXkI4t7nuc http://www.youtube.com/watch?v=6wXkI4t7nuc The people you're facing in court don't care that you've made an effort, post-lawsuit-filing, to be upfront and friendly. Anything you say will not help you, it can only hurt you. In this case, Heroku's reticence in speaking their mind outside of court is the right one.
OGinparadise 14y ago
This is normal, once legal threats are issued "we can't comment on legal issues...but customers are #1...blah blah" is the standard answer. A mea culpa right now might haunt you in the long term.

citricsquid 14y ago

If the decisions made by Heroku to misrepresent the product were made prior to the Salesforce acquisition (December 2010) do the founders of Heroku have any liability? Would this sort of thing be a consideration with an acquisition?

chc 14y ago
I don't believe any decision was made to misrepresent the product. The product just evolved and the product documentation didn't. The end result is the same — the product was advertised with inaccurate claims — but this appears to be more a case of neglect than a deliberate misrepresentation.
- jacques_chester 14y ago
  It's not a criminal matter. Mens rea -- "guilty conscience", the intention to act unlawfully -- is a component of crime. But it's not necessary for lots of civil legal matters. For example, if we both enter into a contract, you can hold me to my end. If I fail to hold up my end without intending to fail, then you can still insist that I make right or seek remedies for my failure. (IANAL, TINLA).
  - chc 14y ago
    That all sounds right to me, but I think you've mistsken my meaning. The point wasn't that they lack mens rea, but that the decision to misrepresent the product can't have been made before the acquisition because that decision was never made at all. It was misrepresented by reality gradually changing out from under initially truthful statements.
    - jacques_chester 14y ago
      You don't have to decide to misrepresent in order to be misrepresentative. If your material is wrong, it's wrong, and depending on the jurisdiction you may be on the hook. Ridiculous? Well consider that it gives a strong incentive to ensure that claims made to customers are somewhat accurate. Reducing the cost of caveat emptor makes society richer by reducing transaction costs and so encouraging trade. Sellers have a strong informational advantage over buyers; in general the cost to them of making a correct statement is much lower than having each customer having to find out the truth independently. Not that this is the purpose of torts law, which evolved mostly to fill glaring, unjust gaps in the remedies of the older forms of common law. But that is the social and economic function that it has come to serve. (IANAL, TINLA).

eli 14y ago

IANAL, but my understanding is that threatening a class action lawsuit is easy, but actually bringing an action to court is extremely difficult. And (at least to me) this seems like an awfully weak case -- I don't see any obvious or intentional fraud. The whole thing seems strange to me. I don't really understand what Rap Genius hopes to gain here. If Heroku wasn't providing enough performance for their money (regardless of technical cause), then why did they stick around so long? Would Rap Genius still have sued if the documentation had been 100% correct and instead the problem was just plain old slow I/O on Heroku's side?

niggler 14y ago
Ianal stands for I am not a lawyer, so "a lawyer" in "IANAL a lawyer" is redundant.
- eli 14y ago
  Hah, fixed. Sometimes my fingers get ahead of my brain when typing quickly.
tomlemon 14y ago
> If Heroku wasn't providing enough performance for their money (regardless of technical cause), then why did they stick around so long? We didn't know how bad the performance was because Heroku's tools (logs and New Relic) reported incorrect performance data (i.e., they said requests weren't queuing when they were actually spending a ton of time queuing) EDIT: > Would Rap Genius still have sued if the documentation had been 100% correct and instead the problem was just plain old slow I/O on Heroku's side? To be clear, we have no official association with http://herokuclassaction.com/ http://herokuclassaction.com/ or the lawyer behind it – I actually found out about the site from the article's author when he interviewed me for the story. (But I do think Heroku owes its customers a refund)
- jaggederest 14y ago
  New Relic isn't Heroku's tool, man. It's like a car dealership selling you a Pioneer stereo.

LAMike 14y ago

On one hand the Rap Genius expose seemed like they got duped and mislead, but on the other hand something about this lawyer mentioned in the article comes off as ambulance-chase-y to me.

lancefisher 14y ago
It's not really ambulance chasing if all he's after is a partial refund. From the article: As for Lehman, he wants some of the $20,000 his company pays Heroku each month returned. “We deserve some sort of refund here,” he says. EDIT: I re-read the article, and it's not even clear that Rap Genius is part of this. Looks like a lawyer trying to drum up support.
rayiner 14y ago
Class action lawyering usually sound ambulance chasey, because any given person doesn't lose that much in a given instance and so the actual people harmed have little incentive to sue. At the same time, it's an edge case in the economy. If you harm a large number of people a small amount, they won't sue you to recover for their injury.

dubcanada 14y ago

http://www.wired.com/wiredenterprise/2013/03/hieroku/ http://www.wired.com/wiredenterprise/2013/03/hieroku/ HIEROKU!? Ignoring the funny url. I doubt this will go anywhere besides in the domain expiry list.

lancefisher 14y ago

From the article, Heroku says, "The problem affected an older version of Heroku’s service, he said, and those who use a new version of the service would not have these issues." As far as I understand that is not really the case. The newer version is the Cedar stack which supports concurrent apps where Bamboo only supports non-concurrent apps. This would reduce the problem with random routing, but not eliminate it.

randall 14y ago
The problem is not that they provide random routing, the problem is they said in their documentation that they provided intelligent routing, a design choice they backed off of when they switched stacks. Customers still have a problem, but Heroku doesn't from a legal / class action-y perspective.

bridgeyman 14y ago

In MIT's Mathematics for Computer Scientists course (6.042), I remember learning from Tom Leighton that in certain cases randomly distributing jobs actually IS the optimal solution to load balancing. This technique was somehow associated with Akamai, Professor Leighton's company. Maybe I am misremembering...

w-ll 14y ago
I dont know, but I would assume it has to deal with the fact you can't predict how long a job will take to complete, and a random selection would have more resources-more free-more often, verses FIFO where a group of really large tasks could halt everything.
- kordless 14y ago
  That's exactly correct. Given jobs cleared out quick, that wouldn't necessarily be the case.
- manmal 14y ago
  And what if they turned the whole thing into a request queue and workers get the "messages" as they become available? To ensure there are no timeouts, they could define a max time a request may stay in the queue, and then take it out and handle it with emergency workers or kill it?
ww520 14y ago
I think the matter is if random distribution is what is documented, no one will raise an eyebrows. I believe the Heroku claimed intelligent routing based on server loads to route traffic to idle servers. That threw people off.
- jonknee 14y ago
  Not only that, but they charged a premium based on their false claims. I'm surprised it took this long for legal rumblings to come around.
14y ago

rdl 14y ago

Heh, nothing will make people go back to taking Heroku's side faster than a class action lawsuit on the other side. This would be an awesome form of submarine marketing if they were actually behind it :)

chubot 14y ago
Um, why? I was one of many plaintiffs in a class action lawsuit against EA (which was settled in our favor). That did not help EA's image at all :) It looks pretty bad when your customers complain about problems that are hidden by your own monitoring tools, and then you tell them to buy more crap to fix it! It's not a small amount either -- as mentioned the queuing time was larger than the request time. And on top of it you're paying out the nose for this crappy monitoring tool.
- rdl 14y ago
  Even people who have been somewhat screwed by this issue with Heroku tend to love Heroku; Salesforce could make this right with generous payouts to affected people and 2-6mo of engineering to offer a better load balancer as a service (Actually, I think I could build this as a third party provider, even) EA is basically a lost cause; their business model and culture is the problem. Having better sacks at Dachau wouldn't have helped the Nazi image, either (to auto-Godwin)
  - chubot 14y ago
    Well that's what they should have done BEFORE the threat of class action lawsuit. It makes them look at a lot worse if it happens after. I'm pretty surprised they didn't offer refunds already. It was pretty clear they were asked for, and with no update by now, it's safe to say they were refused. That seems unreasonable to me. If you were in the position RapGenius, would you honestly not have expected a refund already?
    - rdl 14y ago
      I wonder what happened to Heroku. They used to be awesome. They still have elements of being awesome, but seem either resource constrained or self destructive. I find it hard to believe Salesforce wouldn't give them what they need to do things correctly -- ranging from "go multi-AWS Region or move off AWS" to building a decent load balancing system to handling this fiasco.

scott_meade 14y ago

"[The attorney who set up this site] has spoken with one customer, who has since backed off. “That being said, I am in the process of drafting a complaint, in case their feet warm up,” he told us."

vertis 14y ago

It really disgusts me how quick people are to bite the hand that feeds them. Heroku is not perfect, but they give a whole lot more value than they charge for. As unlikely as it is, I hope that Heroku terminates the account of anyone that joins the class action against them.

jonknee 14y ago
> As unlikely as it is, I hope that Heroku terminates the account of anyone that joins the class action against them. It would be hard to give worse legal advice than this.
- jrockway 14y ago
  Oh come on, it's easy! "In my opinion as a lawyer, you should release radioactive bees into the refrigerators of your customers." Now that's bad legal advice.
Karunamon 14y ago
>It really disgusts me how quick people are to bite the hand that feeds them. Really? Because it really disgusts me when a company: * Misrepresents their product (saying it's A when it's actually B) * Privately acknowledges this misrepresentation under the radar * Makes large and ridiculous amounts of bank on this misrepresentation (since you have to throw more dynos at the app to mitigate the lag caused by the misrouting.. and Heroku may be many things, but economical it isn't) * And on that note, provides very expensive monitoring and performance analysis tools that further perpetuate the misrepresentation! * And then force you to resort to litigation to get some portion of this huge mis-spent investment refunded. (Looking at the suit, this is all that's being asked for) This wasn't a mistake on Heroku's part, this was calculated. And even if you accept that it was a mistake, the fact of the matter is that their negligence in this matter made them a hell of a lot of money. This is definitely grounds for a suit.
- res0nat0r 14y ago
  By misrepresents you mean that the end users didn't exactly understand how underlying architecture worked, didn't like the performance, but kept using the service anyway. If every customer I dealt with while working on the EC2 platform could sue because they didn't understand how every technical detail of AWS worked and they couldn't optimize for it...well there would be more lawyers employed by Jeff Bezos than engineers. This lawsuit will go nowhere and it is sad that such a supposedly tech savvy crowd wants to bring out the pitchforks so fast. But I guess since Heroku has a lot of cash they are now the enemy since they've made it and they should be taken down.

jacques_chester 14y ago

I guess I saw this coming: http://news.ycombinator.com/item?id=5237920 http://news.ycombinator.com/item?id=5237920 http://news.ycombinator.com/item?id=5241686 http://news.ycombinator.com/item?id=5241686 Really, though, given how much money Heroku charge and how much money Salesforce.com would be able to stump up in a settlement or judgement ... it was inevitable that a fast-moving law firm would jump on this.

jrockway 14y ago

TLDR: RapGenius (YCS11), in a painful attempt to remain relevant, decides to sue their web host for running their slow code slowly.

mikeryan 14y ago
If you're going to tldr; an article you should really read it. Rap genius isn't suing. No one, in fact, is actually yet suing.
- jrockway 14y ago
  The first line in the article "Rap Genius founder Tom Lehman says Heroku misled him and cost his company money." I have to imagine they're involved based on the number of articles I've read about them over the last week. Their own article, tech blogs interviewing them about the article, and now the founder's picture at the top of this article.
  - dandelany 14y ago
    Granted, the article is poorly written and seems to imply their involvement. But it explicitly states, near the end, that the law firm involved has yet to find any plaintiffs: He’s spoken with one customer, who has since backed off. “That being said, I am in the process of drafting a complaint, in case their feet warm up,” he told us. Furthermore, Tom (the dude pictured) commented in a thread above saying he didn't even hear about the lawsuit until Wired called him to interview him for the article. Consider putting away your "Jump to Conclusions" mat and joining us in the world of facts.
    - jrockway 14y ago
      You're right. I wish I could delete my comments.
goronbjorn 14y ago
RapGenius just recently raised $15m from Andreessen Horowitz; they hardly need to make 'painful attempt[s] to remain relevant.' And they're not even involved with this lawsuit.

piggity 14y ago

Only lawyers think it's a good idea to deploy lawyers into the routing mesh.

raverbashing 14y ago
But you can bet money will be intelligently routed to their pockets

raverbashing 14y ago

I think the money for this class action lawsuit would be better used if it would go for a sysadmin/devops person to actually deploy the systems involved. "Bait and switch"? If a couple of Dynos is not enough for your load, you should be moving to self-managed hosts

DigitalSea 14y ago

At the end of the day Heroku are in the wrong here. Performance issues which Heroku basically wouldn't admit until they were outed about them meant that companies would have invested tonnes of time and effort which in turn equals money on trying to fix issues that were caused by Heroku the whole time.

bascule 14y ago
I'm sorry, did I miss something since aphyr's analysis? http://aphyr.com/posts/278-timelike-2-everything-fails-all-the-time http://aphyr.com/posts/278-timelike-2-everything-fails-all-t... Why is Heroku "in the wrong" exactly?