3 ms·

I'd hesitate to blame the author for the issues that arose with the project, as the larger issues lie with Elance (et al)'s feedback mechanism. It should be rea

by jmduke 13y ago

I'd hesitate to blame the author for the issues that arose with the project, as the larger issues lie with Elance (et al)'s feedback mechanism. It should be reasonable to assume that someone with, say, a 4.8/5 rating is a sterling developer, but instead feedback gets perverted into a form of currency to be exploited as opposed to a canonical source of information -- just like what happened with eBay. I think a possible solution for this issue would be to include third-party anonymous auditing: get someone with domain expertise to review the spec, the final product, and communications (without being given the personal details of the two other parties, of course) and give an honest rating of both sides. Obviously, this has scaling issues, but I think a decent amount of people would be willing to pay a premium for a better value proxy.

cpayne 13y ago
(Like everyone else) Elance are out to make a dollar. The system isn't perfect, but show me a business that is. I think it was HIGHLY optimistic to have a budget of less than $1,000 and to think things would be pain free. It would have been interesting if the author had added what he had expected. The article is highly factual (this happened, and then this and then this...) but he didn't add any value judgment. What was his expectations? Was it a red flag that no one on Reddit wanted to touch it? Would he do it again? If he had twice the budget, would that have changed things? etc. etc.
vassvdm 13y ago
I think you're on to something. Ratings on sites like Elance are broken. Clients feel compelled to give 5 star feedback to freelancers, sometimes just to get them off their back. This basically means that you can't look at a reputation score to tell you how good the freelancer actually is. I like the idea of a third-party anonymous person doing an audit. That person would need to have access to all the relevant files though, and have the right skills to judge the freelancer's work. Even the prospect of being audited could be a deterrent to the kind of behavior we saw in this case, which means that randomly auditing one out of every x gigs could be enough.
- chii 13y ago
  Real life jobs/recruiting solves this problem via references. Why isn't this something elance can provide? You can get an anonymized contact info for previous clients of the dev. You can ping them for some questions etc.
  - richiepear 13y ago
    Good idea - right now, the best proxy for this is to look at the % of clients who re-hire and their comments. While I agree that 5 star systems across the Internet are inflated, I always use rehire % and really scour the comments when hiring on Elance.
saraid216 13y ago
This is something that needs to be more widely understood. Direct feedback is gamed. Even Amazon reviews, which are more indirect than even professional reviews, are gamed. It's a common response to the need for regulatory systems that we should "trust reputation systems" instead. This is what happens. Regulatory systems can at least be audited and held accountable. In reputation systems, absent an unforeseen innovation, you'll never be able to do better than currency. That's how reputation has been used since the dawn of man. It has always been currency. Sites like Elance merely made efficient.

by jmduke 13y ago

cpayne 13y ago

(Like everyone else) Elance are out to make a dollar. The system isn't perfect, but show me a business that is. I think it was HIGHLY optimistic to have a budget of less than $1,000 and to think things would be pain free. It would have been interesting if the author had added what he had expected. The article is highly factual (this happened, and then this and then this...) but he didn't add any value judgment. What was his expectations? Was it a red flag that no one on Reddit wanted to touch it? Would he do it again? If he had twice the budget, would that have changed things? etc. etc.

vassvdm 13y ago

I think you're on to something. Ratings on sites like Elance are broken. Clients feel compelled to give 5 star feedback to freelancers, sometimes just to get them off their back. This basically means that you can't look at a reputation score to tell you how good the freelancer actually is. I like the idea of a third-party anonymous person doing an audit. That person would need to have access to all the relevant files though, and have the right skills to judge the freelancer's work. Even the prospect of being audited could be a deterrent to the kind of behavior we saw in this case, which means that randomly auditing one out of every x gigs could be enough.

chii 13y ago
Real life jobs/recruiting solves this problem via references. Why isn't this something elance can provide? You can get an anonymized contact info for previous clients of the dev. You can ping them for some questions etc.
- richiepear 13y ago
  Good idea - right now, the best proxy for this is to look at the % of clients who re-hire and their comments. While I agree that 5 star systems across the Internet are inflated, I always use rehire % and really scour the comments when hiring on Elance.

saraid216 13y ago

This is something that needs to be more widely understood. Direct feedback is gamed. Even Amazon reviews, which are more indirect than even professional reviews, are gamed. It's a common response to the need for regulatory systems that we should "trust reputation systems" instead. This is what happens. Regulatory systems can at least be audited and held accountable. In reputation systems, absent an unforeseen innovation, you'll never be able to do better than currency. That's how reputation has been used since the dawn of man. It has always been currency. Sites like Elance merely made efficient.