5 ms·

From my personal experience, I wouldn't use a freelance tester because they won't understand the system well enough to provide value for money. The most valuab

by scott_w 14y ago

From my personal experience, I wouldn't use a freelance tester because they won't understand the system well enough to provide value for money. The most valuable testing is that which exposes your lack of understanding of what the system should do. These also usually require the most extensive rewrites, so you definitely want them as early as possible. Unfortunately, they also require the most extensive understanding of the system as well as an ability to detail their thinking at a level that can be understood by the development team(s). That's why I don't see the value of freelance testers, and I'd much rather have somebody employed on a more permanent basis, either part-time or full-time.

lifeisstillgood 14y ago
That's the same argument for not hiring freelance developers or project managers. And IMHO it leads to lazy coupling of software systems and services, a system that has to be built and tested by different or changing groups has to have much much better scope and decoupling. In short, if you have small (micro) services, they are easy to build and easier to test.
- scott_w 14y ago
  I disagree. With good testing discipline, you can bring in freelance developers. Developers have the luck of not needing 100% understanding, as this is grown through the writing of code. Testing, on the other hand, is a simulation of how your system responds to user input, and how your users respond to the system's outputs. If your tester can't simulate a user, then you can make your services as micro as you like - it'll make no difference when you get the phone call saying the system doesn't work as expected.
  - eru 14y ago
    If new people picking up the app have a problem simulating users, then how do you assume your users will go about picking up the up to actually use it?
    - scott_w 14y ago
      Read my comment below. It's not about users understanding the program per se, it's about testers having the domain knowledge to perform meaningful tests. Without that domain knowledge, the testers may find implementation bugs, but they won't expose the issues caused by the developer's lack of understanding. The latter is almost always more valuable to find and more difficult to fix.
      - eru 14y ago
        Oh, ok. I guess writing automatic tests is one way to nudge developers into understanding the domain better. And getting a person on the team, even freelance, whose job it is to push for tests, might help. Going off another tangent, getting not a generic tester, but a domain expert who's also a tester, might help.
SODaniel 14y ago
As true as your comment is I would like to point out that there is a value in itself to have 'fresh eyes' on QA. An experienced and talented tester unfamiliar with a system/product/service can actually work as both QA/UI and UX resource if he or she has the right background. TLDR; Sometimes you can gain valuable input from a fresh pair of eyes.
Too 14y ago
There's a big difference between blackbox testing and whitebox testing. For blackbox testing, having someone who doesn't now the inner workings of the system to find workarounds can many times be an advantage. Of course you should do both kinds for security critical projects.
- scott_w 14y ago
  I'm not arguing for black/white box testing, I'm arguing that for testing to have real value, your tester needs to behave in a similar manner to a real user. That way you find the gap between your own understanding and your users' understanding of the system. Take a concrete example: a piece of medical equipment prints off reports. If your tester has no medical knowledge, he doesn't have a frame of reference to know if the numbers are correct or not. He's also more inclined to use it exactly how you specify. A real doctor would bring his own experience of similar equipment and try to use it in ways you didn't think of. In addition, he will be able to gauge whether the reports "look right".
  - nicolasedgwick 14y ago
    Very good example! And there is an absolutely huge correlation between good testing and domain experience irrespective of freelance/permanent. I've seen entire teams of permanent testers miss critical bugs because they're busy stressing the system and trying edge cases but don't actually use it like a user because they don't know their users.
nicolasedgwick 14y ago
The question then is how do you assess value for money before you use the service. I work as a freelance tester and I often see that customers see the value after working with me and then return for repeat business (as someone else has mentioned - you build a reputation). As an example, an error in an iPad app was dismissed as a known connection issue by the dev team because there were no server side logs so the request wasn't hitting the server. This was the dev team's first iOS app. My specialist knowledge of mobile testing helped identify quickly that the app was connecting to the servers but that this one request wasn't and provide console logs showing where the communication error was occurring. This customer were to launch in public the following week and this would have killed their reputation due to the critical nature of the function that was failing. This was my first day testing with them after they had put in a year of development effort and dev testing (they brought me in specifically because they felt they were getting "tunnel vision"). This was one bug masking behind another which I spotted due to 3 years experience in testing mobile apps. I think that provides value for money without being employed permanently, however I appreciate that companies/teams don't see that value until it's their reputation that has been protected by the investment.

by scott_w 14y ago

lifeisstillgood 14y ago

That's the same argument for not hiring freelance developers or project managers. And IMHO it leads to lazy coupling of software systems and services, a system that has to be built and tested by different or changing groups has to have much much better scope and decoupling. In short, if you have small (micro) services, they are easy to build and easier to test.

scott_w 14y ago
I disagree. With good testing discipline, you can bring in freelance developers. Developers have the luck of not needing 100% understanding, as this is grown through the writing of code. Testing, on the other hand, is a simulation of how your system responds to user input, and how your users respond to the system's outputs. If your tester can't simulate a user, then you can make your services as micro as you like - it'll make no difference when you get the phone call saying the system doesn't work as expected.
- eru 14y ago
  If new people picking up the app have a problem simulating users, then how do you assume your users will go about picking up the up to actually use it?
  - scott_w 14y ago
    Read my comment below. It's not about users understanding the program per se, it's about testers having the domain knowledge to perform meaningful tests. Without that domain knowledge, the testers may find implementation bugs, but they won't expose the issues caused by the developer's lack of understanding. The latter is almost always more valuable to find and more difficult to fix.
    - eru 14y ago
      Oh, ok. I guess writing automatic tests is one way to nudge developers into understanding the domain better. And getting a person on the team, even freelance, whose job it is to push for tests, might help. Going off another tangent, getting not a generic tester, but a domain expert who's also a tester, might help.

SODaniel 14y ago

As true as your comment is I would like to point out that there is a value in itself to have 'fresh eyes' on QA. An experienced and talented tester unfamiliar with a system/product/service can actually work as both QA/UI and UX resource if he or she has the right background. TLDR; Sometimes you can gain valuable input from a fresh pair of eyes.

Too 14y ago

There's a big difference between blackbox testing and whitebox testing. For blackbox testing, having someone who doesn't now the inner workings of the system to find workarounds can many times be an advantage. Of course you should do both kinds for security critical projects.

scott_w 14y ago
I'm not arguing for black/white box testing, I'm arguing that for testing to have real value, your tester needs to behave in a similar manner to a real user. That way you find the gap between your own understanding and your users' understanding of the system. Take a concrete example: a piece of medical equipment prints off reports. If your tester has no medical knowledge, he doesn't have a frame of reference to know if the numbers are correct or not. He's also more inclined to use it exactly how you specify. A real doctor would bring his own experience of similar equipment and try to use it in ways you didn't think of. In addition, he will be able to gauge whether the reports "look right".
- nicolasedgwick 14y ago
  Very good example! And there is an absolutely huge correlation between good testing and domain experience irrespective of freelance/permanent. I've seen entire teams of permanent testers miss critical bugs because they're busy stressing the system and trying edge cases but don't actually use it like a user because they don't know their users.

nicolasedgwick 14y ago

The question then is how do you assess value for money before you use the service. I work as a freelance tester and I often see that customers see the value after working with me and then return for repeat business (as someone else has mentioned - you build a reputation). As an example, an error in an iPad app was dismissed as a known connection issue by the dev team because there were no server side logs so the request wasn't hitting the server. This was the dev team's first iOS app. My specialist knowledge of mobile testing helped identify quickly that the app was connecting to the servers but that this one request wasn't and provide console logs showing where the communication error was occurring. This customer were to launch in public the following week and this would have killed their reputation due to the critical nature of the function that was failing. This was my first day testing with them after they had put in a year of development effort and dev testing (they brought me in specifically because they felt they were getting "tunnel vision"). This was one bug masking behind another which I spotted due to 3 years experience in testing mobile apps. I think that provides value for money without being employed permanently, however I appreciate that companies/teams don't see that value until it's their reputation that has been protected by the investment.