5 ms·

Show HN: I built testelf - app testing service for your iOS app

77 points by ssong 14y ago

ssong 14y ago
OP here. I quit my full-time job almost a year ago to travel and work on startup ideas, and this is the fruit of my labor in the past six months that I hope you'll find useful. I wanted to make it easy for startups and indie devs to test their iOS applications. How does testelf work? You upload a build of your app without providing any additional instructions, and we'll test your app from a user's perspective, logging what we did or saw. The goal is to charge a flat price per test, provide good coverage and actionable feedback, and have a 36-hour turnaround time. Over the past few months, I recruited and trained a small team of testers, developed internal testing tools, and worked with a handful of startups who are piloting our service. The service is now open for beta -- I would love to hear feedback from hackers and startups on HN who can give it a shot.
- matthuggins 14y ago
  "You upload a build of your app without providing any additional instructions, and we'll test your app from a user's perspective..." It seems like a user should have at least some idea of what the app is. For example, I'll read the first couple sentences of an app's description and look at some screenshots/video before I decide to download it. This might be something you want to consider instead of having absolutely no idea what the app is before running it, since I assume most users have at least some context when opening an app.
  - Cushman 14y ago
    It's usually a good idea when user testing to tell testers as little as possible about the product. It's true that real-world users have more context than that, but it can be worth it to get more unfiltered feedback— the first few moments you watch a user flounder to even understand what they're looking at can be the most valuable.
  - ssong 14y ago
    Good point! There is an option to attach additional information (description, notes, link to website/app store) when uploading an individual build. We do ask for a little bit of information about the app in general when the first build is uploaded so that we have some context.
- jmatthew3 14y ago
  This is a pretty cool idea, and something I would have considered trying to get my boss to pay for when I was a developer. I didn't see on your website - do you test on a wide variety of hardware platforms, or just iPhone 5 and iPad? Different versions of iOS? Also, do you have any terms, in particular, regarding confidentiality?
  - ssong 14y ago
    Thanks for asking! We do test on common devices (retina/non-retina iPad, iOS 5/6), and we're going to carefully consider which devices to cover as we grow. I am also very aware of the importance of confidentiality, such as keeping test results private -- I think the service wouldn't make sense without it. Our exact terms will be available before we publicly launch.
- kevining 14y ago
  Brilliant idea for a business. I would love if you guys would do simple games some day. Hoping to give your service a try soon.
thelarry 14y ago
Pretty cool, I have a rough time getting my lazy friends to test my iphone apps :/
eclipxe 14y ago
This is very cool. Simple but useful. Great work!
timdorr 14y ago
Any reason this is limited to iOS? It seems like it could extend to Android/Windows/etc. extremely easily.
- ssong 14y ago
  Thanks for the comment! Android is probably the next platform for us if all goes well. I started off with iOS because I'm more familiar with it and it has less device variations. Extending the offering won't be trivial though. Before we start support for a new platform, we'd need to build the testing tools, develop new training materials, and acquire new hardware.
tachim 14y ago
Awesome! This looks super useful, I'll definitely give it a shot with my own apps.
jplehmann 14y ago
Good. I'm tired of monkeys. We need something more intelligent.
umut 14y ago
Congrats, looks really nice! (already forwarded to our testing team) If you add a 'proxy layer' to your solution, that would be awesome. To explain the scenario, it is really very valuable for us app startups to see how it behaves in China, in India, in Europe, in US etc. You may keep your current system intact, and 'if' the customer desires so, you can route your connection over those proxies(from real ISS'es) from all over the world and test it 'as if...' This is really important and not so hard to implement... But sometimes, you can catch bugs and usability horrors with slow or dropped connection or some weird proxy mambo jambo played by ISS providers.
- cstrat 14y ago
  I think this is a great idea too! It would be an easy upsell to international developers.
speeder 14y ago
You wrote that you don't test games... Well, I hope we can send our more non-gameish apps :)
gmig 14y ago
Why is "No provisioning necessary"?
- akaru 14y ago
  This is a good question.
PanMan 14y ago
How do you handle UUID signing? Do you tell me the UUID's I need to add? Just upload seems to not work with iOS apps, right?
- pooriaazimi 14y ago
  You can sign ipa files with your developer (private) key. There's an app called InstaSign that I use for "testing" apps before purchasing them. You download a cracked version, sign it with your own key (so iPad thinks YOU have created this app, like when you do in Xcode) and see if it's worthwhile. It's saved me a lot of money that otherwise would've wasted on scammy, buggy, "not-working-as-advertised" apps. If you're a developer (i.e., can sign apps and load them unto your device), give InstaSign a try. They're probably using a similar method (as they seemingly test on iPad/iPhone, retina/non-retina, iOS5/6 devices and it might be a bit more work for you to add at least 6 new UDIDs).
  - akaru 14y ago
    I was wondering how you did that. How do we know you're not simply cracking these apps to resell? It sounds a bit fishy is all.
    - pooriaazimi 14y ago
      Just to be clear, I'm not with testelf! Just wanted to point that out in case you get the wrong impression of them not replying to your (fair) accusation :)
shawiz 14y ago
Congrats! Very useful service.
king_magic 14y ago
This looks great, and I think it's a really good price point. $200 is quite reasonable, IMO, for the kind of detail you guys are getting into. Definitely going to consider this for future testing :)
thomseddon 14y ago
Looks excellently useful, good job. I must say the graph on the test page (http://beta.testelf.com/builds/share/0b78d05c4f5d11e2ab5a22000a1f869a/ http://beta.testelf.com/builds/share/0b78d05c4f5d11e2ab5a220...) is particularly wonderful, great use of D3. Also, would be very interested to know if you have plans as to how (or if) you will overcome the issues you would have if you wanted to scale such a bespoke service?
hariseshadri 14y ago
This looks fantastic. Great work.
swanify 14y ago
Looks great, i run a mobile app agency and we are forever having issues due to developers not testing adequately, having some external company test would be really useful!
ianstallings 14y ago
This is a great idea sir. I signed up and hope to hear from you guys. I can use more eyeballs looking over our work. I really like not having to worry about provisions.

77 points by ssong 14y ago

ssong 14y ago

OP here. I quit my full-time job almost a year ago to travel and work on startup ideas, and this is the fruit of my labor in the past six months that I hope you'll find useful. I wanted to make it easy for startups and indie devs to test their iOS applications. How does testelf work? You upload a build of your app without providing any additional instructions, and we'll test your app from a user's perspective, logging what we did or saw. The goal is to charge a flat price per test, provide good coverage and actionable feedback, and have a 36-hour turnaround time. Over the past few months, I recruited and trained a small team of testers, developed internal testing tools, and worked with a handful of startups who are piloting our service. The service is now open for beta -- I would love to hear feedback from hackers and startups on HN who can give it a shot.

matthuggins 14y ago
"You upload a build of your app without providing any additional instructions, and we'll test your app from a user's perspective..." It seems like a user should have at least some idea of what the app is. For example, I'll read the first couple sentences of an app's description and look at some screenshots/video before I decide to download it. This might be something you want to consider instead of having absolutely no idea what the app is before running it, since I assume most users have at least some context when opening an app.
- Cushman 14y ago
  It's usually a good idea when user testing to tell testers as little as possible about the product. It's true that real-world users have more context than that, but it can be worth it to get more unfiltered feedback— the first few moments you watch a user flounder to even understand what they're looking at can be the most valuable.
- ssong 14y ago
  Good point! There is an option to attach additional information (description, notes, link to website/app store) when uploading an individual build. We do ask for a little bit of information about the app in general when the first build is uploaded so that we have some context.
jmatthew3 14y ago
This is a pretty cool idea, and something I would have considered trying to get my boss to pay for when I was a developer. I didn't see on your website - do you test on a wide variety of hardware platforms, or just iPhone 5 and iPad? Different versions of iOS? Also, do you have any terms, in particular, regarding confidentiality?
- ssong 14y ago
  Thanks for asking! We do test on common devices (retina/non-retina iPad, iOS 5/6), and we're going to carefully consider which devices to cover as we grow. I am also very aware of the importance of confidentiality, such as keeping test results private -- I think the service wouldn't make sense without it. Our exact terms will be available before we publicly launch.
kevining 14y ago
Brilliant idea for a business. I would love if you guys would do simple games some day. Hoping to give your service a try soon.

thelarry 14y ago

Pretty cool, I have a rough time getting my lazy friends to test my iphone apps :/

eclipxe 14y ago

This is very cool. Simple but useful. Great work!

timdorr 14y ago

Any reason this is limited to iOS? It seems like it could extend to Android/Windows/etc. extremely easily.

ssong 14y ago
Thanks for the comment! Android is probably the next platform for us if all goes well. I started off with iOS because I'm more familiar with it and it has less device variations. Extending the offering won't be trivial though. Before we start support for a new platform, we'd need to build the testing tools, develop new training materials, and acquire new hardware.

tachim 14y ago

Awesome! This looks super useful, I'll definitely give it a shot with my own apps.

jplehmann 14y ago

Good. I'm tired of monkeys. We need something more intelligent.

umut 14y ago

Congrats, looks really nice! (already forwarded to our testing team) If you add a 'proxy layer' to your solution, that would be awesome. To explain the scenario, it is really very valuable for us app startups to see how it behaves in China, in India, in Europe, in US etc. You may keep your current system intact, and 'if' the customer desires so, you can route your connection over those proxies(from real ISS'es) from all over the world and test it 'as if...' This is really important and not so hard to implement... But sometimes, you can catch bugs and usability horrors with slow or dropped connection or some weird proxy mambo jambo played by ISS providers.

cstrat 14y ago
I think this is a great idea too! It would be an easy upsell to international developers.

speeder 14y ago

You wrote that you don't test games... Well, I hope we can send our more non-gameish apps :)

gmig 14y ago

Why is "No provisioning necessary"?

akaru 14y ago
This is a good question.

PanMan 14y ago

How do you handle UUID signing? Do you tell me the UUID's I need to add? Just upload seems to not work with iOS apps, right?

pooriaazimi 14y ago
You can sign ipa files with your developer (private) key. There's an app called InstaSign that I use for "testing" apps before purchasing them. You download a cracked version, sign it with your own key (so iPad thinks YOU have created this app, like when you do in Xcode) and see if it's worthwhile. It's saved me a lot of money that otherwise would've wasted on scammy, buggy, "not-working-as-advertised" apps. If you're a developer (i.e., can sign apps and load them unto your device), give InstaSign a try. They're probably using a similar method (as they seemingly test on iPad/iPhone, retina/non-retina, iOS5/6 devices and it might be a bit more work for you to add at least 6 new UDIDs).
- akaru 14y ago
  I was wondering how you did that. How do we know you're not simply cracking these apps to resell? It sounds a bit fishy is all.
  - pooriaazimi 14y ago
    Just to be clear, I'm not with testelf! Just wanted to point that out in case you get the wrong impression of them not replying to your (fair) accusation :)

shawiz 14y ago

Congrats! Very useful service.

king_magic 14y ago

This looks great, and I think it's a really good price point. $200 is quite reasonable, IMO, for the kind of detail you guys are getting into. Definitely going to consider this for future testing :)

thomseddon 14y ago

Looks excellently useful, good job. I must say the graph on the test page (http://beta.testelf.com/builds/share/0b78d05c4f5d11e2ab5a22000a1f869a/ http://beta.testelf.com/builds/share/0b78d05c4f5d11e2ab5a220...) is particularly wonderful, great use of D3. Also, would be very interested to know if you have plans as to how (or if) you will overcome the issues you would have if you wanted to scale such a bespoke service?

hariseshadri 14y ago

This looks fantastic. Great work.

swanify 14y ago

Looks great, i run a mobile app agency and we are forever having issues due to developers not testing adequately, having some external company test would be really useful!

ianstallings 14y ago

This is a great idea sir. I signed up and hope to hear from you guys. I can use more eyeballs looking over our work. I really like not having to worry about provisions.