4 ms·

Sauce Labs launches support for highly-secure OSX & iOS automated testing

49 points by sgrove 14y ago

rdpfeffer 14y ago
OK, this is pretty impressive. What do you guys have for native apps? Could I use the built-in tools that come with instruments to write my iOS tests? For example, I've tested Apps using IMAT (https://code.intuit.com/sf/projects/ginsu/ https://code.intuit.com/sf/projects/ginsu/) Could I do this with sauce? (Full disclosue, I wrote IMAT while I was working at Intuit, not sure if it works with iOS 6, or the latest Xcode)
- hugs 14y ago
  Yesterday's announcement was about mobile web testing. Of course, native app testing is important, too, and high on our radar. In the native app space, automated testing tools are not as mature as they are for the web. For iOS native app testing, I'm investigating tools written on top of Apple's UIAutomation API -- specifically the "ios-auto" project by Dan Cuellar at Zoosk. It lets you write your tests in any language (he uses C#!), and then proxies commands to the Instruments app. https://github.com/penguinho/ios-auto https://github.com/penguinho/ios-auto The goal is to let you test using the Selenium WebDriver API, but with Instruments/UIAutomation seamlessly running as the test's backend. [edit: grammar]
- hugs 14y ago
  I know Intuit is doing cool things with mobile testing. I'll be sure to check out IMAT! :-)
w1150n 14y ago
This is massive! Folks were just asking me if this was possible 2 days ago and I said "No one is doing it", I was wrong for 1 day... Looking forward to trying it!
- maudineormsby 14y ago
  We put together a blog post and screencast to explain some of the 'extras' for mobile: http://sauceio.com/index.php/2012/08/testing-your-site-with-mac-os-ios-and-android/ http://sauceio.com/index.php/2012/08/testing-your-site-with-...
defied 14y ago
Our own startup, testingbot.com has been offering the same OS X + iOS Selenium testing for a couple of months. Turns out the iOS simulator has a couple of quirks, the weirdest thing being it very rarely renders an iPhone view inside the iPad simulator. It's a lot of work to get this stuff work stable all the time.
- hugs 14y ago
  Yup, simulators do have quirks. So just in case, that's why I built this: http://www.youtube.com/watch?v=ABiTxApuLLU http://www.youtube.com/watch?v=ABiTxApuLLU
- hugs 14y ago
  Defied, glad you got it working stable. Did you contribute your fixes back to the Selenium project?
saurik 14y ago
There is a mention of security in one of the paragraphs, and it is only because the people running the testing platform have to make certain that you can't use a bug in the automated browser framework (Selenium) they are using to take over their network. When I read this title (which was modified from the original for HN) it made me wonder "how is security a key selling point for a testing framework", and then I found out that it wasn't ;P.
- brainsik 14y ago
  As a customer, security is important when you are testing your internal, firewalled, infrastructure. This is one of the main reasons Sauce Labs (where I work) gives every test its own VM that's destroyed afterwards. Using a shared or reused VM opens the door to malicious code running in the background. There's an article devoted to this topic here: Security Through Purity — http://sauceio.com/index.php/2011/09/security-through-purity/ http://sauceio.com/index.php/2011/09/security-through-purity...
  - saurik 14y ago
    Interesting: thanks for that perspective; to be clear, the reason I commented on this at all is because the title didn't make it clear why the security was interesting and the title was different from the title of the article (which is part of a massive debate about whether editorializing of titles should happen on HN; currently, the rules say you shouldn't do it, but personally I think it might be important in numerous occasions), but then the article itself didn't seem to indicate that the security was actually that key to the whole thing. (In particular, the one phrase "highly secure" that is used in the first sentence of the last paragraph seems like a frill to make the sentence sound more awesome: it didn't come after any motivation about how this is a key problem with other solutions, or that that is why the "enterprise" tag is on the idea.) Again: thanks!

49 points by sgrove 14y ago

OK, this is pretty impressive. What do you guys have for native apps? Could I use the built-in tools that come with instruments to write my iOS tests? For example, I've tested Apps using IMAT (https://code.intuit.com/sf/projects/ginsu/ https://code.intuit.com/sf/projects/ginsu/) Could I do this with sauce? (Full disclosue, I wrote IMAT while I was working at Intuit, not sure if it works with iOS 6, or the latest Xcode)

hugs 14y ago
Yesterday's announcement was about mobile web testing. Of course, native app testing is important, too, and high on our radar. In the native app space, automated testing tools are not as mature as they are for the web. For iOS native app testing, I'm investigating tools written on top of Apple's UIAutomation API -- specifically the "ios-auto" project by Dan Cuellar at Zoosk. It lets you write your tests in any language (he uses C#!), and then proxies commands to the Instruments app. https://github.com/penguinho/ios-auto https://github.com/penguinho/ios-auto The goal is to let you test using the Selenium WebDriver API, but with Instruments/UIAutomation seamlessly running as the test's backend. [edit: grammar]
hugs 14y ago
I know Intuit is doing cool things with mobile testing. I'll be sure to check out IMAT! :-)

w1150n 14y ago

This is massive! Folks were just asking me if this was possible 2 days ago and I said "No one is doing it", I was wrong for 1 day... Looking forward to trying it!

maudineormsby 14y ago
We put together a blog post and screencast to explain some of the 'extras' for mobile: http://sauceio.com/index.php/2012/08/testing-your-site-with-mac-os-ios-and-android/ http://sauceio.com/index.php/2012/08/testing-your-site-with-...

defied 14y ago

Our own startup, testingbot.com has been offering the same OS X + iOS Selenium testing for a couple of months. Turns out the iOS simulator has a couple of quirks, the weirdest thing being it very rarely renders an iPhone view inside the iPad simulator. It's a lot of work to get this stuff work stable all the time.

hugs 14y ago
Yup, simulators do have quirks. So just in case, that's why I built this: http://www.youtube.com/watch?v=ABiTxApuLLU http://www.youtube.com/watch?v=ABiTxApuLLU
hugs 14y ago
Defied, glad you got it working stable. Did you contribute your fixes back to the Selenium project?

saurik 14y ago

There is a mention of security in one of the paragraphs, and it is only because the people running the testing platform have to make certain that you can't use a bug in the automated browser framework (Selenium) they are using to take over their network. When I read this title (which was modified from the original for HN) it made me wonder "how is security a key selling point for a testing framework", and then I found out that it wasn't ;P.

brainsik 14y ago
As a customer, security is important when you are testing your internal, firewalled, infrastructure. This is one of the main reasons Sauce Labs (where I work) gives every test its own VM that's destroyed afterwards. Using a shared or reused VM opens the door to malicious code running in the background. There's an article devoted to this topic here: Security Through Purity — http://sauceio.com/index.php/2011/09/security-through-purity/ http://sauceio.com/index.php/2011/09/security-through-purity...
- saurik 14y ago
  Interesting: thanks for that perspective; to be clear, the reason I commented on this at all is because the title didn't make it clear why the security was interesting and the title was different from the title of the article (which is part of a massive debate about whether editorializing of titles should happen on HN; currently, the rules say you shouldn't do it, but personally I think it might be important in numerous occasions), but then the article itself didn't seem to indicate that the security was actually that key to the whole thing. (In particular, the one phrase "highly secure" that is used in the first sentence of the last paragraph seems like a frill to make the sentence sound more awesome: it didn't come after any motivation about how this is a key problem with other solutions, or that that is why the "enterprise" tag is on the idea.) Again: thanks!