13 ms·
Amazon has no idea how to run an app store
- danbmil99 11y agoFun fact: Jeff Bezos won't let anyone at Amazon buy a color printer.
- ceejayoz 11y agoIn fairness, it seems this is true for Apple (broken review process) and Google (malware infestation). Oddly, I haven't heard one horror story about the Windows Phone app store.
- VeilEm 11y agoGoogle Play does not have a "malware infestation". I have never once downloaded any malware, lookout has never detected any malware on my phone and I've never known anyone who has downloaded malware from Google Play. Anecdotal for sure, but given you said "infestation" you'd think I'd be crazy to download anything at all from Google Play. Absolute nonsense.
- HillRat 11y agoActually, based in the stats I've seen in the past (I can go hunting for them if needed), you would be crazy to download "anything at all" from the Play store, in the sense that the probability of a randomly-selected app harboring malware or privacy-invading practices is much higher than in the Apple App Store. But this is also like saying that you'd be insane to follow a random URL from your inbox; any random link is probably a phishing site or spam Viagra store, but you are unlikely to select one of those URLs by intent. Likewise, the vast majority of dangerous Android apps aren't ever going to appear on your radar, but that doesn't mean they aren't out there, just that they're poorly-targeted. (The Windows store, by contrast, was at one point full of questionable apps impersonating major brands, which is a much more obvious threat model.)
- JustSomeNobody 11y agoIndeed. This is so overblown it is pathetic. Edit: I wonder what smut the down voters have been downloading that may have given them the malwares. Reputable apps are fine. And there's no infestation. Just stahp.
- detaro 11y agoMalware-containing apps have millions of downloads, and there are/were thousands of them. One might argue if that is an "infestation", given the much larger number of apps in the store, but it also isn't something that has only happened once or twice. And when it comes to e.g. games there isn't much of reputation information outside a very small set of hits. No need to download "smut".
- eloisant 11y agoIt doesn't have a "malware infestation" problem, but rather a "arbitrary suspend/ban" problem.
- megablast 11y ago> Google Play does not have a "malware infestation". I have never once downloaded any malware, lookout has never detected any malware on my phone and I've never known anyone who has downloaded malware from Google Play. Seriously, what is the point of saying that? There are lots of articles about the problem, but you stick to your own experiences then!
- BillinghamJ 11y agoWhat is broken about Apple's review process? Sure it's stringent, but is that not a good thing?
- um_ya 11y agoWell lets see... You have to pay $99 a year, you need an apple computer to add an app to their store, it takes weeks or months to add an app, and I'm sure there's more.
- BillinghamJ 11y agoNone of those things are problems with the review process.
- chetanahuja 11y ago"takes weeks or months to add an app" I'd say that's a problem with the review process.
- BillinghamJ 11y agoAs an iOS user personally, I greatly value the diligence, so I'm not sure I'd call it a problem.
- JustSomeNobody 11y agoYou can google and read about lots of frustrated developers trying to get their app into the App Store. Nebulous responses from Apple, no response from Apple, rediculous response from Apple. It's all out there.
- LegNeato 11y agoA lot. It's the largest pain point shipping on iOS. More specific details and some suggestions: http://christian.legnitto.com/blog/2016/01/02/ways-i-would-fix-the-app-store-app-review-part-1-of-9/ http://christian.legnitto.com/blog/2016/01/02/ways-i-would-f...
- FigBug 11y agoThe Windows store is flooded with fake and scam apps. http://www.howtogeek.com/194993/the-windows-store-is-a-cesspool-of-scams-why-doesnt-microsoft-care/ http://www.howtogeek.com/194993/the-windows-store-is-a-cessp... http://forums.windowscentral.com/general-windows-phone-discussion-lounge/357587-rip-off-apps-flooding-store.html http://forums.windowscentral.com/general-windows-phone-discu...
- yeukhon 11y agoI was going to say the same, as we have been reading about the problems in Mac App Store lately on HN.
- lkrubner 11y agoThis is perhaps the key bit of silliness: "And this is where we run into the first bit of craziness. Amazon decided that they should model the Alexa app store after the iPhone app store. So there is a certification process to get your app into the store. But think about the difference: you are not uploading a binary file to the Alexa app store, you are simply registering an URL. So Amazon has no real control over your software. You could get an app approved, and then you could swap out the app for any other app, and the Certification team at Amazon would never know. They don’t control your code. Your code is not in their store, so they have no control over what you do. And yet they modeled this process after the iPhone store, where Apple does have control over your app." But that doesn't get at how crazily broken the certification system. You have to read the quotes from the other developers to understand that.
- dperfect 11y agoAmazon has just as much reason as Apple (or any platform provider) to want to restrict the types of apps that get published. As for the effectiveness of that control, there's little practical difference between "simply registering an URL" and uploading a binary; both can load external content or be modified (in terms of what the user actually sees/experiences) almost entirely after the initial review. There will always be apps that try to abuse that possibility, and they usually get reported or re-evaluated, but it's probably a very small number of apps that will do that. The point is to simply filter out a larger number of spammy, malicious, or otherwise offensive apps at the onset rather than spending more resources constantly monitoring everything after the fact. That said, it's unfortunate when the review process is as flawed as it seems to be in this case (poor communication, inconsistent policies, etc).
- rdw 11y agoApple has tools to see whether your app is loading external content and modifying itself, though. I think you can deceive them by making it seem as though it does so in an allowed manner (e.g. no code execution) during review, but they'll definitely know that your app has the theoretical capability.
- inflagranti 11y ago
- jjaquinta 11y agoThis is my take on it here: http://ocean-of-storms.com/tsatsatzu/explaining-amazons-indifference/ http://ocean-of-storms.com/tsatsatzu/explaining-amazons-indi...
- lkrubner 11y agoThis bit from your blog post is a great summary: "There’s no public tracker for tickets. Features and suggestions go off into the void, and you’re left guessing if they are ever going to happen. Releases come infrequently and unannounced. There’s no product roadmap. No indication of what’s coming up. Nothing to create a development plan against. There’s no bidirectional collaboration with the community. The closest any developer gets to the team is during skill certification. And, even then, names are hidden and you only get anonymous mails from the “Alexa Skills Team”."
- astazangasta 11y agoThe 'app store' is the adulterated form of the Linux package repository. No shock that when you remove community management, accountability, openness, verifiability and add in monetization things get shittier.
- nailer 11y agoI think that's how app stores began - do you remember 'Click N Run' in 2002? http://i133.photobucket.com/albums/q41/mp3tunes/cnr-linspire-installer.jpg http://i133.photobucket.com/albums/q41/mp3tunes/cnr-linspire...
- SixSigma 11y agoThe NeXT Computer and the NeXTSTEP operating system was the platform used for creating the first App Store, originally demonstrated to Steve Jobs in 1993. https://en.wikipedia.org/wiki/NeXT https://en.wikipedia.org/wiki/NeXT
- nailer 11y agoDo you have any screenshots? Your comment originally said NeXT created bundles, which is very different from an online store for software.
- SixSigma 11y agoYes, the bundle thing is also true. I found the bit about app stores while I was trying to find a year. No screenshots. I just remembered that the bundle thing was something I was excited about wrt Next at the time. And booting from Cdrom, and display postscript. I wanted one so badly but by the time I had money they were very obsolete.
- nailer 11y agoThere's no reference though. It's just: > creating the first App Store which was originally demonstrated to Steve Jobs in 1993 without any footnotes. Would have loved to actually read something on the topic.
- umanwizard 11y agoIt's really remarkable how much Amazon's reputation has been self-destructing in the last year or so. Even 6 months ago when I told people I thought Amazon was an all-around shitty company (having worked there) and shouldn't be mentioned in the same breath as Google, FB, etc., people looked at me like I was insane. Somehow the stock keeps going up... I don't have any material insider info but just on a hunch I doubt that will remain true forever.
- nailer 11y agoTheir consumer electronics are getting more popular, which is making people aware of Amazon quality. Right now there's a Fire TV in front of me: - If you can get something for free because it's included in Prime, it still offers the ability to spend money on it, right beside the 'watch now for free'. If you're not paying attention, you can pay for something you already own. - Its killer feature is voice recognition. Say 'game of thrones' and it will play Game of Thrones. It also has apps. The killer feature doesn't actually work in apps though: if you say 'game of thrones' in Netflix, it kicks you out of the app and sends you back to the home screen. There was a post a little while ago about people wrecking their lives while working at Amazon. I could understand, but not support, someone doing that at say Apple or Tesla. But Amazon?
- aschampion 11y agoMy worst experience with Amazon yet is they've locked the Amazon video Android app from every Android TV device except a few select smart TV vendors and their own Fire TV devices. The result is I can't watch content I paid them for (and would continue paying for) on a device I bought through Amazon (Shield TV). They're treating lackluster Fire devices, which were supposed to be loss leaders to get you into their ecosystem, into privileged, Apple-tier garden walls. Since they lack the Apple polish and simplicity, the result is making customers like me, who are already in the garden, want to leave. Between this, inscrutable device compatibility of Kindle content, and 2-day delivery becoming unreliable, I'm cancelling Prime after a decade because the value is gone. BTW, Netflix lacks voice search in all of its apps. Normal Android TV is at least reasonable enough to just bring up the Netflix text search if you try to voice search, rather than kicking you out.
- johndandison 11y agoThe post-approval switcheroo is similar to the Office/SharePoint app stores - really all the 'app' is is a delegation of permission. If your app can change and operate within the permission you're initially granted during 'installation,' you can really do anything you'd like in the actual application itself. A rather large hole.
- yeukhon 11y agoI think the whole approval process is just a ToS legal scheme of "we did our part, so we can go after you and blame you if something happened." I don't know if will ever scan your URL for any intelligent malicious misuse but it may. If someone can explain exactly how "skill" works with Alexa... sounds like some kind of Pipe / IFTT kind of service.
- scottu 11y agoOther than the operational issues, like not have a good way to contact amazon, this reminds me of a 900-number system we ran at MCI. When you registered for a 900-number, you had to submit a script along with it. MCI would try to limit the possibility that you'd use it for some nefarious purpose, all CYA. They had a team (probably one person) that audited the numbers periodically. That didn't stop people re-using the numbers for bogus financial services and adult things.
- ceejayoz 11y agoWe went through the same thing a couple years ago with an SMS short code. Had to submit a script (despite it being dynamically driven), set a max number of messages per month (despite it being user-initiated), etc. The real kicker is we once got a massive nastygram from Verizon threatening us with shutdown - they claimed we were sending porn spam messages via the number. After quite some time auditing our entire stack we got a "our bad, one of our techs had malware on their Android phone" email.
- deleted 11y ago[deleted]
- vonklaus 11y agoMaybe they should add support for the certification process. They could price this in nano-seconds using utc time based on the region of the developer (except u.s east virginia). If you are rejected you could get max-support integration (4 pico dollars a parsec) to resubmit. Overall, amazon's coherent pricing strategy and simplicity make it a pleasure to work with
- rdtsc 11y agoOr Amazon video on Android device I have -- in order to have play prime videos it told me to disable app source checking and side-load some .apk file, which will then help me get the Amazon video apk file... or something. Yeah forget that. At first I thought it was a joke of some sort. But I guess that's how you are supposed to do it. I'll just stick to Netflix and watch videos there. Somehow that manages to work without side-loading a bunch of crap.
- Dylan16807 11y ago"some .apk file" being the Amazon store. It's annoying but it's not nonsense. The store will then notify you about updates and perform them. It's half google's fault for not allowing the amazon store app to be listen in Play.
- cwyers 11y agoWhy is that Google's fault? Amazon could use the Play Store to update the app, the same way that Netflix, Hulu, etc. does. Amazon doesn't want to because they want people to use their own Android platform. Except their Android platform is awful. They have no idea how to run a store (which is a hell of a liability for an ecommerce company, but their infrastructure and fulfillment processes are so top notch and their margins so small that everyone just routes around it when it comes to buying books or whatever else). They make regular Android OEMs look prompt in how long it takes them to incorporate updates.
- Dylan16807 11y agoIt's more that google's main benefit with their store is not having arbitrary restrictions, so the ones they do have stick out. Oh you want to make a youtube player that acts like every browser in the world, with the ability to minimize? Banned.
- jamespo 11y agoAre you really surprised that Google has policies like this?
- melted 11y agoNo one has any idea how to run an App Store. Even Apple's App Store is vastly inadequate to sell the gigantic number of apps available on it. Top N apps make out like bandits, everyone else is basically dying. It will always be power law, but IMO the tail could extend quite a bit further than it does today.
- woah 11y agoThe App Store is not there to serve developers
- melted 11y agoIt's not clear who or what it serves right now. It stands to reason (from simple arithmetic) that the more money developers make the more money Apple will make as well. They do make billions off that store. Making more money means surfacing more apps to more users, adapting to user preferences, employing prediction, etc. Apple doesn't seem to be doing much (if any) of that.
- notlisted 11y agoI have an echo. I like it, but the skills store is awful for the consumer too. Too many silly apps (like the current #1 listing a fart app referenced below), and the app features a skills list, but no categorization, no option to filter apps by stars so unknown gems can be discovered, no stats on installs, no rating distribution graphs. All very strange. Worse still, there's no standard vocabulary to interact with skills and the flexibility of the wording depends on the developer. You need to mention the app, which can have weird long names (e.g. Agog Reader) or near-duplicate names (e.g. three bitcoin apps). All-in-all, the skills feature (and the skills store) feels like an after-thought.
- lkrubner 11y ago> All-in-all, the skills feature (and the skills store) feels like an after-thought. Joseph Jaquinta expressed a similar sentiment when he wrote: "Amazon's traditional areas (everything on AWS) follows a different model. They are more like a utility. Minimal engagement. No direct support. They provide a service. You take it or leave it. While this I think is a workable approach for a wide audience in a mature area, I do not think it is a wise approach for Alexa. But, that's if they give a hoot about 3rd party developers and think they are going to add any significant value to their platform. We don't know if this is so, because they won't say."
- Apocryphon 11y agoWhat would be the solution? Is there even a way to verify that web services are not malicious?
- galactoise 11y agoThe solution is to not try to verify the content of a rest service. There is a lot of good that Amazon can do with their certification process - things like helping a developer figure out appropriate example phrases, or making sure that all of the appropriate fields are filled in (like providing a thumbnail for the appstore, etc). Their weird pedantism about an arbitrary set of rules, however, serves only to scare off those of us who are actually interested in advancing the default user experience. Out of the box experimentation is frowned upon, and the result is that they're basically handing the ecosystem to the shovelware creators.
- meesterdude 11y agoNot surprised by this at all. >However, the Amazon system is so broken that it potentially offers a fix for itself. When the Certification Team rejects your app, you don’t need to change the app. You do not need to respond to their requests. You do not need to make any of the changes that they demand. Since a new person reviews each submission, and since there is no limit on submissions, one way to get through the certification process is to simply roll the dice and spam the certification team. Submit an app 10 times, or 20 times, or 30 times. At some point you will probably get lucky, and someone will approve your app. Amazon has clearly put a lot of thought into how this is going to work. They also have no idea how to run a video streaming service; or they think they do, and just don't know how much it sucks. They also have started sucking in buying things, with things like "pantry" which require you to buy a box for $5 so you can get things like soap. I canceled my prime and have been buying things from stores. I buy my videos (ala DVD) so I can watch them when i want and not be worried about them getting pulled by disney from the "prime" section and being forced to buy it. Honestly, the only thing I have any respect for is AWS. But even that is severely lacking in polish in clarity, and it's almost a full time job trying to keep up with their additions and changes.
- erikpukinskis 11y ago> They also have started sucking in buying things, with things like "pantry" which require you to buy a box for $5 so you can get things like soap. Delivering batches of household goods of arbitrary size profitably is a hard problem that no one has figured out. Everyone is fiddling with business models trying to make it work. I think at the end of the day shipping is expensive and trying to hide that from customers without incentivizing profit-negative transactions is nigh impossible. Things will probably change when we have droids roaming the sidewalks, but until then these strange charges are going to keep popping up.
- CamperBob2 11y agoDelivering batches of household goods of arbitrary size profitably is a hard problem that no one has figured out. Wait, wait, I think I've figured it out. Raise the price. Repeat until either the transaction is sufficiently profitable, or the customers go away. That will work better for everyone than what Amazon has started to do lately, which is prevent you from buying certain items at any price until your order exceeds $25. Basically, Amazon has started to tell their customers "No." That's new, and I agree with the other posters who call it a bearish sign.
- kohanz 11y agoInteresting timing: I just got an e-mail for a (free) app that I purchased in the Amazon app store. Only problem, I have never visited the Amazon app store. The Amazon rep on the live chat explains to me that this is because someone entered their e-mail address incorrectly when making a purchase. How does that make sense? Shouldn't you need to be signed in to your Amazon account (or some other account) to make an app purchase? It actually makes you type in your confirmation e-mail address manually? Can someone with experience in their app store comment? That seems like a brutal design decision...
- cm2187 11y agoThe amazon music application is also one of the most appalling software too, it is messy, it keeps flashing, there doesn't seem to be any logic in how the screens are organised, it's impossible to know where you are or how to go back to one place. I love amazon, I buy half of my music on amazon music, but downloading the music is an unnecessary painful step. Also they need to improve the quality of the samples vs itunes. When you buy a rock song, there is usually is only one recording of that song. But if you buy classical music there will be many recordings with very different qualities. With the quality of the samples available now, it's almost impossible to tell if a recoding will be of a good quality or not. Itunes has much better quality.
- erikpukinskis 11y agoI just wish it could play back music without a gap between songs. I cringe sometimes listening to an album. I mean, audio players could do that in the 90s.
- JoshuaJB 11y agoI've had a good experience with the web app [1]. It's a surprisingly well designed piece of software compared to it's competitors (e.g. Spotify or Pandora). The library could definitely use better curation though. [1] https://www.amazon.com/gp/dmusic/cloudplayer/player https://www.amazon.com/gp/dmusic/cloudplayer/player
- cm2187 11y agoI am referring to the windows app.
- renownedmedia 11y agoThe Firefox Marketplace does the exact same thing. One simply submits a URL, it gets approved, then becomes available. Any changes made afterwards go unnoticed by moderators.
- rhelmer 11y agoFor FirefoxOS "hosted" apps this is true, but they also don't have access to any significant APIs. Only "packaged" apps do. This is also not true for add-ons (for desktop/mobile or FirefoxOS), these are packaged as zip files and can't be arbitrarily changed. However, there is the problem however of the app or add-on downloading and running code from the internet unchecked, which is probably intractable to do in an automated way (at least for the current APIs exposed, and for a dynamic language like JS), manual review is required. Mozilla requires manual review, and Apple disallows interpreters. I'm not sure what if anything Google does about this problem for Android.
- dahart 11y agoApple's review process also sometimes felt arbitrary, especially early on, and there's been a long list of complaints over the years that sound just like these. I used to work in games, and the approvals for Nintendo, Sony, and Microsoft were all super, ridiculously frustrating at times. We had experiences very much along the same lines of failing again for doing the very thing they asked of us in the previous rejection. Anyway, usually it works out eventually. No problem with making some noise about it being bad, and hopefully they hear it, but it is not uncommon, nor unique to Amazon. The point about registering a URL isn't entirely accurate, IMO; plenty of Apple App Store apps consist partially or entirely of webviews. Even for fully native apps, Apple doesn't really have "control" of the code in the sense described in the article. Registration and the approval process are primarily there to give the publishers control over what appears in their store, as well as identify the submitters in a way that attempts to keep a little bit of accountability should they do something bad. Those reasons for requiring registration and review are just as valid for a url as for a bundle of code.
- bananaboy 11y agoMy experience in games has been the opposite. As opposed to Apple and Amazon, the console certification requirements are set in stone. The documents are very clear about what you have to do.
- MaulingMonkey 11y agoThe requirements are generally much better documented for consoles. But some of the requirements can still be subjective. Edge cases still pop up as well, and the requirements can change over time (especially early in the console cycle when everything is still getting ironed out - nevermind when switching between console generations.) Even ignoring the edge cases and subjective items, they can be extremely picky about such important things as... leaderboard username terminology. Which will not necessarily be the same across all storefronts from a given company, even if they're all using the same service under the hood. And then the moment you step into the territory of needing waivers for anything - better hope your publisher has a good relationship to secure those.
- AndrewUnmuted 11y agoThis article gave me an enormous sense of deja vu, as a former Amazon employee. I managed the QA of user-submitted media for a service run by one of its subsidiaries, and the requirements for this service, too, were unreasonable and communicated very poorly. Somehow, Amazon's "working backwards" principle always fails when it comes to establishing reasonable procedures, SLAs, and solid documentation.
- dang 11y agoThat title is over the top, so we replaced it with the much more neutral first sentence of the article.
- Candelbra 11y agofuck you
- switch007 11y agoI'm looking forward to seeing "over the top" added to https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html
- colinmegill 11y agoyeah, agreed with Candelabra - that title got lots of upvotes and was what it was
- dang 11y agoThe problem is that such titles routinely get upvotes, presumably because of some reflex they trigger in the brain. If we want to have a site that gratifies intellectual curiosity—which we do—we have no choice but to counteract them. That's a fundamental principle of this place.
- colinmegill 11y agoyeah, agreed with Candelbra - that title got lots of upvotes and was what it was
- lkrubner 11y agoI am sorry to say this contributes to the feeling that the HN rule about titles is entirely arbitrary. Most of the time the rule seems stupidly rigid and inflexible. After all, why can't developers highlight the aspect of the article that would be of most interest to the readership of HN? But then, suddenly, an exception is made, based on wholly subjective criteria. How do you define "over the top"? Have you tried to get an Alexa app certified? Do you understand how bad the situation is? I'm in favor of flexibility regarding the titles, and if HN supported such flexibility, then your decision in this case would seem less arbitrary. But given the rigidness applied at other times, this seems like an odd exception. I'll point out that the current title does not come close to expressing the level of frustration that developers are currently feeling regarding the Alexa app store. Shouldn't the title give some indication about the real conflict?
- colinmegill 11y agolol title change
- jrcii 11y agoYeah that sure got editorialized, didn't it? For the newcomers, pretty much all day this post was titled "Amazon has no idea how to run an app store"
- galactoise 11y agoI'm the same "Galactoise" quoted in the original story. I'm really happy to see the conversation that cropped up here, about the lack of value around trying to certify a mutable black box, and hopefully the power of HN is enough to get some action from the Alexa team. I also wanted to pass along a blog post I threw together on the topic. http://www.derpgroup.com/blog/on-the-topic-of-certification http://www.derpgroup.com/blog/on-the-topic-of-certification It's pretty long, but the tldr is as follows: We're all worked up about this idea of having to build switches into our code to meet arbitrary certification guidelines, only to turn them off the moment we are live in prod. It's worth noting, though, that Amazon pulls these sort of antics (bait-n-switches) on us regularly and without remorse, and that there's really no way we can even know it happened. This is arguably even more destructive to the quality of our products than trying to build for a set of capricious guidelines.
- meesterdude 11y agoActual title: Amazon has absolutely no idea how to run an app store Why did the title change? It used to be the same as the original article, but now it's been needlessly editorialized to be kinder to amazon. Seriously? What the fuck. That's not HN's job, and is a disservice to the article, and the community.
- kordless 11y agoWelcome to Silicon Valley.
- timrpeterson 11y agohttps://en.m.wikipedia.org/wiki/Jumping_the_shark https://en.m.wikipedia.org/wiki/Jumping_the_shark
- meesterdude 11y agoGood comment, never knew about this as a concept but I've definitely seen it play out. (and what an amusing source of origin!) But you might be right. I've been on HN for 3+ years and i've never seen anything like this happen before. This is clearly an abuse of power and is not in any way an act on the behalf of the community.
- derefr 11y agoAn abuse of power to achieve... what, exactly? dang is here as a moderator precisely "on behalf of the community." As in, he acts to facilitate productive discussion. You might disagree with his decision here, but I can't see any way in which he has different goals than you would want him to. At worst, he has made a mistake, and the backlash from it will make him not do it again. (At best, he is being "the hero we need" by attempting to stop a flamewar argument before it starts, even if that means taking the heat that that flamewar would have spawned entirely himself.)
- meesterdude 11y agoYou're talking about goals and motive; I don't know why. I took issue with the behavior, the specific act of editing the title, in a way that contrasts the actual articles title.
- deleted 11y ago[deleted]
- tempodox 11y agoApple also has no idea how to run an app store. Does anyone?
- soyiuz 11y agoFrom a practical standpoint, why would you not try to pass the certification process first with a bare-bones, minimally viable service designed specifically to pass certification? Then iterate on top once it is in.
- deleted 11y ago[deleted]
- galactoise 11y agoFollow-up, this post getting love on HN seems to have helped us get their attention. We had a conference call with them where we were able to work through a lot of the issues we were facing: http://www.derpgroup.com/blog/the-certification-saga-problem-solving http://www.derpgroup.com/blog/the-certification-saga-problem...