5 ms·

i think the point of the article, or at least our interest in it, is that this is a terrible business arrangement. if you can't look at past tolerance as eviden

by nqzero 11y ago

i think the point of the article, or at least our interest in it, is that this is a terrible business arrangement. if you can't look at past tolerance as evidence of acceptable use, then you effectively can't ever afford to invest in building a business unless you're lawyered-up and that's bad for tech, and bad for america

cookiecaper 11y ago
Yes. The current legal situation with access to online resources is an absolute, unmitigated disaster. My advice to anyone building anything significant off an API or scraped access: do it anonymously. Never reveal your real identity. Never use your real IP. Don't process credit cards. Don't register an identifiable LLC. Run it out of China or Russia where American companies will have a hell of a time trying to get to you. If you depend on one entity's API, that entity is not going acquire you. At best, your product will be ripped off and they'll make you irrelevant. This happens to popular mobile applications all the time. At worst, you'll get sued civilly, get your wages and bank accounts garnished, lose all of your possessions, and get criminally prosecuted under the CFAA, end up doing some time, and eventually get released with the stipulation that you never touch a computer or access the internet again for the rest of your life. Unless you can ramp up to doing tens of millions of revenue per year (or have investors willing to pony that up) before the company you depend on notices/decides they don't like you anymore and sends out their law firm, you're dead meat, no matter what the details of your case are and no matter how wrong they are. These are not problems you want. It's easier to put your service in the onion and run it from there, access all external data via proxy, only accept Bitcoin for payment, and never tell anyone the link to your real-world identity. Granted it takes good opsec to continue this for a long time, which is really hard to pull off, but it may be doable depending on your level of commitment.
- chinathrow 11y ago
  My advice is: Obey robots.txt and provide a real user agent. End of story. Seriously.
- pjc50 11y ago
  This is a little paranoid if you're at least trying to stay within the TOS and not overtly get in a fight with the big entity whose API you're using. (And having the LLC gives you great protection against lawsuits!)
  - cookiecaper 11y ago
    LLCs do not protect against tort liability. They'll pierce the veil. See Facebook v. Power Ventures, Inc., where the founder was found personally liable for 3 million dollars in damages despite the fact that he was a) accessing Facebook on a specific user's behalf at their request and therefore was essentially a browsing device and b) was not violating copyright by any reasonable standard (Ticketmaster v. RMG is not reasonable) since the content he was downloading was owned by the user requesting the download.
- basseq 11y ago
  Scraped content is one thing, but most APIs require a unique key. Forget "trying to get to you": they already know everything they need to know to cut off your API access.
  - cookiecaper 11y ago
    Yeah, but you can usually acquire an network of API keys without making the connection obvious (depends on their API access policies of course) and rotate them as appropriate. Also, many APIs offer the same data through a public interface that can be accessed by scraping, so you can scrape and avoid identifying yourself.
  - kuschku 11y ago
    Break their mobile apps. Use their own API key against them. Also break their websites. For example, for Google, look at Google Keep – that one leaks API keys directly in the list of accessed URLs, the key has been the same for years, and provides access to Maps and Keep. Same with YouTube (the app packages an API key for the v3 data API) or the WolframAlpha app. Many more apps, from simple "what’s for lunch at my uni’s cafeteria" to Transit apps all leak API keys. Preferably you use the key of an app from the same company which maintains the API, so you can guarantee to always find a recent one. I spent a few weeks last summer extracting API keys for next to all services out of apps, and breaking some DRM solutions, just to get experience with reversing software (which was something I had a course about at uni at the same time, and the experience helped me with homework).
    - basseq 11y ago
      A rotating schema of pirated API keys seems even less sustainable than just risking use of a proprietary API. Not something on which I'd want to build a business either. At some point, the effort of reverse engineering exceeds that of actually building the damn thing for yourself.
      - kuschku 11y ago
        The reverse engineering can be automated (as the official apps have to use the key at some point), and as the official app won’t get cut off from support, you can just continue using the latest version of it.
blantonl 11y ago
Why is any other business obligated to help you invest in building your own business? Obviously, Google and others who have innovated in the space can set the terms of service as onerous or ambiguous as they want to. You can choose whether or not you want to invest in building something around someone else's products. Don't like the TOS? Develop it yourself or use another product.
- habitue 11y ago
  The grandparent was saying why Google should be less ambiguous: because more people will build on top of their apis if the legal situation is predictable. The comment was written from the point of view that Google is setting up bad incentives if they want developers to use their apis, not that developers have a moral right to Google being reasonable.
- cookiecaper 11y ago
  There is a bunch of laws that govern what someone who has innovated, developed, and authored can and can't dictate; they're called "intellectual property laws". Even if they've collected it, Google does not own information about the road system; facts themselves, like "Road A exists at lat X and long Z", cannot be copyrighted. See Feist v. Rural Telephone for more on this. The only reason application of Feist is generally limited in cases of online access is because we have a theory that accessing a publicly available database containing phone numbers is akin to accessing someone else's private property, and therefore they can dictate whatever they want under theories like trespass to chattels, whereas consulting a printed telephone book in your home is decidely not accessing the phone company's property. This is a pernicious, subtle issue now that we're becoming so dependent on online access provided by someone else's servers instead of accessing printed documents that we possess in our homes. Possession is 9/10 of the law, as they say. We need to rethink how "possession" applies to digital resources. Likewise, there are laws around how much access a private property owner must grant to members of the public, especially if that property owner is running a business that is generally accessible to the public. Private property owners are allowed to declare some people trespassers, but each jurisdiction has different laws surrounding what the public can do on private property and when a trespass is allowable. For example, California's state constitution guarantees the right to hold reasonable protests in private shopping centers that are generally accessible to the public, whether the property owner likes it or not. No one is saying there is a requirement for another business "to help invest" in any other; there may be requirements not to interfere with someone else's business by attempting to block what is otherwise public and freely available access to non-copyrightable information. IANAL. Routebuilder is fortunate that OpenStreetMap has collected much of the same data and that they can just code against a different API and continue to operate.

by nqzero 11y ago

cookiecaper 11y ago

Yes. The current legal situation with access to online resources is an absolute, unmitigated disaster. My advice to anyone building anything significant off an API or scraped access: do it anonymously. Never reveal your real identity. Never use your real IP. Don't process credit cards. Don't register an identifiable LLC. Run it out of China or Russia where American companies will have a hell of a time trying to get to you. If you depend on one entity's API, that entity is not going acquire you. At best, your product will be ripped off and they'll make you irrelevant. This happens to popular mobile applications all the time. At worst, you'll get sued civilly, get your wages and bank accounts garnished, lose all of your possessions, and get criminally prosecuted under the CFAA, end up doing some time, and eventually get released with the stipulation that you never touch a computer or access the internet again for the rest of your life. Unless you can ramp up to doing tens of millions of revenue per year (or have investors willing to pony that up) before the company you depend on notices/decides they don't like you anymore and sends out their law firm, you're dead meat, no matter what the details of your case are and no matter how wrong they are. These are not problems you want. It's easier to put your service in the onion and run it from there, access all external data via proxy, only accept Bitcoin for payment, and never tell anyone the link to your real-world identity. Granted it takes good opsec to continue this for a long time, which is really hard to pull off, but it may be doable depending on your level of commitment.

chinathrow 11y ago
My advice is: Obey robots.txt and provide a real user agent. End of story. Seriously.
pjc50 11y ago
This is a little paranoid if you're at least trying to stay within the TOS and not overtly get in a fight with the big entity whose API you're using. (And having the LLC gives you great protection against lawsuits!)
- cookiecaper 11y ago
  LLCs do not protect against tort liability. They'll pierce the veil. See Facebook v. Power Ventures, Inc., where the founder was found personally liable for 3 million dollars in damages despite the fact that he was a) accessing Facebook on a specific user's behalf at their request and therefore was essentially a browsing device and b) was not violating copyright by any reasonable standard (Ticketmaster v. RMG is not reasonable) since the content he was downloading was owned by the user requesting the download.
basseq 11y ago
Scraped content is one thing, but most APIs require a unique key. Forget "trying to get to you": they already know everything they need to know to cut off your API access.
- cookiecaper 11y ago
  Yeah, but you can usually acquire an network of API keys without making the connection obvious (depends on their API access policies of course) and rotate them as appropriate. Also, many APIs offer the same data through a public interface that can be accessed by scraping, so you can scrape and avoid identifying yourself.
- kuschku 11y ago
  Break their mobile apps. Use their own API key against them. Also break their websites. For example, for Google, look at Google Keep – that one leaks API keys directly in the list of accessed URLs, the key has been the same for years, and provides access to Maps and Keep. Same with YouTube (the app packages an API key for the v3 data API) or the WolframAlpha app. Many more apps, from simple "what’s for lunch at my uni’s cafeteria" to Transit apps all leak API keys. Preferably you use the key of an app from the same company which maintains the API, so you can guarantee to always find a recent one. I spent a few weeks last summer extracting API keys for next to all services out of apps, and breaking some DRM solutions, just to get experience with reversing software (which was something I had a course about at uni at the same time, and the experience helped me with homework).
  - basseq 11y ago
    A rotating schema of pirated API keys seems even less sustainable than just risking use of a proprietary API. Not something on which I'd want to build a business either. At some point, the effort of reverse engineering exceeds that of actually building the damn thing for yourself.
    - kuschku 11y ago
      The reverse engineering can be automated (as the official apps have to use the key at some point), and as the official app won’t get cut off from support, you can just continue using the latest version of it.

blantonl 11y ago

Why is any other business obligated to help you invest in building your own business? Obviously, Google and others who have innovated in the space can set the terms of service as onerous or ambiguous as they want to. You can choose whether or not you want to invest in building something around someone else's products. Don't like the TOS? Develop it yourself or use another product.

habitue 11y ago
The grandparent was saying why Google should be less ambiguous: because more people will build on top of their apis if the legal situation is predictable. The comment was written from the point of view that Google is setting up bad incentives if they want developers to use their apis, not that developers have a moral right to Google being reasonable.
cookiecaper 11y ago
There is a bunch of laws that govern what someone who has innovated, developed, and authored can and can't dictate; they're called "intellectual property laws". Even if they've collected it, Google does not own information about the road system; facts themselves, like "Road A exists at lat X and long Z", cannot be copyrighted. See Feist v. Rural Telephone for more on this. The only reason application of Feist is generally limited in cases of online access is because we have a theory that accessing a publicly available database containing phone numbers is akin to accessing someone else's private property, and therefore they can dictate whatever they want under theories like trespass to chattels, whereas consulting a printed telephone book in your home is decidely not accessing the phone company's property. This is a pernicious, subtle issue now that we're becoming so dependent on online access provided by someone else's servers instead of accessing printed documents that we possess in our homes. Possession is 9/10 of the law, as they say. We need to rethink how "possession" applies to digital resources. Likewise, there are laws around how much access a private property owner must grant to members of the public, especially if that property owner is running a business that is generally accessible to the public. Private property owners are allowed to declare some people trespassers, but each jurisdiction has different laws surrounding what the public can do on private property and when a trespass is allowable. For example, California's state constitution guarantees the right to hold reasonable protests in private shopping centers that are generally accessible to the public, whether the property owner likes it or not. No one is saying there is a requirement for another business "to help invest" in any other; there may be requirements not to interfere with someone else's business by attempting to block what is otherwise public and freely available access to non-copyrightable information. IANAL. Routebuilder is fortunate that OpenStreetMap has collected much of the same data and that they can just code against a different API and continue to operate.