6 ms·

Lots of those that build on the Twitter platform gets burned at some point. They have shown not themselves to be a reliable partner. Any company doing social me

Lots of those that build on the Twitter platform gets burned at some point. They have shown not themselves to be a reliable partner. Any company doing social media management at some point gets hurt by changing APIs and increasing fees, sometimes causing fatal damage to their business.

TekMol 7y ago
Now that scraping has been deemed legal by the highest curts, how can anybody still get burned by API fees?
- ceejayoz 7y ago
  "Legal" and "won't be interfered with" are not the same thing. That it's legal to scrape doesn't mean they can't put captchas, IP blocks, etc. in your way. Scraping doesn't really help when publishing tweets, either.
  - colejohnson66 7y ago
    I thought the point of the LinkedIn lawsuit was that they were trying to stop them from scraping?
    - ceejayoz 7y ago
      LinkedIn was trying to do so via legal measures (specifically, alleging violations of the Computer Fraud and Abuse Act in a C&D letter).
- godzillabrennus 7y ago
  Scraping, while legal now, is still not free. A scrape does not give you the delta between two points in time without a completely new scrape. You cannot economically sustainably scrape the entirety of a large site like Twitter. If I’m wrong I’d love to learn from someone smarter than me. Even google doesn’t scrape the entire web every day.
  - CaptArmchair 7y ago
    You're not wrong. Library of Congress started archiving every public tweet in 2010 but abandoned the effort in 2017 mainly because of the sheer volume. https://www.npr.org/sections/thetwo-way/2017/12/26/573609499/library-of-congress-will-no-longer-archive-every-tweet https://www.npr.org/sections/thetwo-way/2017/12/26/573609499... I think it might be feasible to scrape a subset: a single account or a few accounts at most. But even then, you're still fighting problems ranging from wonky HTML generated through JavaScript towards sudden, unannounced changes to the DOM structure. It's not a managed API after all.
riffic 7y ago
>They have shown themselves to be a reliable partner I hope you mean the opposite? Twitter has long been an unreliable partner in the developer ecosystem.
- Jaygles 7y ago
  Perhaps they mean Twitter is reliably unreliable?
- bmelton 7y ago
  Not sure if edited, but your quote is slightly off. Should be > They have shown not themselves to be a reliable partner The "not" there feels like it's in an ambiguous spot, but at least gives a faint connotation of the author's intent, which reads as you'd like it to.
  - rhizome 7y ago
    Even GP flailed it and put the 'not' in the wrong place. Should be "They have shown themselves not to be a reliable partner."
    - dorkwood 7y ago
      Also acceptable would be "they have shown themselves to be a reliable partner... not"
      - a1369209993 7y ago
        That could be interpreted as "they have not shown themselves to be a reliable partner" (ie we don't know) instead of the correct "they have shown themselves to not be a reliable partner" (ie we know they aren't), though.
    - buboard 7y ago
      yeah it should be not (themselves to be (a reliable partner))

TekMol 7y ago

Now that scraping has been deemed legal by the highest curts, how can anybody still get burned by API fees?

ceejayoz 7y ago
"Legal" and "won't be interfered with" are not the same thing. That it's legal to scrape doesn't mean they can't put captchas, IP blocks, etc. in your way. Scraping doesn't really help when publishing tweets, either.
- colejohnson66 7y ago
  I thought the point of the LinkedIn lawsuit was that they were trying to stop them from scraping?
  - ceejayoz 7y ago
    LinkedIn was trying to do so via legal measures (specifically, alleging violations of the Computer Fraud and Abuse Act in a C&D letter).
godzillabrennus 7y ago
Scraping, while legal now, is still not free. A scrape does not give you the delta between two points in time without a completely new scrape. You cannot economically sustainably scrape the entirety of a large site like Twitter. If I’m wrong I’d love to learn from someone smarter than me. Even google doesn’t scrape the entire web every day.
- CaptArmchair 7y ago
  You're not wrong. Library of Congress started archiving every public tweet in 2010 but abandoned the effort in 2017 mainly because of the sheer volume. https://www.npr.org/sections/thetwo-way/2017/12/26/573609499/library-of-congress-will-no-longer-archive-every-tweet https://www.npr.org/sections/thetwo-way/2017/12/26/573609499... I think it might be feasible to scrape a subset: a single account or a few accounts at most. But even then, you're still fighting problems ranging from wonky HTML generated through JavaScript towards sudden, unannounced changes to the DOM structure. It's not a managed API after all.

riffic 7y ago

>They have shown themselves to be a reliable partner I hope you mean the opposite? Twitter has long been an unreliable partner in the developer ecosystem.

Jaygles 7y ago
Perhaps they mean Twitter is reliably unreliable?
bmelton 7y ago
Not sure if edited, but your quote is slightly off. Should be > They have shown not themselves to be a reliable partner The "not" there feels like it's in an ambiguous spot, but at least gives a faint connotation of the author's intent, which reads as you'd like it to.
- rhizome 7y ago
  Even GP flailed it and put the 'not' in the wrong place. Should be "They have shown themselves not to be a reliable partner."
  - dorkwood 7y ago
    Also acceptable would be "they have shown themselves to be a reliable partner... not"
    - a1369209993 7y ago
      That could be interpreted as "they have not shown themselves to be a reliable partner" (ie we don't know) instead of the correct "they have shown themselves to not be a reliable partner" (ie we know they aren't), though.
  - buboard 7y ago
    yeah it should be not (themselves to be (a reliable partner))