5 ms·

Fuck it, I am going to build an open source project with the sole goal of automating access to all of these sites with selenium or something. Get ready for war

Fuck it, I am going to build an open source project with the sole goal of automating access to all of these sites with selenium or something. Get ready for war(in the nice, data science-y way)

antsar 11y ago
That's what these "finance sites" already do. That's why you have to enter your bank username and password to use them - they just scrape the website. Incidentally, I have never used one, because that requirement is completely insane.
- phdp 11y ago
  I think the point was so that the banks couldn't block access via ip address. Each person would be able to aggregate their bank info on their home computer or they could secure their own server to do it for them constantly.
  - antsar 11y ago
    I would use that in a heartbeat. Just wanted to point out that it the incumbents' approach as well, rather than some actual API which banks decided to discontinue.
  - theseatoms 11y ago
    Not my area of expertise, but isn't it fairly trivial to circumvent an IP address block, (given a willingness to devote some resources to it)?
    - antsar 11y ago
      Yes. But that sounds like a great way to get sued.
- ajkjk 11y ago
  Why is that insane? It's inelegant, but it's not functionally different than having an API and a system for registering API keys. Either you trust the aggregator or you don't; if you don't you shouldn't give them access to your data either way. If they do something they shouldn't with your data, they'll be ruined if it becomes public. (Of course there's a huge gray area of things they could do that you wouldn't like, but that would't ruin them. But there's a clear difference between questionable things, like selling your data to advertisers, and completely not-ok things, like stealing your money).
  - snowwrestler 11y ago
    > Why is that insane? It's inelegant, but it's not functionally different than having an API and a system for registering API keys. Because there is such thing as a read-only API. But logging into the banking site as a customer permits the aggregator to take actions as the customer, like wire transfers--which are not reversible, and not typically covered by fraud protection. Sure, I might trust Mint not to do that, but what if Mint gets hacked? By definition, Mint can access my bank creds in plain text, since it must paste them into my bank's website. That is worse security than even my own bank's website, which undoubtedly stores only the hash of my password. My bank is not going to indemnify me for fraud caused by Mint's security problems. Bank agreements usually say "don't share your account creds," so if you do, and then lose your money, the bank is not likely to make you whole.
  - wdewind 11y ago
    > but it's not functionally different than having an API and a system for registering API keys No, it's completely different. The entire point of keys is avoid having to store credentials, it has nothing to do with permissioning. In the case of Mint etc., they have to store credentials because there is no token/key system being provided from the banks (ie: oAuth). In the case of something like oAuth creds are never sent to the 3rd party. And the ironic part of this is if they banks simply provided something like oAuth then they would have MUCH more control over who they worked with and have an easier time managing 3rd party data.
warkon 11y ago
If you're serious about this, let me know. I've been thinking about doing the same for some time now. How would you solve for accounts with 2-factor auth enabled?
daxelrod 11y ago
Wesabe was a competitor to Mint that did tons of scraping, including having a browser extension that did the scraping client side. They've open sourced much of their code. Take a look at https://github.com/wesabe/ssu https://github.com/wesabe/ssu and the other projects in that organization.

Fuck it, I am going to build an open source project with the sole goal of automating access to all of these sites with selenium or something. Get ready for war(in the nice, data science-y way)

antsar 11y ago

That's what these "finance sites" already do. That's why you have to enter your bank username and password to use them - they just scrape the website. Incidentally, I have never used one, because that requirement is completely insane.

phdp 11y ago
I think the point was so that the banks couldn't block access via ip address. Each person would be able to aggregate their bank info on their home computer or they could secure their own server to do it for them constantly.
- antsar 11y ago
  I would use that in a heartbeat. Just wanted to point out that it the incumbents' approach as well, rather than some actual API which banks decided to discontinue.
- theseatoms 11y ago
  Not my area of expertise, but isn't it fairly trivial to circumvent an IP address block, (given a willingness to devote some resources to it)?
  - antsar 11y ago
    Yes. But that sounds like a great way to get sued.
ajkjk 11y ago
Why is that insane? It's inelegant, but it's not functionally different than having an API and a system for registering API keys. Either you trust the aggregator or you don't; if you don't you shouldn't give them access to your data either way. If they do something they shouldn't with your data, they'll be ruined if it becomes public. (Of course there's a huge gray area of things they could do that you wouldn't like, but that would't ruin them. But there's a clear difference between questionable things, like selling your data to advertisers, and completely not-ok things, like stealing your money).
- snowwrestler 11y ago
  > Why is that insane? It's inelegant, but it's not functionally different than having an API and a system for registering API keys. Because there is such thing as a read-only API. But logging into the banking site as a customer permits the aggregator to take actions as the customer, like wire transfers--which are not reversible, and not typically covered by fraud protection. Sure, I might trust Mint not to do that, but what if Mint gets hacked? By definition, Mint can access my bank creds in plain text, since it must paste them into my bank's website. That is worse security than even my own bank's website, which undoubtedly stores only the hash of my password. My bank is not going to indemnify me for fraud caused by Mint's security problems. Bank agreements usually say "don't share your account creds," so if you do, and then lose your money, the bank is not likely to make you whole.
- wdewind 11y ago
  > but it's not functionally different than having an API and a system for registering API keys No, it's completely different. The entire point of keys is avoid having to store credentials, it has nothing to do with permissioning. In the case of Mint etc., they have to store credentials because there is no token/key system being provided from the banks (ie: oAuth). In the case of something like oAuth creds are never sent to the 3rd party. And the ironic part of this is if they banks simply provided something like oAuth then they would have MUCH more control over who they worked with and have an easier time managing 3rd party data.

warkon 11y ago

If you're serious about this, let me know. I've been thinking about doing the same for some time now. How would you solve for accounts with 2-factor auth enabled?

daxelrod 11y ago

Wesabe was a competitor to Mint that did tons of scraping, including having a browser extension that did the scraping client side. They've open sourced much of their code. Take a look at https://github.com/wesabe/ssu https://github.com/wesabe/ssu and the other projects in that organization.