6 ms·

I noticed a few comments specifically referencing FTP (and who can blame them since the HN title as of this moment specifically references it). In the first po

by mdip 9y ago

I noticed a few comments specifically referencing FTP (and who can blame them since the HN title as of this moment specifically references it). In the first post of the series, the author refers to the server as a "Secure FTP" server, which can be confusing to read[0]. In later parts (and a little googling of my own), it's clear that the server is actually an SFTP server, not a plain-old FTP server. It's still plenty archaic, but takes the headline's shock value down a small peg[1]. [0] It adds a mental pause -- a Secure ... FTP server. It hints that, possibly, it's a reference to a different aspect of the server's security (a non-technical person might refer to a server as being a "secure" server simply because it's protected by an ID and password, for instance). [1] Based on my personal interaction with banks and software, as well as several friends who had previously been members of a few banks' IT departments, my first -- very sarcastic thought -- was "of course it works that way!"

bct 9y ago
Even calling it archaic is too harsh. Granted, the batch-centric nature is not ideal (and it's hard to imagine a system with this kind of latency built in being designed today), but if you're designing a system based on batch processing then shipping files over SFTP is a pretty reasonable way to do it.
- mdip 9y ago
  That's a fair point. It's very easy to get stuck inside the developer bubble, where 'archaic' means 'something that was state-of-the-art four years ago (or in the web/design world, 3.5 minutes ago)'. SFTP is a nice technology which I make quite frequent use of in my job and at home. The idea that I plug in my Yubikey, provide a passphrase to unlock the key, authenticate to my server which that's got a verifiable certificate installed, and the private key never leaves my Yubikey is about as state-of-the-art as I could ever ask for security-wise. Granted, precisely how the SFTP site is secured isn't specified and there's plenty of ways to do that wrong, but as a technology, it's always impressed me how seamlessly it works once it's setup.
- lojack 9y ago
  SFTP tends to be the go-to method of integrating software between organizations, especially ones that aren't used to building actual APIs. It's standardized, everyone knows how to use it, and it essentially boils down to: generate a file and upload it to a server. There's all sorts of problems with it, and it tends to be a pretty short sighted decision. The biggest problem I've found is that there's no real feedback loop (for this class of integration, not sftp specific). Company A uploads a file -- often in the middle of the night -- and Company B processes that file some time later. Could be a minute, could be a day. Company B fails to process the file, possibly because of a change on their end, possibly because of a change on Company A's end. There's usually a bunch of back and forth, but ultimately the earliest you can determine if a fix works is a full 24 hours after the file was initially dropped. Both companies could create integration environments with shorter feedback loops to help, but ultimately its a problem of not getting an immediate response that the integration is working. 95% of my job involves writing software to integrate systems from different companies, and the integrations that involve SFTP are almost always the biggest pains. I've thought about writing an SFTP server that operated in real time, and either rejected bad files or had some other way of responding to the inputs. Never really found the time to do it, and ultimately it'd be patching a process that should have just been a well designed API from the start.
donatj 9y ago
Is it sftp or ftps? They're very different things and I am fairly certain it's the latter.
- deleted 9y ago
  [deleted]
- kyrra 9y ago
  Most payment vendors are on SFTP at this point. Very few still use only FTPS. But then they do silly things with SFTP, like requiring both a password and the ssh key.
  - yellowapple 9y ago
    "like requiring both a password and the ssh key." That doesn't sound that unreasonable. Both satisfy the "something you know" and "something you have" portions of multi-factor authentication (respectively). Considering that this is money on the line, the idea of banks actually taking security seriously is actually refreshing. Too bad they don't seem to apply those standards to their customers.
    - mjevans 9y ago
      If you can duplicate it, then it's something you know, not something you have. Everything you know in sum-total is one meta-passphrase, even if it's a combination of key-files and other data. The 'something you have' would have to be a specific hardware token (E.G. that VPN tunnel device someone else mentioned that is a black box you aren't allowed to open). Tamper resistance and physical security locks to ensure that connections /must/ route through a given location would be enough.
      - maha420 9y ago
        It's a key pair, for SSH keys, which is what SFTP uses to tunnel the FTP protocol. You cannot derive the private key from the public key. So no, you cannot duplicate someone's private key unless you have access to their machine. In which case you could just install a keylogger. Hardware tokens are vulnerable too, by your own admission. A 6 digit pin created by a token is still "something you know" even if it's only good for a couple of minutes.
tinbad 9y ago
I was just going to say, working for one of the largest destributors in the world, this is still how many large corps buy stuff: by downloading a catalog through FTP and sending the order with XML. I know, shocking but not everyone is Amazon.
- fweespeech 9y ago
  You may want to look at MWS then. It's the same batch processed feeds and XML over https instead of sftp.
- the_angry_angel 9y ago
  > I know, shocking but not everyone is Amazon. Amazon Vendor Central only has support for EDI (at least in the UK) - as in EDIFACT files sent over FTP. If you want to do business with them you either need to talk that or be comfortable farming out to a 3rd party (I'm 1 day into a project where I'm finding all this out, and the hell of EDI file formats).
djsumdog 9y ago
My first job in 2005 was to take over an ACH processing system that was written for our company. It was all in Perl. :-P ACH format is super weird .. tons of lines with 99999 on them as separators. And yes, we had to go through a bank to do any ACH transactions. They shipped us a router that setup an IPSec tunnel and it had a backup ISDN connection on it as well. I wish ACH was more accessible. In many other countries, you give people your Name, Bank and Account number (Australia: BSB + Acct, NZ: it's one big number) and people can just send you money. Appears the next day, even on holidays. In the US we still have to photo checks with a phone app? WTF? I wrote a thing about our banking system here: http://penguindreams.org/blog/the-american-banking-system-is-still-in-the-1990s/ http://penguindreams.org/blog/the-american-banking-system-is...
- geomark 9y ago
  As an American expat living in a developing country it seems very weird to me how backwards the US ACH system is. Where I live now it's the same as the countries you mentioned, anyone can send money to your bank account using just the account number. And it appears in minutes. The banks even offer you an SMS alert when founds arrive and the sender can elect to send an SMS to alert the receiver to check their account for the received funds. Everyone uses it, including many (most?) businesses.
  - geon 9y ago
    Swish in Sweden?
    - kpil 9y ago
      Swish allows to send using an arbitrary number (phone or arbitrarily assigned) rather than an actual account number. It's also credits the recipient immediately. That's possible since all the participating banks have deposited a number of billions in a mutual fund designed to cover the losses in the case one of the banks runs out of money, and also probably by having an internal account to debit during the day. They are tranfering money between the banks at one (or maybe a few) standard clearing runs involving the Riksbank, that works more or less like ACH. I think all of them are scheduled in the morning. All Swedish banks are working onreplacing the payment infrastructure (the clearing. ) It's supposed to be finished in 2024 or something like that.
TomK32 9y ago
I've written an SFTP server (nodejs) that passes the files to the proper API this summer. No magic there and the user who doesn't have the resources to implement an API client is happy.
ivanhoe 9y ago
Secure FTP is actually a superset of SFTP, because there is also a FTPS protocol that is also a kind of "secure FTP".
- mdip 9y ago
  Correct, you are. The later parts in the article, though, specifically call out SFTP by name. Based on the other comments and some searches I did before hitting that second part, I'm fairly certain they're actually using SFTP rather than FTPS, but having never actually touched the ACH system, I can't speak first-hand on that.
lilbobbytables 9y ago
For confirmation, SFTP is mentioned directly (at least once) in the second article of the series.
h0undawg 9y ago
not necessarily SFTP, always FTP in some form in the US. I have seen FTPS, I have also seen banks just use FTP by default without encryption (shocking? not really).
okreallywtf 9y ago
I came to ask the same question, I followed the comments link on the tfa and notice it linked back to HN 1200 days ago and that was one of the top questions.
zeveb 9y ago
I don't think that there's anything archaic about using SFTP to move files around!

by mdip 9y ago

bct 9y ago

Even calling it archaic is too harsh. Granted, the batch-centric nature is not ideal (and it's hard to imagine a system with this kind of latency built in being designed today), but if you're designing a system based on batch processing then shipping files over SFTP is a pretty reasonable way to do it.

mdip 9y ago
That's a fair point. It's very easy to get stuck inside the developer bubble, where 'archaic' means 'something that was state-of-the-art four years ago (or in the web/design world, 3.5 minutes ago)'. SFTP is a nice technology which I make quite frequent use of in my job and at home. The idea that I plug in my Yubikey, provide a passphrase to unlock the key, authenticate to my server which that's got a verifiable certificate installed, and the private key never leaves my Yubikey is about as state-of-the-art as I could ever ask for security-wise. Granted, precisely how the SFTP site is secured isn't specified and there's plenty of ways to do that wrong, but as a technology, it's always impressed me how seamlessly it works once it's setup.
lojack 9y ago
SFTP tends to be the go-to method of integrating software between organizations, especially ones that aren't used to building actual APIs. It's standardized, everyone knows how to use it, and it essentially boils down to: generate a file and upload it to a server. There's all sorts of problems with it, and it tends to be a pretty short sighted decision. The biggest problem I've found is that there's no real feedback loop (for this class of integration, not sftp specific). Company A uploads a file -- often in the middle of the night -- and Company B processes that file some time later. Could be a minute, could be a day. Company B fails to process the file, possibly because of a change on their end, possibly because of a change on Company A's end. There's usually a bunch of back and forth, but ultimately the earliest you can determine if a fix works is a full 24 hours after the file was initially dropped. Both companies could create integration environments with shorter feedback loops to help, but ultimately its a problem of not getting an immediate response that the integration is working. 95% of my job involves writing software to integrate systems from different companies, and the integrations that involve SFTP are almost always the biggest pains. I've thought about writing an SFTP server that operated in real time, and either rejected bad files or had some other way of responding to the inputs. Never really found the time to do it, and ultimately it'd be patching a process that should have just been a well designed API from the start.

donatj 9y ago

Is it sftp or ftps? They're very different things and I am fairly certain it's the latter.

deleted 9y ago
[deleted]
kyrra 9y ago
Most payment vendors are on SFTP at this point. Very few still use only FTPS. But then they do silly things with SFTP, like requiring both a password and the ssh key.
- yellowapple 9y ago
  "like requiring both a password and the ssh key." That doesn't sound that unreasonable. Both satisfy the "something you know" and "something you have" portions of multi-factor authentication (respectively). Considering that this is money on the line, the idea of banks actually taking security seriously is actually refreshing. Too bad they don't seem to apply those standards to their customers.
  - mjevans 9y ago
    If you can duplicate it, then it's something you know, not something you have. Everything you know in sum-total is one meta-passphrase, even if it's a combination of key-files and other data. The 'something you have' would have to be a specific hardware token (E.G. that VPN tunnel device someone else mentioned that is a black box you aren't allowed to open). Tamper resistance and physical security locks to ensure that connections /must/ route through a given location would be enough.
    - maha420 9y ago
      It's a key pair, for SSH keys, which is what SFTP uses to tunnel the FTP protocol. You cannot derive the private key from the public key. So no, you cannot duplicate someone's private key unless you have access to their machine. In which case you could just install a keylogger. Hardware tokens are vulnerable too, by your own admission. A 6 digit pin created by a token is still "something you know" even if it's only good for a couple of minutes.

tinbad 9y ago

I was just going to say, working for one of the largest destributors in the world, this is still how many large corps buy stuff: by downloading a catalog through FTP and sending the order with XML. I know, shocking but not everyone is Amazon.

fweespeech 9y ago
You may want to look at MWS then. It's the same batch processed feeds and XML over https instead of sftp.
the_angry_angel 9y ago
> I know, shocking but not everyone is Amazon. Amazon Vendor Central only has support for EDI (at least in the UK) - as in EDIFACT files sent over FTP. If you want to do business with them you either need to talk that or be comfortable farming out to a 3rd party (I'm 1 day into a project where I'm finding all this out, and the hell of EDI file formats).

djsumdog 9y ago

My first job in 2005 was to take over an ACH processing system that was written for our company. It was all in Perl. :-P ACH format is super weird .. tons of lines with 99999 on them as separators. And yes, we had to go through a bank to do any ACH transactions. They shipped us a router that setup an IPSec tunnel and it had a backup ISDN connection on it as well. I wish ACH was more accessible. In many other countries, you give people your Name, Bank and Account number (Australia: BSB + Acct, NZ: it's one big number) and people can just send you money. Appears the next day, even on holidays. In the US we still have to photo checks with a phone app? WTF? I wrote a thing about our banking system here: http://penguindreams.org/blog/the-american-banking-system-is-still-in-the-1990s/ http://penguindreams.org/blog/the-american-banking-system-is...

geomark 9y ago
As an American expat living in a developing country it seems very weird to me how backwards the US ACH system is. Where I live now it's the same as the countries you mentioned, anyone can send money to your bank account using just the account number. And it appears in minutes. The banks even offer you an SMS alert when founds arrive and the sender can elect to send an SMS to alert the receiver to check their account for the received funds. Everyone uses it, including many (most?) businesses.
- geon 9y ago
  Swish in Sweden?
  - kpil 9y ago
    Swish allows to send using an arbitrary number (phone or arbitrarily assigned) rather than an actual account number. It's also credits the recipient immediately. That's possible since all the participating banks have deposited a number of billions in a mutual fund designed to cover the losses in the case one of the banks runs out of money, and also probably by having an internal account to debit during the day. They are tranfering money between the banks at one (or maybe a few) standard clearing runs involving the Riksbank, that works more or less like ACH. I think all of them are scheduled in the morning. All Swedish banks are working onreplacing the payment infrastructure (the clearing. ) It's supposed to be finished in 2024 or something like that.

TomK32 9y ago

I've written an SFTP server (nodejs) that passes the files to the proper API this summer. No magic there and the user who doesn't have the resources to implement an API client is happy.

ivanhoe 9y ago

Secure FTP is actually a superset of SFTP, because there is also a FTPS protocol that is also a kind of "secure FTP".

mdip 9y ago
Correct, you are. The later parts in the article, though, specifically call out SFTP by name. Based on the other comments and some searches I did before hitting that second part, I'm fairly certain they're actually using SFTP rather than FTPS, but having never actually touched the ACH system, I can't speak first-hand on that.

lilbobbytables 9y ago

For confirmation, SFTP is mentioned directly (at least once) in the second article of the series.

h0undawg 9y ago

not necessarily SFTP, always FTP in some form in the US. I have seen FTPS, I have also seen banks just use FTP by default without encryption (shocking? not really).

okreallywtf 9y ago

I came to ask the same question, I followed the comments link on the tfa and notice it linked back to HN 1200 days ago and that was one of the top questions.

zeveb 9y ago

I don't think that there's anything archaic about using SFTP to move files around!