SFTP to S3 with S3FS and EC2

I’ve been looking for an SFTP to S3 solution for years now. In an infinite number of universes, I’ve had an infinite number of experiences where team members and/or outsource teams need to upload files to clients’ S3 buckets. If you’re just seeing the AWS console for the first time, it’s overwhelming. Even limiting access (in my experience this is lukewarm at best: functional but not visually reflected; users can see what’s set up even if they can’t access them).

Perhaps you’ve seen AWS now offers an SFTP to S3 endpoint service. What you might not know (yet): you’ll spend $0.30 an hour plus $0.04/GB (up and down). While there are plenty of customers that simply don’t care about the cost, there are enough that do, so I went off in search of what I could do with a simple EC2 instance instead.

First, I used a couple of different tutorials: one from WinSCP, and a more in-depth one from Cloud Academy. For my time and effort, the Cloud Academy one was a little more detailed and, IMO, better for a first-time run. From a security standpoint, I prefer Cloud Academy’s use of IAM roles instead of local aws-cli config files.

The core to making this transition work is utilizing the S3FS-FUSE filesystem, which lets you mount an S3 bucket as a local drive. I won’t go into details here, feel free to read up on it if you so desire.

Improving Security

I suggest making a handful of changes to the tutorials above to further secure your FTP server:

  1. don’t install a local FTP server; use FTP over SSH (SFTP); avoid unnecessary services and running processes whenever possible
  2. if you want to be even more secure, use EC2 Security Groups and firewall the server to only necessary users (you could also change the port SSH runs on your server if you feel it necessary)
  3. use key exchanges over SSH instead of passwords (and disable SSH passwords altogether)
  4. when setting up the IAM Policy, limit the access only to your local AWS VPC