sftp_hub.yaml
---

SFTP Hub CloudFormation Deployment

This CloudFormation template will build an OpsWorks stack to support a central SFTP server with multiple EFS systems mounted.

AWSTemplateFormatVersion: '2010-09-09' Description: SFTP Hub OpsWorks Stack

Metadata

Metadata is mostly for organizing and presenting Parameters in a better way when using CloudFormation in the AWS Web UI.

Metadata: AWS::CloudFormation::Interface: ParameterGroups: - Label: default: Application Information Parameters: - EnvAppName - AppSlug - Label: default: Instance Settings Parameters: - KeyName - CustomCookbooksSource - CustomCookbooksBucket - Label: default: Application Settings Parameters: - AppDockerImage - DockerRepoAccessKey - DockerRepoSecretKey - AppEFSMountInfo - ELMEFSStackName - Label: default: Tags Parameters: - TagOwner - TagNetid - TagProjectName ParameterLabels: EnvAppName: default: 'Service Name:' AppSlug: default: 'Service Slug:' CustomCookbooksBucket: default: 'S3 Resource Bucket' AppEFSMountInfo: default: 'EFS Mount Info' ELMEFSStackName: default: 'Name of the ELM EFS CloudFormation Stack'

Parameters

These are the input parameters for this template. All of these parameters must be supplied for this template to be deployed.

Parameters: EnvAppName: MinLength: '3' Type: String Default: MyApp SFTP Hub Description: Full Application name, ie 'MyApp SFTP Hub' AppSlug: MinLength: '3' Type: String Default: sftp-hub Description: Short application slug, ie 'sftp-hub'. Lowercase letters, numbers and dashes only AllowedPattern: "[a-z0-9-]*" KeyName: Description: Amazon EC2 Key Pair Type: AWS::EC2::KeyPair::KeyName CustomCookbooksSource: Description: URL to S3 cookbooks, ie 'https://s3.amazonaws.com/edu-arizona-ps-nonprod-eas/cookbooks.tar.gz' Default: https://s3-us-west-2.amazonaws.com/edu-arizona-ps-nonprod-eas/shared/ecs-utilities-cookbooks.tar.gz Type: String CustomCookbooksBucket: Default: edu-arizona-pilots-eas Description: S3 Bucket containing custom cookbooks. Just the bucket name, not a full arn. Type: String AppDockerImage: Description: 'Docker Image, i.e.: ecs/sftphub:latest' Default: 760232551367.dkr.ecr.us-west-2.amazonaws.com/eas/sftphub:latest Type: String DockerRepoAccessKey: Description: 'ie: ABCDEFG' Type: String DockerRepoSecretKey: Description: AWS Secret Access Key Type: String NoEcho: 'true' ELMEFSStackName: Description: 'Name of the ELM EFS CloudFormation Stack' Type: String Default: 'elm-efs' TagOwner: Description: Name of the Owner of this resource (Full Name) Type: String TagNetid: Description: UA NetID of responsible person Type: String TagProjectName: Description: Name of the Project this is for, or JIRA ticket for more info Type: String

Resources

These are all of the actual AWS resources created for this application.

Resources:

Application Layer

This defines a Layer in the OpsWorks stack. It sets up the Chef cookbooks to pull in from S3, as well as a list of specific Chef recipies to run.

EnvApplicationLayer: Type: AWS::OpsWorks::Layer Properties: Name: sftp-hub Shortname: app Type: custom AutoAssignElasticIps: 'false' AutoAssignPublicIps: 'false' CustomRecipes: Setup: - ecs-utilities::efs_mount - ecs-docker::docker_install Deploy: - ecs-docker::docker_login_ecr - ecs-docker::docker_pull_deploy CustomSecurityGroupIds: - Ref: InstanceSecurityGroup - Fn::ImportValue: !Sub "${ELMEFSStackName}-target-sg" EnableAutoHealing: 'false' StackId: !Ref EnvStack

OpsWorks Application

This defines the application for the OpsWorks stack. There's one Application for each Docker container to be deployed. For KFS its just the one container.

The Application mostly provides a secure place to store secret texts for use with the Docker container passed in via environment variables.

KFSApp: Type: AWS::OpsWorks::App Properties: AppSource: Type: other Name: !Ref EnvAppName Shortname: !Ref AppSlug StackId: !Ref EnvStack Type: other Environment: - Key: layer Value: sftp-hub - Key: efs_mounts Value: !Sub - "elm:${ElmEfsId}" - {

Retrieve the ID of the ELM EFS Volume

ElmEfsId: !ImportValue elm-efs-fs-id } - Key: s3resource_bucket Value: !Ref CustomCookbooksBucket - Key: docker_repository_access_key Value: !Ref DockerRepoAccessKey - Key: docker_repository_secret_key Secure: true Value: !Ref DockerRepoSecretKey - Key: docker_container Value: !Ref AppDockerImage - Key: docker_ports Value: 0.0.0.0:2022:22 - Key: docker_privileged Value: privileged - Key: docker_volumes Value: /efs/elm:/mosaic/ELM - Key: SFTP_ENV Value: ELM

OpsWorks Stack

This defines the OpsWork stack itself. Mostly its network configs and defaults. The main thing in here is the source for the Chef Cookbooks to be used by any layers.

EnvStack: Type: AWS::OpsWorks::Stack Properties: Name: !Ref EnvAppName Attributes: Color: rgb(209, 105, 41) ConfigurationManager: Name: Chef Version: '12' CustomCookbooksSource: Type: s3 Url: !Ref CustomCookbooksSource ServiceRoleArn: !GetAtt EnvServiceRole.Arn DefaultInstanceProfileArn: !GetAtt EnvInstanceProfile.Arn DefaultOs: Amazon Linux 2016.09 DefaultSshKeyName: !Ref KeyName DefaultRootDeviceType: ebs DefaultSubnetId: !ImportValue peoplesoft-vpc-private-subnet-a HostnameTheme: Layer_Dependent UseCustomCookbooks: 'true' VpcId: !ImportValue peoplesoft-vpc-vpcid CustomJson: opsworks_tags: instances: owner: !Ref TagOwner netid: !Ref TagNetid projectname: !Ref TagProjectName

Application Instance Node

This creates and turns on an OpsWorks EC2 Instance to actually run the application. It is assigned to a specific layer, and that's where it gets the Chef recipes to run.

AppInstanceZoneA: Type: AWS::OpsWorks::Instance Properties: InstallUpdatesOnBoot: true StackId: !Ref EnvStack InstanceType: t2.micro SubnetId: !ImportValue peoplesoft-vpc-private-subnet-a SshKeyName: !Ref KeyName LayerIds: - !Ref EnvApplicationLayer

Elastic Load Balancer (ELB)

This is an internal facing Load Balancer which will only be accessed from within the VPC, or campus through the VPN.

EnvApplicationLayerLoadBalancer: Type: AWS::ElasticLoadBalancing::LoadBalancer Properties: Scheme: internal ConnectionSettings: IdleTimeout: '300' HealthCheck: HealthyThreshold: '5' Interval: '60' Target: TCP:2022 Timeout: '10' UnhealthyThreshold: '3' LBCookieStickinessPolicy: - CookieExpirationPeriod: '28800' PolicyName: DefaultSessionTimeout Listeners: - LoadBalancerPort: '22' Protocol: TCP InstancePort: '2022' InstanceProtocol: TCP LoadBalancerName: !Sub "${AppSlug}-app-lb" SecurityGroups: - Ref: LoadBalancerSecurityGroup Subnets: - !ImportValue peoplesoft-vpc-private-subnet-a - !ImportValue peoplesoft-vpc-private-subnet-b Tags: - Key: Name Value: !Sub "${AppSlug}-lb" - Key: owner Value: !Ref TagOwner - Key: netid Value: !Ref TagNetid - Key: projectname Value: !Ref TagProjectName

Link the Load Balancer to the Application Layer

EnvApplicationLayerLoadBalancerAttachment: Type: AWS::OpsWorks::ElasticLoadBalancerAttachment Properties: ElasticLoadBalancerName: !Ref EnvApplicationLayerLoadBalancer LayerId: !Ref EnvApplicationLayer

OpsWorks Service Role

This is the role that is given to the OpsWorks service, which allows OpsWorks to manage AWS resources. This is a standard policy provided by AWS. See the AWS Documentation for OpsWorks Service Role

EnvServiceRole: Type: AWS::IAM::Role Properties: AssumeRolePolicyDocument: Version: '2012-10-17' Statement: - Effect: Allow Principal: Service: - opsworks.amazonaws.com Action: - sts:AssumeRole Path: "/" Policies: - PolicyName: opsworks-service PolicyDocument: Statement: - Action: - ec2:* - iam:PassRole - cloudwatch:GetMetricStatistics - elasticloadbalancing:* - rds:* Effect: Allow Resource: - "*"

Instance Role

This is the IAM role that will be applied to the OpsWorks EC2 Instances. Any AWS specific permissions that the node might need should be defined here.

EnvInstanceRole: Type: AWS::IAM::Role Properties: AssumeRolePolicyDocument: Version: '2012-10-17' Statement: - Effect: Allow Principal: Service: - ec2.amazonaws.com Action: - sts:AssumeRole Path: "/" Policies: - PolicyName: !Sub "${AppSlug}-s3policy" PolicyDocument: Version: '2012-10-17' Statement: - Sid: Stmt1452033379000 Effect: Allow Action: - s3:GetObject - s3:ListBucket Resource: - !Sub "arn:aws:s3:::${CustomCookbooksBucket}/*"

Instance Profile

This is just a little construct to connect a set of roles together into a profile. The profile is referenced in the OpsWorks stack itself.

EnvInstanceProfile: Type: AWS::IAM::InstanceProfile Properties: Path: "/" Roles: - !Ref EnvInstanceRole

Instance Security Group

Security group for the OpsWorks application instances themselves. Needs to permit incoming traffice from the ELB, and any other authorized incoming sources.

InstanceSecurityGroup: Type: AWS::EC2::SecurityGroup Properties: GroupDescription: Allow Load Balancer and SSH to host VpcId: !ImportValue peoplesoft-vpc-vpcid SecurityGroupIngress: - IpProtocol: tcp FromPort: '22' ToPort: '22' CidrIp: 0.0.0.0/0 - IpProtocol: tcp FromPort: '2022' ToPort: '2022' SourceSecurityGroupId: !Ref LoadBalancerSecurityGroup Tags: - Key: Name Value: !Sub "${AppSlug}-instance-sg" - Key: owner Value: !Ref TagOwner - Key: netid Value: !Ref TagNetid - Key: projectname Value: !Ref TagProjectName

Load Balancer Security Group

This is the Security Group that wraps the Load Balancer. This controls what network traffic is allowed into the ELB. Just web traffic is allowed from anywhere.

LoadBalancerSecurityGroup: Type: AWS::EC2::SecurityGroup Properties: GroupDescription: Allow SFTP traffic to the Load Balancer VpcId: !ImportValue peoplesoft-vpc-vpcid SecurityGroupIngress: - IpProtocol: tcp FromPort: '22' ToPort: '22' CidrIp: 0.0.0.0/0 Tags: - Key: Name Value: !Sub "${AppSlug}-load-balancer-sg" - Key: owner Value: !Ref TagOwner - Key: netid Value: !Ref TagNetid - Key: projectname Value: !Ref TagProjectName

Route53 DNS Record

Create a DNS entry in Route53 for this environment. This creates a CNAME pointing at the DNS name of the Load Balancer.

AppDnsRecord: Type: AWS::Route53::RecordSet Properties:

Append a period after the hosted zone DNS name

HostedZoneName: "Fn::Sub": - "${HostedZoneName}." - { HostedZoneName: !ImportValue peoplesoft-route53-dns } Name: "Fn::Sub": - "sftp.${AppSlug}.${HostedZoneName}." - { HostedZoneName: !ImportValue peoplesoft-route53-dns } Type: CNAME TTL: '900' ResourceRecords: - !GetAtt EnvApplicationLayerLoadBalancer.DNSName

Outputs

Output values that can be viewed from the AWS CloudFormation console.

Outputs: LoadBalancerDNS: Value: !GetAtt EnvApplicationLayerLoadBalancer.DNSName Export: Name: !Sub "${AWS::StackName}-lb-dns" SftpEndpointDNS: Value: !Ref AppDnsRecord Export: Name: !Sub "${AWS::StackName}-dns"