kfs_sftp_hub.yaml
---

KFS Environment CloudFormation Deployment

This CloudFormation template will build an OpsWorks stack to support a central SFTP server with multiple EFS systems mounted.

AWSTemplateFormatVersion: '2010-09-09' Description: KFS SFTP OpsWorks Stack

Metadata

Metadata is mostly for organizing and presenting Parameters in a better way when using CloudFormation in the AWS Web UI.

Metadata: AWS::CloudFormation::Interface: ParameterGroups: - Label: default: Application Information Parameters: - EnvAppName - AppSlug - Label: default: Instance Settings Parameters: - KeyName - CustomCookbooksSource - CustomCookbooksBucket - Label: default: Application Settings Parameters: - AppDockerImage - DockerRepoAccessKey - DockerRepoSecretKey - AppEFSMountInfo - Label: default: Tags Parameters: - TagService - TagName - TagEnvironment - TagCreatedBy - TagContactNetId - TagAccountNumber - TagSubAccount - TagTicketNumber - TagResourceFunction ParameterLabels: EnvAppName: default: 'Service Name:' AppSlug: default: 'Service Slug:' CustomCookbooksBucket: default: 'S3 Resource Bucket' AppEFSMountInfo: default: 'EFS Mount Info'

Parameters

These are the input parameters for this template. All of these parameters must be supplied for this template to be deployed.

Parameters: EnvAppName: MinLength: '3' Type: String Default: Kuali SFTP Hub Description: Full Application name, ie 'Kuali SFTP Hub' AppSlug: MinLength: '3' Type: String Default: kuali-sftp-nonprod Description: Short application slug, ie 'kuali-sftp-nonprod'. Lowercase letters, numbers and dashes only AllowedPattern: "[a-z0-9-]*" KeyName: Description: Amazon EC2 Key Pair Type: AWS::EC2::KeyPair::KeyName CustomCookbooksSource: Description: URL to S3 cookbooks, ie 'https://s3.amazonaws.com/edu-arizona-pilots-eas/cookbooks.tar.gz' Default: https://s3-us-west-2.amazonaws.com/edu-arizona-pilots-eas/shared/ecs-utilities-cookbooks.tar.gz Type: String CustomCookbooksBucket: Default: edu-arizona-pilots-eas Description: S3 Bucket containing custom cookbooks. Just the bucket name, not a full arn. Type: String AppDockerImage: Description: 'Docker Image, i.e.: ecs/sftphub:latest' Default: 760232551367.dkr.ecr.us-west-2.amazonaws.com/eas/sftphub:latest Type: String DockerRepoAccessKey: Description: 'ie: ABCDEFG' Type: String DockerRepoSecretKey: Description: AWS Secret Access Key Type: String NoEcho: 'true' EFSStackName: Description: 'Name of the EFS CloudFormation Stack' Type: String Default: 'kfs-efs' TagService: Description: Service name (from the service catalog) that is utilizing this resource Type: String TagName: Description: Descriptive identifier of resource. Type: String TagEnvironment: Description: Type of environment that is using this resource, such as 'dev', 'tst', 'prd'. Type: String TagCreatedBy: Description: NetID of the user that created this resource Type: String TagContactNetId: Description: NetID of the person to contact for information about this resource Type: String TagAccountNumber: Description: Financial system account number for the service utilizing this resource Type: String TagSubAccount: Description: Financial system subaccount number for the service utilizing this resource Type: String TagTicketNumber: Description: Ticket number that this resource is for Type: String TagResourceFunction: Description: Human-readable description of what function this resource is providing Type: String

Resources

These are all of the actual AWS resources created for this application.

Resources:

Application Layer

This defines a Layer in the OpsWorks stack. It sets up the Chef cookbooks to pull in from S3, as well as a list of specific Chef recipies to run.

EnvApplicationLayer: Type: AWS::OpsWorks::Layer Properties: Name: sftp-hub Shortname: app Type: custom AutoAssignElasticIps: 'false' AutoAssignPublicIps: 'false' CustomRecipes: Setup: - ecs-utilities::efs_mount - ecs-docker::docker_install Deploy: - ecs-docker::docker_login_ecr - ecs-docker::docker_pull_deploy CustomSecurityGroupIds: - Ref: InstanceSecurityGroup - Fn::ImportValue: !Sub "${EFSStackName}-target-sg" EnableAutoHealing: 'false' StackId: !Ref EnvStack

OpsWorks Application

This defines the application for the OpsWorks stack. There's one Application for each Docker container to be deployed. For KFS its just the one container.

The Application mostly provides a secure place to store secret texts for use with the Docker container passed in via environment variables.

KFSApp: Type: AWS::OpsWorks::App Properties: AppSource: Type: other Name: !Ref EnvAppName Shortname: !Ref AppSlug StackId: !Ref EnvStack Type: other Environment: - Key: layer Value: sftp-hub - Key: efs_mounts Value: "Fn::Sub": - "kfs:${EFSID}" - EFSID: Fn::ImportValue: !Sub "${EFSStackName}-fs-id" - Key: s3resource_bucket Value: !Ref CustomCookbooksBucket - Key: docker_repository_access_key Value: !Ref DockerRepoAccessKey - Key: docker_repository_secret_key Secure: true Value: !Ref DockerRepoSecretKey - Key: docker_container Value: !Ref AppDockerImage - Key: docker_ports Value: 0.0.0.0:2022:22 - Key: docker_privileged Value: privileged - Key: docker_volumes Value: /efs/kfs:/mosaic/KFS_NONPROD - Key: SFTP_ENV Value: KFS_NONPROD

OpsWorks Stack

This defines the OpsWork stack itself. Mostly its network configs and defaults. The main thing in here is the source for the Chef Cookbooks to be used by any layers.

EnvStack: Type: AWS::OpsWorks::Stack Properties: Name: !Ref EnvAppName Attributes: Color: rgb(209, 105, 41) ConfigurationManager: Name: Chef Version: '12' CustomCookbooksSource: Type: s3 Url: !Ref CustomCookbooksSource ServiceRoleArn: !GetAtt EnvServiceRole.Arn DefaultInstanceProfileArn: !GetAtt EnvInstanceProfile.Arn DefaultOs: Amazon Linux 2016.09 DefaultSshKeyName: !Ref KeyName DefaultRootDeviceType: ebs DefaultSubnetId: !ImportValue kuali-vpc-private-subnet-a HostnameTheme: Layer_Dependent UseCustomCookbooks: 'true' VpcId: !ImportValue kuali-vpc-vpcid CustomJson: opsworks_tags: instances: service: !Ref TagService Name: !Ref TagName environment: !Ref TagEnvironment createdby: !Ref TagCreatedBy contactnetid: !Ref TagContactNetId accountnumber: !Ref TagAccountNumber subaccount: !Ref TagSubAccount ticketnumber: !Ref TagTicketNumber resourcefunction: !Ref TagResourceFunction

Application Instance Node

This creates and turns on an OpsWorks EC2 Instance to actually run the application. It is assigned to a specific layer, and that's where it gets the Chef recipes to run.

AppInstanceZoneA: Type: AWS::OpsWorks::Instance Properties: InstallUpdatesOnBoot: true StackId: !Ref EnvStack InstanceType: t2.micro SubnetId: !ImportValue kuali-vpc-private-subnet-a SshKeyName: !Ref KeyName LayerIds: - !Ref EnvApplicationLayer

Elastic Load Balancer (ELB)

This is an internal facing Load Balancer which will only be accessed from within the VPC, or campus through the VPN.

EnvApplicationLayerLoadBalancer: Type: AWS::ElasticLoadBalancing::LoadBalancer Properties: Scheme: internal ConnectionSettings: IdleTimeout: '300' HealthCheck: HealthyThreshold: '5' Interval: '60' Target: TCP:2022 Timeout: '10' UnhealthyThreshold: '3' LBCookieStickinessPolicy: - CookieExpirationPeriod: '28800' PolicyName: DefaultSessionTimeout Listeners: - LoadBalancerPort: '22' Protocol: TCP InstancePort: '2022' InstanceProtocol: TCP LoadBalancerName: !Sub "${AppSlug}-app-lb" SecurityGroups: - Ref: LoadBalancerSecurityGroup Subnets: - !ImportValue kuali-vpc-private-subnet-a - !ImportValue kuali-vpc-private-subnet-b Tags: - Key: service Value: !Ref TagService - Key: Name Value: !Sub "${TagName}-lb" - Key: environment Value: !Ref TagEnvironment - Key: createdby Value: !Ref TagCreatedBy - Key: contactnetid Value: !Ref TagContactNetId - Key: accountnumber Value: !Ref TagAccountNumber - Key: subaccount Value: !Ref TagSubAccount - Key: ticketnumber Value: !Ref TagTicketNumber - Key: resourcefunction Value: !Ref TagResourceFunction

Link the Load Balancer to the Application Layer

EnvApplicationLayerLoadBalancerAttachment: Type: AWS::OpsWorks::ElasticLoadBalancerAttachment Properties: ElasticLoadBalancerName: !Ref EnvApplicationLayerLoadBalancer LayerId: !Ref EnvApplicationLayer

OpsWorks Service Role

This is the role that is given to the OpsWorks service, which allows OpsWorks to manage AWS resources. This is a standard policy provided by AWS. See the AWS Documentation for OpsWorks Service Role

EnvServiceRole: Type: AWS::IAM::Role Properties: AssumeRolePolicyDocument: Version: '2012-10-17' Statement: - Effect: Allow Principal: Service: - opsworks.amazonaws.com Action: - sts:AssumeRole Path: "/" Policies: - PolicyName: opsworks-service PolicyDocument: Statement: - Action: - ec2:* - iam:PassRole - cloudwatch:GetMetricStatistics - elasticloadbalancing:* - rds:* Effect: Allow Resource: - "*"

Instance Role

This is the IAM role that will be applied to the OpsWorks EC2 Instances. Any AWS specific permissions that the node might need should be defined here.

EnvInstanceRole: Type: AWS::IAM::Role Properties: AssumeRolePolicyDocument: Version: '2012-10-17' Statement: - Effect: Allow Principal: Service: - ec2.amazonaws.com Action: - sts:AssumeRole Path: "/" Policies: - PolicyName: !Sub "${AppSlug}-s3policy" PolicyDocument: Version: '2012-10-17' Statement: - Sid: Stmt1452033379000 Effect: Allow Action: - s3:GetObject - s3:ListBucket Resource: - !Sub "arn:aws:s3:::${CustomCookbooksBucket}/*"

Instance Profile

This is just a little construct to connect a set of roles together into a profile. The profile is referenced in the OpsWorks stack itself.

EnvInstanceProfile: Type: AWS::IAM::InstanceProfile Properties: Path: "/" Roles: - !Ref EnvInstanceRole

Instance Security Group

Security group for the OpsWorks application instances themselves. Needs to permit incoming traffice from the ELB, and any other authorized incoming sources.

InstanceSecurityGroup: Type: AWS::EC2::SecurityGroup Properties: GroupDescription: Allow Load Balancer and SSH to host VpcId: !ImportValue kuali-vpc-vpcid SecurityGroupIngress: - IpProtocol: tcp FromPort: '22' ToPort: '22' CidrIp: 0.0.0.0/0 - IpProtocol: tcp FromPort: '2022' ToPort: '2022' SourceSecurityGroupId: !Ref LoadBalancerSecurityGroup Tags: - Key: service Value: !Ref TagService - Key: Name Value: !Sub "${TagName}-instance-sg" - Key: environment Value: !Ref TagEnvironment - Key: createdby Value: !Ref TagCreatedBy - Key: contactnetid Value: !Ref TagContactNetId - Key: accountnumber Value: !Ref TagAccountNumber - Key: subaccount Value: !Ref TagSubAccount - Key: ticketnumber Value: !Ref TagTicketNumber - Key: resourcefunction Value: !Ref TagResourceFunction

Load Balancer Security Group

This is the Security Group that wraps the Load Balancer. This controls what network traffic is allowed into the ELB. Just web traffic is allowed from anywhere.

LoadBalancerSecurityGroup: Type: AWS::EC2::SecurityGroup Properties: GroupDescription: Allow SFTP traffic to the Load Balancer VpcId: !ImportValue kuali-vpc-vpcid SecurityGroupIngress: - IpProtocol: tcp FromPort: '22' ToPort: '22' CidrIp: 0.0.0.0/0 Tags: - Key: service Value: !Ref TagService - Key: Name Value: !Sub "${TagName}-load-balancer-sg" - Key: environment Value: !Ref TagEnvironment - Key: createdby Value: !Ref TagCreatedBy - Key: contactnetid Value: !Ref TagContactNetId - Key: accountnumber Value: !Ref TagAccountNumber - Key: subaccount Value: !Ref TagSubAccount - Key: ticketnumber Value: !Ref TagTicketNumber - Key: resourcefunction Value: !Ref TagResourceFunction

Ingress Rule to EFS Volume Security Group

We need to add an ingress rule to the security group for our EFS volume

SecurityGroupIngressToEFS: Type: AWS::EC2::SecurityGroupIngress Properties: GroupId: Fn::ImportValue: !Sub ${EFSStackName}-sg IpProtocol: tcp FromPort: 2049 ToPort: 2049 SourceSecurityGroupId: !Ref InstanceSecurityGroup

Route53 DNS Record

Create a DNS entry in Route53 for this environment. This creates a CNAME pointing at the DNS name of the Load Balancer.

AppDnsRecord: Type: AWS::Route53::RecordSet Properties:

Append a period after the hosted zone DNS name

HostedZoneName: "Fn::Sub": - "${HostedZoneName}." - { HostedZoneName: !ImportValue kfs-route53-dns } Name: "Fn::Sub": - "sftp.${AppSlug}.${HostedZoneName}." - { HostedZoneName: !ImportValue kfs-route53-dns } Type: CNAME TTL: '900' ResourceRecords: - !GetAtt EnvApplicationLayerLoadBalancer.DNSName

Outputs

Output values that can be viewed from the AWS CloudFormation console.

Outputs: LoadBalancerDNS: Value: !GetAtt EnvApplicationLayerLoadBalancer.DNSName Export: Name: !Sub "${AWS::StackName}-lb-dns" SftpEndpointDNS: Value: !Ref AppDnsRecord Export: Name: !Sub "${AWS::StackName}-dns"