uar-opsworks.yaml
---

UAR Environment CloudFormation Deployment

This CloudFormation template will build out a whole UAR environment, including an OpsWorks stack, Load Balancer, Application nodes and Database.

AWSTemplateFormatVersion: '2010-09-09' Description: UAR OpsWorks Stack - Development (dev, tst, trn, etc)

Metadata

Metadata is mostly for organizing and presenting Parameters in a better way when using CloudFormation in the AWS Web UI.

Metadata: AWS::CloudFormation::Interface: ParameterGroups: - Label: default: Tags Parameters: - ServiceTag - TagName - EnvironmentTag - TagCreatedBy - ContactNetidTag - AccountNumberTag - TicketNumberTag - TagResourceFunction

Parameters

These are the input parameters for this template. All of these parameters must be supplied for this template to be deployed.

Parameters: HostedZoneId: Type: String Description: Hosted zone ID (NOT the full ARN) describing the Route53 hosted zone to self-register with. HostedZoneName: MinLength: '3' Type: String Description: 'Name of Route53 Hosted Zone: ie ''aws.arizona.edu''' EnvAppName: MinLength: '3' Type: String Description: Full Application name, ie 'Kuali Financials' EnvSlug: MinLength: '2' Type: String Description: Short environment slug, ie 'dev', or 'markdev'. Lowercase letters, numbers and dashes only AllowedPattern: "[a-z0-9]*" AppSlug: MinLength: '3' Type: String Description: Short application slug, ie 'UAR'. Lowercase letters, numbers and dashes only AllowedPattern: "[a-z0-9-]*" VPCID: Description: Target VPC Type: AWS::EC2::VPC::Id AppSubnetA: Description: Application Subnet for Zone A Type: AWS::EC2::Subnet::Id AppSubnetB: Description: Application Subnet for Zone B Type: AWS::EC2::Subnet::Id SSLCertARN: Description: Application SSL Certificate ARN Type: String LBSubnetA: Description: Load Balancer Subnet for Zone A Type: AWS::EC2::Subnet::Id LBSubnetB: Description: Load Balancer Subnet for Zone B Type: AWS::EC2::Subnet::Id KeyName: Description: Amazon EC2 Key Pair Type: AWS::EC2::KeyPair::KeyName CustomCookbooksSource: Description: URL to S3 cookbooks, ie 'https://s3.amazonaws.com/edu-arizona-pilots-eas/cookbooks.tar.gz' Default: https://s3-us-west-2.amazonaws.com/edu-arizona-pilots-eas/financials/UAR-cookbooks.tar.gz Type: String CookbookRepositoryVersion: Description: Tag or Revision of git repository to use Default: 25 Type: String EFSTargetSecurityGroup: Description: EFS Target Security group, ie 'sg-28b8a14e' Type: AWS::EC2::SecurityGroup::Id Default: sg-28b8a14e AppEFSID: Description: 'EFS Volume id: fs-49b14ce0' Type: String AppBaseURL: Description: 'Base Application URL: i.e.: https://ka-foo.mosaic.arizona.edu/' Type: String AppBuildVersion: Description: 'Application Build version : i.e.: ua-release34' Default: "ua-release34" Type: String AppRdsDbInstance: Description: RDS Database Name Default: "uar-prd-dr" Type: String AppDBSchemaName: Description: Oracle Database Name Default: "KCPRD" Type: String AppDBUsername: Description: Database Username Type: String AppDBPassword: Description: Database Password Type: String NoEcho: 'true' AppKeystorePassword: Description: Key Store Password Type: String NoEcho: 'true' AppCACertsPassword: Description: CA Certs Password Type: String NoEcho: 'true' AppLdapUsername: Description: EDS LDAP Username Type: String NoEcho: 'true' AppLdapPassword: Description: EDS LDAP Password Type: String NoEcho: 'true' AppLdapHostname: Description: EDS LDAP Hostname Type: String AppNewRelicKey: Description: NewRelic License Key Type: String NoEcho: 'true' AppS3ResourceBucket: Description: S3 Bucket containing /security and /classes folders. Just the bucket name, not a full arn. Type: String AppSMTPUsername: Description: AWS SES Username Credentials Type: String NoEcho: 'true' AppSMTPPassword: Description: AWS SES Password Credentials Type: String NoEcho: 'true' DBSecurityGroup: Description: Database Security Group Type: String ServiceTag: Description: Service name (from the service catalog) that is utilizing this resource Type: String TagName: Description: Descriptive identifier of resource. Type: String EnvironmentTag: Description: Type of environment that is using this resource, such as 'dev', 'tst', 'prd'. Type: String TagCreatedBy: Description: NetID of the user that created this resource Type: String ContactNetidTag: Description: NetID of the person to contact for information about this resource Type: String AccountNumberTag: Description: Financial system account number for the service utilizing this resource Type: String TicketNumberTag: Description: Ticket number that this resource is for Type: String TagResourceFunction: Description: Human-readable description of what function this resource is providing Type: String #Logic is:

If DEV, TST, we only want a single, timer-based application instance If STG, we want A1 to be a 24/7 UAR application instance, B1 to be timer-based UAR application instance. Rice will be 24/7 instances. If SUP, we don't do anything extra since its deployment is controlled by MET If PRD, want A to be 24/7 application instance, B to be timer-based application instance If it is a prototype, this will be a single NON timer instance handled by Jenkins and cloud-custodian

Conditions: IsPrototypeEnv: !Not [ !Or [ !Equals [!Ref EnvSlug, "dev"], !Equals [!Ref EnvSlug, "tst"], !Equals [!Ref EnvSlug, "stg"], !Equals [!Ref EnvSlug, "sup"], !Equals [!Ref EnvSlug, "prd"], !Equals [!Ref EnvSlug, "trn"] ] ] IsStg: !Equals [!Ref EnvSlug, "stg"] IsSup: !Equals [!Ref EnvSlug, "sup"] IsPrd: !Equals [!Ref EnvSlug, "prd"] IsSingleTimerEnv: !Or [ !Equals [!Ref EnvSlug, "dev"], !Equals [!Ref EnvSlug, "tst"], !Equals [!Ref EnvSlug, "trn"] ] IsMultiInstance: !Or [Condition: IsStg, Condition: IsPrd]

Resources

These are all of the actual AWS resources created for this application.

Resources:

Application Layer

This defines a Layer in the OpsWorks stack. It sets up the Chef cookbooks to pull in from S3, as well as a list of specific Chef recipes to run.

EnvApplicationLayer: Type: AWS::OpsWorks::Layer Properties: Name: UAR

There is currently a chef recipe dependency on this shortname prefix

Shortname: "app-" Type: custom AutoAssignElasticIps: 'false' AutoAssignPublicIps: 'false' CustomRecipes:

You have to mount the EFS volumes BEFORE installing and running docker, otherwise docker doesn't pick up the mounts.

Setup: - kci::application_setup - kinst::update_dns - kci::add_users - kci::tomcat_cron - kci::cloudwatch_logs Configure: - kci::application_configure Deploy: - kci::application_deploy Undeploy: - kci::application_undeploy Shutdown: - kci::application_shutdown CustomSecurityGroupIds: - !Ref InstanceSecurityGroup - !Ref EFSTargetSecurityGroup EnableAutoHealing: 'false' StackId: Ref: EnvStack

OpsWorks Application

This defines the application for the OpsWorks stack. There's one Application for eac Docker container to be deployed. For UAR its just the one container.

The Application mostly provides a secure place to store secret texts for use with the Docker container passed in via environment variables.

UARApp: Type: AWS::OpsWorks::App Properties: AppSource: Type: other Name: Ref: EnvAppName Shortname: "kc" StackId: Ref: EnvStack Type: other DataSources: - Arn: !Sub "arn:aws:rds:${AWS::Region}:${AWS::AccountId}:db:${AppRdsDbInstance}" DatabaseName: !Ref AppDBSchemaName Type: RdsDbInstance

OpsWorks Stack

The prototype Stack will now have -proto appended to their names. Because of the limitations of Cloud custodian + Cloud Formation templates we are going this route.

This defines the OpsWork stack itself. Mostly its network configs and defaults. The main thing in here is the source for the Chef Cookbooks to be used by any layers.

EnvStack: Type: AWS::OpsWorks::Stack Properties: # if statement below determines if the name has -proto in it. Name: !Sub "UAR ${EnvSlug}" VpcId: !Ref VPCID ServiceRoleArn: !ImportValue fdn-iam-opsworks-service-role-arn DefaultInstanceProfileArn: !GetAtt EnvInstanceProfile.Arn DefaultOs: "Amazon Linux 2017.09" DefaultSshKeyName: !Ref KeyName DefaultRootDeviceType: ebs DefaultSubnetId: !Ref AppSubnetA HostnameTheme: Layer_Dependent UseCustomCookbooks: 'true' Attributes: Color: rgb(209, 105, 41) ConfigurationManager: Name: Chef Version: '11.10' ChefConfiguration: BerkshelfVersion: "3.2.0" ManageBerkshelf: True CustomCookbooksSource: Type: git Url: !Ref CustomCookbooksSource Revision: !Ref CookbookRepositoryVersion RdsDbInstances: - DbUser: !Ref AppDBUsername DbPassword: !Ref AppDBPassword RdsDbInstanceArn: !Sub "arn:aws:rds:${AWS::Region}:${AWS::AccountId}:db:${AppRdsDbInstance}" CustomJson: opsworks_tags: instances: service: !Ref ServiceTag Name: !Ref TagName environment: !Ref EnvironmentTag createdby: !Ref TagCreatedBy contactnetid: !Ref ContactNetidTag accountnumber: !Ref AccountNumberTag ticketnumber: !Ref TicketNumberTag resourcefunction: !Ref TagResourceFunction deploy: kc: s3_basename: kuali-coeus upstream_version: 5.2.1 local_version: !Ref AppBuildVersion application_host: !Ref AppBaseURL ldap_url: !Ref AppLdapHostname ldap_username: !Ref AppLdapUsername ldap_password: !Ref AppLdapPassword cacerts_password: !Ref AppCACertsPassword keystore_password: !Ref AppKeystorePassword kinst: system: net: domain: !Ref HostedZoneName hosted_zone_id: !Ref HostedZoneId region: !Ref AWS::Region component: liquibase: dist_bucket: !Ref AppS3ResourceBucket sqlplus: dist_bucket: !Ref AppS3ResourceBucket tomcat: dist_bucket: !Ref AppS3ResourceBucket newrelic: dist_bucket: !Ref AppS3ResourceBucket jdk: dist_bucket: !Ref AppS3ResourceBucket kci: application: application: kc: environment: !Ref EnvSlug efs_resource_id: !Sub "${AppEFSID}.efs.${AWS::Region}.amazonaws.com" s3_bucket: !Ref AppS3ResourceBucket s3_prefix: kci/builds s3_region: !Ref AWS::Region keystore_dist_bucket: !Ref AppS3ResourceBucket grants_gov_host: https://ws07.grants.gov:446/grantsws-applicant/services/v2/ keystore_file_name: KeyStore_UofA_KCPRD.jks context_name: !Sub "kra-${EnvSlug}" install_as: kra component: tomcat: setenv_mem_opts: "-Xms6g -Xmx30g -XX:MaxPermSize=1g -XX:+UseTLAB "

UAR Elastic Load Balancer (ELB)

Defines the Load Balancer for UAR. http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-ec2-elb.html

EnvApplicationLayerLoadBalancer: Type: AWS::ElasticLoadBalancing::LoadBalancer Properties: Scheme: internal CrossZone: true ConnectionSettings: IdleTimeout: '300' HealthCheck: HealthyThreshold: '5' Interval: '60' Target: TCP:80 Timeout: '10' UnhealthyThreshold: '3' ConnectionDrainingPolicy: Enabled: 'true' Timeout: '300' LBCookieStickinessPolicy: - CookieExpirationPeriod: '28800' PolicyName: DefaultSessionTimeout Listeners: - LoadBalancerPort: '443' Protocol: HTTPS InstancePort: '80' InstanceProtocol: HTTP PolicyNames: - DefaultSessionTimeout SSLCertificateId: !Ref SSLCertARN LoadBalancerName: !Sub "${AppSlug}-${EnvSlug}-app-lb" SecurityGroups: - Ref: LoadBalancerSecurityGroup Subnets: - Ref: LBSubnetA - Ref: LBSubnetB Tags: - Key: service Value: !Ref ServiceTag - Key: Name Value: !Sub "${TagName}-uar-lb" - Key: environment Value: !Ref EnvironmentTag - Key: createdby Value: !Ref TagCreatedBy - Key: contactnetid Value: !Ref ContactNetidTag - Key: accountnumber Value: !Ref AccountNumberTag - Key: ticketnumber Value: !Ref TicketNumberTag - Key: resourcefunction Value: !Ref TagResourceFunction EnvApplicationLayerLoadBalancerAttachment: Type: AWS::OpsWorks::ElasticLoadBalancerAttachment Properties: ElasticLoadBalancerName: !Ref EnvApplicationLayerLoadBalancer LayerId: !Ref EnvApplicationLayer

Instance Role

This is the IAM role that will be applied to the OpsWorks EC2 Instances. Any AWS specific permissions that the node might need should be defined here.

EnvInstanceRole: Type: AWS::IAM::Role Properties: AssumeRolePolicyDocument: Version: '2012-10-17' Statement: - Effect: Allow Principal: Service: - ec2.amazonaws.com Action: - sts:AssumeRole Path: "/" ManagedPolicyArns: - arn:aws:iam::aws:policy/service-role/AmazonEC2RoleforSSM - arn:aws:iam::aws:policy/AmazonEC2ContainerRegistryReadOnly Policies:

Access to the S3 Bucket which holds application specific files that need to be loaded on each application node. (ojdbc.jar, encrypted keystores, etc)

- PolicyName: !Sub "${AppSlug}-${EnvSlug}-s3policy" PolicyDocument: Version: '2012-10-17' Statement: - Sid: Stmt1452033379000 Effect: Allow Action: - s3:GetObject - s3:ListBucket Resource: !Sub "arn:aws:s3:::${AppS3ResourceBucket}/*"

Access to CloudWatch for the log group and log stream from each application environment.

- PolicyName: !Sub "${AppSlug}-${EnvSlug}-cloudwatchlogspolicy" PolicyDocument: Version: '2012-10-17' Statement: - Sid: cloudwatchlogsaccess Effect: Allow Action: - logs:CreateLogGroup - logs:CreateLogStream - logs:Describe* - logs:PutLogEvents Resource: - "*"

Access to the S3 Bucket which holds application specific files that need to be loaded on each application node. (ojdbc.jar, encrypted keystores, etc)

- PolicyName: !Sub "${AppSlug}-${EnvSlug}-route53policy" PolicyDocument: Version: '2012-10-17' Statement: - Sid: Stmt1491867038000 Effect: Allow Action: - route53:ChangeResourceRecordSets Resource: !Sub "arn:aws:route53:::hostedzone/${HostedZoneId}"

Allow the Rhubarb container on this host to update ELB Listeners

- PolicyName: !Sub "${AppSlug}-${EnvSlug}-elbpolicy" PolicyDocument: Version: '2012-10-17' Statement: - Sid: elbdescribepolicy Effect: Allow Action: - elasticloadbalancing:DescribeListeners - elasticloadbalancing:DescribeLoadBalancers - elasticloadbalancing:DescribeLoadBalancerAttributes - elasticloadbalancing:CreateLBCookieStickinessPolicy Resource: "*" - Sid: elbupdatepolicy Effect: Allow Action: - elasticloadbalancing:ConfigureHealthCheck - elasticloadbalancing:CreateLoadBalancerListeners - elasticloadbalancing:DeleteLoadBalancerListeners - elasticloadbalancing:ModifyLoadBalancerAttributes - elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer - elasticloadbalancing:SetLoadBalancerPoliciesOfListener Resource: - !Sub "arn:aws:elasticloadbalancing:${AWS::Region}:${AWS::AccountId}:loadbalancer/${EnvApplicationLayerLoadBalancer}" - PolicyName: !Sub "${AppSlug}-${EnvSlug}-zabbixPolicy" PolicyDocument: Version: '2012-10-17' Statement: - Sid: StmtSsmParameterAccess Effect: Allow Action: - "ssm:GetParameters" Resource: - !Sub "arn:aws:ssm:us-west-2:${AWS::AccountId}:parameter/Zabbix*" - Sid: StmtKmsAccess Effect: Allow Action: - "kms:Decrypt" Resource: - !Sub "arn:aws:kms:us-west-2:${AWS::AccountId}:key/alias/aws/ssm"

CloudWatch Logs Group

Create a CloudWatch Log Group for each application environment. This allows us to set the retention timeframe. UAFAWS-302 - Create dependency on CWlog Log group and EC2 instance with CWlogs agent. During CF template delete CW agent has access to recreate log group after first pass

EnvLogGroup: Type: "AWS::Logs::LogGroup" DependsOn: EnvStack Properties: LogGroupName: !Sub "${AppSlug}-${EnvSlug}-application" RetentionInDays: 7

Instance Profile

This is just a little construct to connect a set of roles together into a profile. The profile is referenced in the OpsWorks stack itself.

EnvInstanceProfile: Type: AWS::IAM::InstanceProfile Properties: Path: "/" Roles: - !Ref EnvInstanceRole

Instance Security Group

Security group for the OpsWorks application instances themselves. Needs to permit incoming traffice from the ELB, and any other authorized incoming sources.

InstanceSecurityGroup: Type: AWS::EC2::SecurityGroup Properties: GroupDescription: Allow Load Balancer and SSH to host VpcId: !Ref VPCID SecurityGroupIngress: - IpProtocol: "tcp" FromPort: "22" ToPort: "22" CidrIp: "150.135.112.0/22" Description: "Infradev VPN" - IpProtocol: "tcp" FromPort: "22" ToPort: "22" CidrIp: "10.138.0.0/17" Description: "Mosaic VPN" - IpProtocol: "tcp" FromPort: "22" ToPort: "22" CidrIp: "128.196.130.211/32" Description: "ben.uits bastion host" - IpProtocol: "tcp" FromPort: "22" ToPort: "22" CidrIp: "128.196.135.64/26" Description: "337 wired" - IpProtocol: tcp FromPort: '80' ToPort: '80' SourceSecurityGroupId: !Ref LoadBalancerSecurityGroup

Zabbix port

- IpProtocol: tcp FromPort: '10050' ToPort: '10050' CidrIp: 128.196.130.92/32 Description: "Allow zabbix-proxy02.uits.arizona.edu" Tags: - Key: service Value: !Ref ServiceTag - Key: Name Value: !Sub "${TagName}-instance-sg" - Key: environment Value: !Ref EnvironmentTag - Key: createdby Value: !Ref TagCreatedBy - Key: contactnetid Value: !Ref ContactNetidTag - Key: accountnumber Value: !Ref AccountNumberTag - Key: ticketnumber Value: !Ref TicketNumberTag - Key: resourcefunction Value: !Ref TagResourceFunction

Load Balancer Security Group

This is the Security Group that wraps the Load Balancer. This controls what network traffic is allowed into the ELB. Just web traffic is allowed from anywhere.

LoadBalancerSecurityGroup: Type: AWS::EC2::SecurityGroup Properties: GroupDescription: Allow web traffic to the Load Balancer VpcId: !Ref VPCID SecurityGroupIngress: - IpProtocol: tcp FromPort: '443' ToPort: '443' CidrIp: 0.0.0.0/0 Description: "Normal Application Flow" Tags: - Key: service Value: !Ref ServiceTag - Key: Name Value: !Sub "${TagName}-lb-sg" - Key: environment Value: !Ref EnvironmentTag - Key: createdby Value: !Ref TagCreatedBy - Key: contactnetid Value: !Ref ContactNetidTag - Key: accountnumber Value: !Ref AccountNumberTag - Key: ticketnumber Value: !Ref TicketNumberTag - Key: resourcefunction Value: !Ref TagResourceFunction

UAR Route53 DNS Record

Create a DNS entry in Route53 for the UAR ELB. This creates a CNAME pointing at the DNS name of the UAR Load Balancer.

AppDnsRecord: Type: AWS::Route53::RecordSet Properties: HostedZoneName: !Sub "${HostedZoneName}." Name: !Sub "${AppSlug}-${EnvSlug}.${HostedZoneName}." Type: CNAME TTL: '900' ResourceRecords: - !GetAtt EnvApplicationLayerLoadBalancer.DNSName #Use to add the Instance SG to the DB SG #This is needed to have the app servers connect to the DB AppInstToDB1521: Type: "AWS::EC2::SecurityGroupIngress" Properties: GroupId: !Ref DBSecurityGroup IpProtocol: "tcp" FromPort: "1521" ToPort: "1521" SourceSecurityGroupId: !Ref InstanceSecurityGroup Description: "App Instance"

Outputs

Output values that can be viewed from the AWS CloudFormation console.

Outputs: LoadBalancerDNS: Value: !GetAtt EnvApplicationLayerLoadBalancer.DNSName AppDNS: Value: !Sub "https://${AppSlug}-${EnvSlug}.${HostedZoneName}"