uar-nonprod-opsworks.yaml
---

UAR Environment CloudFormation Deployment

This CloudFormation template will build out a whole UAR environment, including an OpsWorks stack, Load Balancer, Application nodes.

AWSTemplateFormatVersion: '2010-09-09' Description: UAR OpsWorks Stack - Development (dev, tst, trn, etc)

Metadata

Metadata is mostly for organizing and presenting Parameters in a better way when using CloudFormation in the AWS Web UI.

Metadata: AWS::CloudFormation::Interface: ParameterGroups: - Label: default: Tags Parameters: - UarInstanceTypeStg - UarInstanceTypeDevTst - ServiceTag - TagName - EnvironmentTag - VPCStackName - TagCreatedBy - ContactNetidTag - AccountNumberTag - SubAccountTag - TicketNumberTag - TagResourceFunction

Parameters

These are the input parameters for this template. All of these parameters must be supplied for this template to be deployed.

Parameters: HostedZoneId: Type: String Description: Hosted zone ID (NOT the full ARN) describing the Route53 hosted zone to self-register with. HostedZoneName: MinLength: '3' Type: String Description: 'Name of Route53 Hosted Zone: ie ''aws.arizona.edu''' VPCStackName: Type: String Description: Name of the VPC CF stack Default: 'admin-tech-nonprod-vpc' EnvAppName: MinLength: '3' Type: String Description: Full Application name, ie 'Uaccess Research' EnvSlug: MinLength: '2' Type: String Description: Short environment slug, ie 'dev', or 'markdev'. Lowercase letters, numbers and dashes only AllowedPattern: "[a-z0-9]*" UarInstanceTypeDevTst: Description: EC2 Instance Type Type: String Default: m5.large UarInstanceTypeStg: Description: EC2 Instance Type Type: String Default: r5.large AppSlug: MinLength: '3' Type: String Description: Short application slug, ie 'UAR'. Lowercase letters, numbers and dashes only AllowedPattern: "[a-z0-9-]*" VPCID: Description: Target VPC Type: AWS::EC2::VPC::Id AppSubnetA: Description: Application Subnet for Zone A Type: AWS::EC2::Subnet::Id AppSubnetB: Description: Application Subnet for Zone B Type: AWS::EC2::Subnet::Id SSLCertARN: Description: Application SSL Certificate ARN Type: String LBSubnetA: Description: Load Balancer Subnet for Zone A Type: AWS::EC2::Subnet::Id LBSubnetB: Description: Load Balancer Subnet for Zone B Type: AWS::EC2::Subnet::Id KeyName: Description: Amazon EC2 Key Pair Type: AWS::EC2::KeyPair::KeyName GrantsGovURL: Description: URL for grants gov Type: String Default: https://trainingws.grants.gov:446/grantsws-applicant/services/v2/ CustomCookbooksSource: Description: URL to UAR cookbooks, ie 'https://kuali-uar-artifacts.s3-us-west-2.amazonaws.com/cookbooks/uar-cookbooks-UAR-2544.tar.gz' Default: https://kuali-uar-artifacts.s3-us-west-2.amazonaws.com/cookbooks/uar-cookbooks-UAR-2544.tar.gz Type: String CookbookRepositoryVersion: Description: Tag or Revision of git repository to use Default: 26 Type: String EFSTargetSecurityGroup: Description: EFS Target Security group, ie 'sg-28b8a14e' Type: AWS::EC2::SecurityGroup::Id Default: sg-28b8a14e AppEFSID: Description: 'EFS Volume id: fs-49b14ce0' Type: String AppBaseURL: Description: 'Base Application URL: i.e.: https://ka-foo.mosaic.arizona.edu/' Type: String AppBuildVersion: Description: 'Application Build version : i.e.: ua-release34' Default: "ua-release36" Type: String AppRdsDbInstance: Description: RDS Database Name Default: "uar-nonprod" Type: String AppDBSchemaName: Description: Oracle Database Name Default: "KCDEV" Type: String AppDBUsername: Description: Database Username Type: String AppDBPassword: Description: Database Password Type: String NoEcho: 'true' AppKeystorePassword: Description: Key Store Password Type: String NoEcho: 'true' AppCACertsPassword: Description: CA Certs Password Type: String NoEcho: 'true' AppLdapUsername: Description: EDS LDAP Username Type: String NoEcho: 'true' AppLdapPassword: Description: EDS LDAP Password Type: String NoEcho: 'true' AppLdapHostname: Description: EDS LDAP Hostname Type: String AppNewRelicKey: Description: NewRelic License Key Type: String NoEcho: 'true' AppS3ResourceBucket: Description: S3 Bucket containing /security and /classes folders. Just the bucket name, not a full arn. Type: String AppSMTPUsername: Description: AWS SES Username Credentials Type: String NoEcho: 'true' AppSMTPPassword: Description: AWS SES Password Credentials Type: String NoEcho: 'true' DBSecurityGroup: Description: Database Security Group Type: String ServiceTag: Description: Service name (from the service catalog) that is utilizing this resource Type: String TagName: Description: Descriptive identifier of resource. Type: String EnvironmentTag: Description: Type of environment that is using this resource, such as 'dev', 'tst', 'prd'. Type: String TagCreatedBy: Description: NetID of the user that created this resource Type: String ContactNetidTag: Description: NetID of the person to contact for information about this resource Type: String AccountNumberTag: Description: Financial system account number for the service utilizing this resource Type: String SubAccountTag: Description: Financial system SUB account number for the service utilizing this resource Type: String TicketNumberTag: Description: Ticket number that this resource is for Type: String TagResourceFunction: Description: Human-readable description of what function this resource is providing Type: String #Logic is:

All of these are 24/7 instances. If STG we will use an instance type to match PRD If Dev or TST we are using a small instance type Prototype is a placeholder in case we want to start adding prototypes for testing in UAR at a later date

Conditions: IsPrototypeEnv: !Not [ !Or [ !Equals [!Ref EnvSlug, "dev"], !Equals [!Ref EnvSlug, "tst"], !Equals [!Ref EnvSlug, "stg"], !Equals [!Ref EnvSlug, "prd"], !Equals [!Ref EnvSlug, "trn"] ] ] IsStg: !Equals [!Ref EnvSlug, "stg"] IsDevTstTrn: !Or [ !Equals [!Ref EnvSlug, "dev"], !Equals [!Ref EnvSlug, "tst"], !Equals [!Ref EnvSlug, "trn"]]

Resources

These are all of the actual AWS resources created for this application.

Resources:

Application Layer

This defines a Layer in the OpsWorks stack. It sets up the Chef cookbooks to pull in from S3, as well as a list of specific Chef recipes to run.

EnvApplicationLayer: Type: AWS::OpsWorks::Layer Properties: Name: UAR

There is currently a chef recipe dependency on this shortname prefix

Shortname: "app-" Type: custom AutoAssignElasticIps: 'false' AutoAssignPublicIps: 'false' CustomRecipes: Setup: - kci::application_setup - kinst::update_dns - kci::add_users - kci::tomcat_cron - kci::cloudwatch_logs Configure: - kci::application_configure Deploy: - kci::application_deploy Undeploy: - kci::application_undeploy Shutdown: - kci::application_shutdown CustomSecurityGroupIds: - !Ref InstanceSecurityGroup - !Ref EFSTargetSecurityGroup EnableAutoHealing: 'false' StackId: Ref: EnvStack

OpsWorks Application

This defines the application for the OpsWorks stack. NOTE: UAR does NOT use docker like KFS does, so the UAR application is installed directly onto the created EC2 instances by Opsworks

UARApp: Type: AWS::OpsWorks::App Properties: AppSource: Type: other Name: Ref: EnvAppName Shortname: "kc" StackId: Ref: EnvStack Type: other DataSources: - Arn: !Sub "arn:aws:rds:${AWS::Region}:${AWS::AccountId}:db:${AppRdsDbInstance}" DatabaseName: !Ref AppDBSchemaName Type: RdsDbInstance

OpsWorks Stack

The prototype Stack will now have -proto appended to their names. Because of the limitations of Cloud custodian + Cloud Formation templates we are going this route.

This defines the OpsWork stack itself. Mostly its network configs and defaults. The main thing in here is the source for the Chef Cookbooks to be used by any layers.

EnvStack: Type: AWS::OpsWorks::Stack Properties: # if statement below determines if the name has -proto in it. Name: !Sub "UAR ${EnvSlug}" VpcId: !Ref VPCID ServiceRoleArn: !ImportValue fdn-iam-opsworks-service-role-arn DefaultInstanceProfileArn: !GetAtt EnvInstanceProfile.Arn DefaultOs: "Amazon Linux 2018.03" DefaultSshKeyName: !Ref KeyName DefaultRootDeviceType: ebs DefaultSubnetId: !Ref AppSubnetA HostnameTheme: Layer_Dependent UseCustomCookbooks: 'true' Attributes: Color: rgb(209, 105, 41) ConfigurationManager: Name: Chef Version: '11.10' ChefConfiguration: BerkshelfVersion: "3.2.0" ManageBerkshelf: True CustomCookbooksSource: Type: s3 Url: !Ref CustomCookbooksSource Revision: !Ref CookbookRepositoryVersion RdsDbInstances: - DbUser: !Ref AppDBUsername DbPassword: !Ref AppDBPassword RdsDbInstanceArn: !Sub "arn:aws:rds:${AWS::Region}:${AWS::AccountId}:db:${AppRdsDbInstance}" CustomJson: opsworks_tags: instances: service: !Ref ServiceTag Name: !Ref TagName environment: !Ref EnvironmentTag createdby: !Ref TagCreatedBy contactnetid: !Ref ContactNetidTag accountnumber: !Ref AccountNumberTag subaccount: !Ref SubAccountTag ticketnumber: !Ref TicketNumberTag resourcefunction: !Ref TagResourceFunction deploy: kc: s3_basename: kuali-coeus upstream_version: 5.2.1 local_version: !Ref AppBuildVersion application_host: !Ref AppBaseURL ldap_url: !Ref AppLdapHostname ldap_username: !Ref AppLdapUsername ldap_password: !Ref AppLdapPassword cacerts_password: !Ref AppCACertsPassword keystore_password: !Ref AppKeystorePassword kinst: system: net: domain: !Ref HostedZoneName hosted_zone_id: !Ref HostedZoneId region: !Ref AWS::Region component: liquibase: dist_bucket: !Ref AppS3ResourceBucket sqlplus: dist_bucket: !Ref AppS3ResourceBucket tomcat: dist_bucket: !Ref AppS3ResourceBucket newrelic: dist_bucket: !Ref AppS3ResourceBucket jdk: dist_bucket: !Ref AppS3ResourceBucket kci: application: application: kc: environment: !Ref EnvSlug efs_resource_id: !Sub "${AppEFSID}.efs.${AWS::Region}.amazonaws.com" s3_bucket: !Ref AppS3ResourceBucket s3_prefix: kci/builds s3_region: !Ref AWS::Region keystore_dist_bucket: !Ref AppS3ResourceBucket grants_gov_host: !Ref GrantsGovURL keystore_file_name: UofA_KC_Keystore.jks context_name: !Sub "kra-${EnvSlug}" install_as: kra component: tomcat: setenv_mem_opts: "-Xms6g -Xmx30g -XX:MaxPermSize=1g -XX:+UseTLAB " AppInstanceDevTstEnv: Type: AWS::OpsWorks::Instance Condition: IsDevTstTrn Properties: StackId: !Ref EnvStack InstanceType: !Ref UarInstanceTypeDevTst SubnetId: Fn::ImportValue: !Sub "${VPCStackName}-private-subnet-a" SshKeyName: !Ref KeyName Hostname: !Sub "uar-${EnvSlug}" InstallUpdatesOnBoot: true LayerIds: - Ref: EnvApplicationLayer RootDeviceType: ebs BlockDeviceMappings: - DeviceName: ROOT_DEVICE Ebs: DeleteOnTermination: true VolumeSize: 20 VolumeType: gp2 AppInstancePrototypeEnv: Type: AWS::OpsWorks::Instance Condition: IsPrototypeEnv Properties: StackId: !Ref EnvStack InstanceType: !Ref UarInstanceTypeDevTst SubnetId: Fn::ImportValue: !Sub "${VPCStackName}-private-subnet-a" SshKeyName: !Ref KeyName Hostname: !Sub "uar-${EnvSlug}" InstallUpdatesOnBoot: true LayerIds: - Ref: EnvApplicationLayer RootDeviceType: ebs BlockDeviceMappings: - DeviceName: ROOT_DEVICE Ebs: DeleteOnTermination: true VolumeSize: 20 VolumeType: gp2 AppInstanceStgEnv: Type: AWS::OpsWorks::Instance Condition: IsStg Properties: StackId: !Ref EnvStack InstanceType: !Ref UarInstanceTypeStg SubnetId: Fn::ImportValue: !Sub "${VPCStackName}-private-subnet-a" SshKeyName: !Ref KeyName Hostname: !Sub "uar-${EnvSlug}" InstallUpdatesOnBoot: true LayerIds: - Ref: EnvApplicationLayer RootDeviceType: ebs BlockDeviceMappings: - DeviceName: ROOT_DEVICE Ebs: DeleteOnTermination: true VolumeSize: 20 VolumeType: gp2

UAR Elastic Load Balancer (ELB)

Defines the Load Balancer for UAR. http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-ec2-elb.html

EnvApplicationLayerLoadBalancer: Type: AWS::ElasticLoadBalancing::LoadBalancer Properties: Scheme: internal CrossZone: true ConnectionSettings: IdleTimeout: '300' HealthCheck: HealthyThreshold: '5' Interval: '60' Target: TCP:80 Timeout: '10' UnhealthyThreshold: '3' ConnectionDrainingPolicy: Enabled: 'true' Timeout: '300' LBCookieStickinessPolicy: - CookieExpirationPeriod: '28800' PolicyName: DefaultSessionTimeout Policies: - Attributes: - Name: "Reference-Security-Policy" Value: "ELBSecurityPolicy-TLS-1-2-2017-01" PolicyName: UarTlsSecurityPolicy PolicyType: SSLNegotiationPolicyType Listeners: - LoadBalancerPort: '443' Protocol: HTTPS InstancePort: '80' InstanceProtocol: HTTP PolicyNames: - DefaultSessionTimeout - UarTlsSecurityPolicy SSLCertificateId: !Ref SSLCertARN LoadBalancerName: !Sub "${AppSlug}-${EnvSlug}-app-lb" SecurityGroups: - Ref: LoadBalancerSecurityGroup Subnets: - Ref: LBSubnetA - Ref: LBSubnetB Tags: - Key: service Value: !Ref ServiceTag - Key: Name Value: !Sub "${TagName}-uar-lb" - Key: environment Value: !Ref EnvironmentTag - Key: createdby Value: !Ref TagCreatedBy - Key: contactnetid Value: !Ref ContactNetidTag - Key: accountnumber Value: !Ref AccountNumberTag - Key: subaccount Value: !Ref SubAccountTag - Key: ticketnumber Value: !Ref TicketNumberTag - Key: resourcefunction Value: !Ref TagResourceFunction EnvApplicationLayerLoadBalancerAttachment: Type: AWS::OpsWorks::ElasticLoadBalancerAttachment Properties: ElasticLoadBalancerName: !Ref EnvApplicationLayerLoadBalancer LayerId: !Ref EnvApplicationLayer

Instance Role

This is the IAM role that will be applied to the OpsWorks EC2 Instances. Any AWS specific permissions that the node might need should be defined here.

EnvInstanceRole: Type: AWS::IAM::Role Properties: AssumeRolePolicyDocument: Version: '2012-10-17' Statement: - Effect: Allow Principal: Service: - ec2.amazonaws.com Action: - sts:AssumeRole Path: "/" ManagedPolicyArns: - arn:aws:iam::aws:policy/service-role/AmazonEC2RoleforSSM - arn:aws:iam::aws:policy/AmazonEC2ContainerRegistryReadOnly Policies:

Access to the S3 Bucket which holds application specific files that need to be loaded on each application node. (ojdbc.jar, encrypted keystores, etc)

- PolicyName: !Sub "${AppSlug}-${EnvSlug}-s3policy" PolicyDocument: Version: '2012-10-17' Statement: - Sid: Stmt1452033379000 Effect: Allow Action: - s3:GetObject - s3:ListBucket Resource: !Sub "arn:aws:s3:::${AppS3ResourceBucket}/*"

Access to CloudWatch for the log group and log stream from each application environment.

- PolicyName: !Sub "${AppSlug}-${EnvSlug}-cloudwatchlogspolicy" PolicyDocument: Version: '2012-10-17' Statement: - Sid: cloudwatchlogsaccess Effect: Allow Action: - logs:CreateLogGroup - logs:CreateLogStream - logs:Describe* - logs:PutLogEvents Resource: - "*"

IAM Access for Route53 to the UAR envs

- PolicyName: !Sub "${AppSlug}-${EnvSlug}-route53policy" PolicyDocument: Version: '2012-10-17' Statement: - Sid: Stmt1491867038000 Effect: Allow Action: - route53:ChangeResourceRecordSets Resource: !Sub "arn:aws:route53:::hostedzone/${HostedZoneId}"

CloudWatch Logs Group

Create a CloudWatch Log Group for each application environment. This allows us to set the retention timeframe. UAFAWS-302 - Create dependency on CWlog Log group and EC2 instance with CWlogs agent. During CF template delete CW agent has access to recreate log group after first pass

EnvLogGroup: Type: "AWS::Logs::LogGroup" DependsOn: EnvStack Properties: LogGroupName: !Sub "${AppSlug}-${EnvSlug}-application" RetentionInDays: 7

Instance Profile

This is just a little construct to connect a set of roles together into a profile. The profile is referenced in the OpsWorks stack itself.

EnvInstanceProfile: Type: AWS::IAM::InstanceProfile Properties: Path: "/" Roles: - !Ref EnvInstanceRole

Instance Security Group

Security group for the OpsWorks application instances themselves. Needs to permit incoming traffice from the ELB, and any other authorized incoming sources.

InstanceSecurityGroup: Type: AWS::EC2::SecurityGroup Properties: GroupDescription: Allow Load Balancer and SSH to host VpcId: !Ref VPCID SecurityGroupIngress: - IpProtocol: "tcp" FromPort: "22" ToPort: "22" CidrIp: "150.135.112.0/22" Description: "Infradev VPN" - IpProtocol: "tcp" FromPort: "22" ToPort: "22" CidrIp: "10.138.0.0/17" Description: "Mosaic VPN" - IpProtocol: "tcp" FromPort: "22" ToPort: "22" CidrIp: "128.196.130.211/32" Description: "ben.uits bastion host" - IpProtocol: "tcp" FromPort: "22" ToPort: "22" CidrIp: "128.196.135.64/26" Description: "337 wired" - IpProtocol: tcp FromPort: '80' ToPort: '80' SourceSecurityGroupId: !Ref LoadBalancerSecurityGroup Tags: - Key: service Value: !Ref ServiceTag - Key: Name Value: !Sub "${TagName}-instance-sg" - Key: environment Value: !Ref EnvironmentTag - Key: createdby Value: !Ref TagCreatedBy - Key: contactnetid Value: !Ref ContactNetidTag - Key: accountnumber Value: !Ref AccountNumberTag - Key: subaccount Value: !Ref SubAccountTag - Key: ticketnumber Value: !Ref TicketNumberTag - Key: resourcefunction Value: !Ref TagResourceFunction

Load Balancer Security Group

This is the Security Group that wraps the Load Balancer. This controls what network traffic is allowed into the ELB. Just web traffic is allowed from anywhere.

LoadBalancerSecurityGroup: Type: AWS::EC2::SecurityGroup Properties: GroupDescription: Allow web traffic to the Load Balancer VpcId: !Ref VPCID SecurityGroupIngress: - IpProtocol: tcp FromPort: '443' ToPort: '443' CidrIp: 0.0.0.0/0 Description: "Normal Application Flow" Tags: - Key: service Value: !Ref ServiceTag - Key: Name Value: !Sub "${TagName}-lb-sg" - Key: environment Value: !Ref EnvironmentTag - Key: createdby Value: !Ref TagCreatedBy - Key: contactnetid Value: !Ref ContactNetidTag - Key: accountnumber Value: !Ref AccountNumberTag - Key: subaccount Value: !Ref SubAccountTag - Key: ticketnumber Value: !Ref TicketNumberTag - Key: resourcefunction Value: !Ref TagResourceFunction

UAR Route53 DNS Record

Create a DNS entry in Route53 for the UAR ELB. This creates a CNAME pointing at the DNS name of the UAR Load Balancer.

AppDnsRecord: Type: AWS::Route53::RecordSet Properties: HostedZoneName: !Sub "${HostedZoneName}." Name: !Sub "${AppSlug}-${EnvSlug}.${HostedZoneName}." Type: CNAME TTL: '900' ResourceRecords: - !GetAtt EnvApplicationLayerLoadBalancer.DNSName #Use to add the Instance SG to the DB SG #This is needed to have the app servers connect to the DB AppInstToDB1521: Type: "AWS::EC2::SecurityGroupIngress" Properties: GroupId: !Ref DBSecurityGroup IpProtocol: "tcp" FromPort: "1521" ToPort: "1521" SourceSecurityGroupId: !Ref InstanceSecurityGroup Description: "App Instance"

Outputs

Output values that can be viewed from the AWS CloudFormation console.

Outputs: LoadBalancerDNS: Value: !GetAtt EnvApplicationLayerLoadBalancer.DNSName AppDNS: Value: !Sub "https://${AppSlug}-${EnvSlug}.${HostedZoneName}"