Rule: Auto Scaling groups with a load balancer should use health checks
This rule ensures that Auto Scaling groups with a load balancer apply health checks for better performance and reliability.
| Rule | Auto Scaling groups with a load balancer should use health checks |
| Framework | Federal Financial Institutions Examination Council (FFIEC) |
| Severity | ✔ Critical |
Auto Scaling Groups with Load Balancer Health Checks for FFIEC Compliance
Overview of the Rule
When operating under the guidelines of the Federal Financial Institutions Examination Council (FFIEC), it is required that organizations implement effective system health checks for all critical components. This includes the configuration of Auto Scaling groups in cloud environments, where the use of health checks is a critical aspect of maintaining system availability and reliability. For services running on AWS, this would require that the Auto Scaling groups, which automatically adjust the number of instances in response to traffic demand, are configured to utilize health check mechanisms provided by attached load balancers.
Detailed Rule Description
Auto Scaling leverages both EC2 status checks and Elastic Load Balancer (ELB) health checks to determine the health status of instances. It is recommended that you configure your Auto Scaling group with ELB health checks for a more accurate determination of instance health. Here's a detailed description of how and why these checks should be implemented:
- EC2 Status Checks: Monitor the system and instance status. They help identify underlying problems with the instance that might prevent it from running your application.
- ELB Health Checks: Monitor the health of the application running on the instance and ensure traffic is only routed to healthy instances.
Using ELB health checks with Auto Scaling ensures that instances not passing the health check are automatically removed from the Auto Scaling group and replaced with healthy ones. This is critical for meeting FFIEC requirements, which mandate the ability to maintain functionality in the face of component failures.
Troubleshooting Steps
Troubleshooting Auto Scaling and load balancer health checks typically involves the following steps:
- 1.
Check Instance Status:
- Ensure instances are passing EC2 status checks.
- Look at system log files for any application or service-level errors.
- 2.
Verify Load Balancer Configuration:
- Confirm that the load balancer is properly configured with health check parameters (such as health check intervals, health thresholds, and unhealthy thresholds).
- Check the security group and network ACL associated with the load balancer to make sure it allows incoming traffic on health check ports.
- 3.
Review Health Check Responses:
- Analyze the response data from the health checks to diagnose if there're consistent timeouts or HTTP errors.
- Adjust the thresholds or health check parameters if necessary to prevent instances from being wrongly marked as unhealthy.
- 4.
Monitor Auto Scaling Activity:
- Use CloudWatch to monitor Auto Scaling events and alarms.
- Watch for "EC2 Instance-terminate Lifecycle Action" and investigate instances terminated by the Auto Scaling.
- 5.
Inspect Application Logs:
- Investigate application logs for any errors during startup or operation that could affect the response to the load balancer’s health checks.
Necessary Configuration
Step-by-Step Guide for Remediation
1. Configure Health Check for ELB:
aws elbv2 modify-target-group-attributes --target-group-arn ARN --attributes Key=healthy_threshold_count,Value=3 Key=unhealthy_threshold_count,Value=3
Replace
ARN2. Update Auto Scaling Group:
Update the Auto Scaling group to enable the use of ELB health checks:
aws autoscaling update-auto-scaling-group --auto-scaling-group-name ASGName --health-check-type ELB --health-check-grace-period 300
Replace
ASGName3. Monitor Auto Scaling Instances:
Use AWS CLI commands or AWS Management Console to monitor the instances in your Auto Scaling group for their health status.
Necessary CLI Command for Verification
Verify your Auto Scaling group's health check configuration:
aws autoscaling describe-auto-scaling-groups --auto-scaling-group-names ASGName
Replace
ASGNameReconciliation of an Auto Scaling configuration with FFIEC compliance guidelines involves close attention to the health checks in place. It is crucial to employ both EC2 and ELB health checks to ensure resiliency and operational integrity, a core value that aligns with the goals established by the FFIEC for financial institutions' technology infrastructure. Implementing these measures is not only a step toward compliance but also a best practice to enhance the reliability of cloud-based services.