Automate CloudWatch Agent Setup on EC2 with Terraform and AWS SSM

Avatar photoByAdil

Automate CloudWatch Agent Setup on EC2 with Terraform and AWS SSM

Managing monitoring agents across EC2 instances can be challenging, but AWS Systems Manager (SSM) simplifies this with seamless automation. This post demonstrates how to use Terraform to deploy the CloudWatch Agent automatically to EC2 instances tagged for monitoring with both Linux and Windows configurations.

This Terraform configuration sets up AWS Systems Manager (SSM) to automatically install and configure the CloudWatch Agent on EC2 instances tagged for monitoring, supporting both Linux and Windows. It includes IAM roles, SSM parameters, CloudWatch log groups, an S3 bucket for logs, CloudWatch dashboards, and SSM associations for automation.

Key Components in the Terraform Configuration

  • IAM Role and Instance Profile

    Creates an IAM role with permissions for EC2 to use SSM and CloudWatch and attaches the AmazonSSMManagedInstanceCore and CloudWatchAgentServerPolicy AWS managed policies. An instance profile is created for EC2 instances to assume this role.

  • CloudWatch Log Groups

    Separate log groups for Linux (/aws/ec2/var/log/messages) and Windows (/aws/ec2/windows/logs) with 14-day retention. These are used to collect logs from the monitored instances.

  • SSM Parameters for CloudWatch Agent Configuration

    Stores JSON configurations for the CloudWatch agent. Linux and Windows use different parameters specifying metrics to collect like CPU, memory, disk, network, and system events, adhering to best practices and AWS guidelines.

  • S3 Bucket for SSM Logs

    A versioned and encrypted S3 bucket with lifecycle rules to store and manage logs from SSM associations, ensuring logs are kept secure and cleaned up after 30 days.

  • CloudWatch Dashboard

    A dashboard showing CPU, memory, and disk usage metrics for EC2 instances monitored by the CloudWatch agent, aiding in quick health visualization.

  • SSM Associations

    Three associations automate CloudWatch Agent installation and configuration:

    • install_cloudwatch_agent: Installs the agent on EC2 instances tagged cloudwatch=enabled.
    • configure_cloudwatch_agent_linux: Uses the Linux configuration parameter to configure the agent on Linux instances.
    • configure_cloudwatch_agent_windows: Uses the Windows configuration parameter to configure the agent on Windows instances.

How to Use This Setup

  1. Tag your EC2 instances with cloudwatch=enabled and os=Linux or os=Windows.

    This ensures only targeted instances receive the CloudWatch agent installation and configuration.

  2. Attach the IAM instance profile produced by this Terraform to your EC2 instances. For example:

    iam_instance_profile_name output contains the profile name to use.

  3. Deploy the Terraform configuration. This will create all the resources, including IAM roles, log groups, SSM parameters, S3 bucket, dashboards, and associations.

  4. SSM will run the associations on schedule (every 30–40 minutes) ensuring the CloudWatch agent is installed and configured automatically.

  5. Monitor using the CloudWatch dashboard created to visualize key metrics.

Benefits of this Approach

  • Automated setup and configuration without manual intervention on each instance.
  • Centralized management of monitoring configurations via SSM parameters.
  • Cost-effective log management with lifecycle policies on S3.
  • Support for both Linux and Windows instances with tailored metric collection.
  • Easy scaling by tagging more instances.

GitHub Repository

This Terraform code is pushed and maintained in a GitHub repository for easy access and version control. You can clone, customize, or contribute enhancements as needed.

Similar Posts