AWS data collection

Capacity Planner Cloud Cost Optimization analyses your AWS EC2 and other service estates to identify opportunities to remove costs and optimise the running of your AWS cloud.

To do this, read-only access is acquired to various components within AWS in order to understand the configuration of instances, the utilisation of the resources, your preconfigured reserved instances, savings plans, and tagging.

Note that when extracting data from cloud providers, Capacity Planner the data collectors are run from the environment using secure read-only credentials provided by the customer. This avoids unnecessary network transfer, the need to upgrade and maintain on-premise data collectors, and ensures that data collection is always at the most up-to-date release.

The preferred way to access data is to set up an IAM role in each AWS account to be analysed, and allow the Data Collector to assume that role in order to collect data. For more information, see Set up IAM role based Data Collector access.

Access and security Copied

Once retrieved, data is stored on the hosting machine inside the firewall. It is segregated in separate folders by customer and project. Once collected, the data is zipped and encrypted using AES encryption and uploaded to a database exclusively created for the given customer’s data.

ITRS is ISO 27001 accredited. For more information, see our Security policies.

Connectivity Copied

Access to an AWS estate is done on an account by account basis. To allow for minimum required access while leaving control in your hands, we use IAM role security.

IAM role security requires you to create a new security role on your estate, with read only permissions for just the data that ITRS requires, and to then grant access to that role to the Data Collector. In this way, you can stop access at any time, or adjust permissions.

Please note that Capacity Planner requires a minimum amount of data in order to provide analysis and recommendations for the your estate. See What data is collected.

Full details on how to set up an IAM role and access for AWS data collection are described in Set up IAM role based Data Collector access.

Once the IAM role has been granted, the Data Collector will connect to the AWS API within a secure HTTPS session and obtain an access token to authenticate each subsequent message until the collection is complete.

What data is collected Copied

The Data Collector will attempt to get the following data from the AWS estate:

• RDS and EC2 virtual machine instances and their properties.

• Disk images associated with EC2 instances.

• Metrics for RDS and EC2 instances, specifically:

  • CPU Utilization
  • CPU Credit Balance
  • CPU Credit Usage
  • CPU Surplus Credit Balance
  • CPU Surplus Credits Charged
  • Network In
  • Network Out

• Additional metrics for RDS instances, specifically:

  • Database Connections
  • Disk Queue Depth
  • Freeable Memory
  • Free Storage Space
  • Network Receive Throughput
  • Network Transmit Throughput
  • Read IOPS
  • Read Latency
  • Read Throughput
  • Swap Usage
  • Write IOPS
  • Write Latency
  • Write Throughput

• Cloud trail events relating to RDS and EC2 instances, specifically

  • RunInstances
  • TerminateInstances
  • StartInstances
  • StopInstances
  • CreateDBInstance
  • DeleteDBInstance
  • StartDBInstance
  • StopDBInstance

• RDS and EC2 Reserved instances

  • Savings plans
  • Accounts names and costs from Cost Explorer

Note

While the Data Collector will attempt to retrieve the above listed metrics, they may not be available depending on the system setup.

In addition to the EC2 metrics listed, it is possible to configure the Data Collector to retrieve additional custom metrics depending on availability, for example Memory Utilization.