Cloud-based architectures are the future, and AWS is a main player in the cloud space. So chances are high that every DevOps engineer will need to monitor infrastructure on AWS at some point.
The cloud eases many pain points in the long run, such as maintaining low-level resources like servers. But it also introduces some new challenges when you’re just starting out. For example, when deploying software on-premises, it used to be much easier to replicate the production environment on a local machine. But with managed services like AWS Lambda and Amazon DynamoDB, this isn’t the case anymore.
Luckily, AWS offers many monitoring services (Amazon CloudWatch, AWS Config, AWS X-Ray, AWS CloudTrail, and more) that give you insights into other AWS services, which may otherwise be black boxes. This might seem a bit overwhelming at first, but don’t worry. In this post, we’ll go through each option one by one to clear things up.
Amazon CloudWatch
First up is CloudWatch, the central monitoring and logging service on the AWS cloud platform. It integrates with virtually every service on AWS and can give you deep technical insights into your infrastructure.
CloudWatch follows AWS’s mantra that every service needs an API to enable customers to use it to its full potential. This makes it possible to create services that aren’t just monitored by CloudWatch, but can also react to the monitoring data emitted.
CloudWatch comes with pre-defined metrics for all AWS services and even allows you to define your own application-specific metrics. It also offers dashboards you can deploy alongside your infrastructure, so you can find all the information you need at once glance.
AWS Config
AWS Config is a configuration monitoring service that watches the configuration used to deploy AWS resources. By defining rules for configurations, AWS Config can check if your infrastructure is in compliance and notify you if any resources deviate from these rules. This is crucial when deploying services that need to adhere to specific laws, like HIPAA.
Infrastructure as code (IaC) plays a crucial role in software development today. If you want to stay agile and deliver features and bug fixes quickly, you don’t have time to constantly mess around with the AWS console. AWS Config helps keep your IaC clean of bad practices and potential compliance violations. It also lets you define rules and checks if your resource configuration conforms to those rules. For some, it can even offer automatic remediation actions.
AWS X-Ray
X-Ray is AWS’s central cloud call tracer. Providing similar functionality to application performance management (APM) solutions, it helps correlate events happening in your infrastructure. This is helpful because when your system consists of multiple services that all talk to each other, it can be very frustrating to follow an event manually.
For example, let’s say a request comes in at an API Gateway and is transformed by a Lambda function, which reads data from a DynamoDB table and saves a file into an S3 bucket. Then, a background process is started with Step Functions, which will notify some users via SES. All this can get quite cumbersome.
X-Ray links events together by one ID to follow the routes a request took through your system, eventually finding the point in the chain where things went awry. It also comes with a nice visualization that makes it easy to follow the path of an event.
AWS CloudTrail
CloudTrail is a managed audit log in the cloud. When running a business and working with private data, it’s most important to know what’s happening with that data, and deploying infrastructure in a data center miles away doesn’t make this task any simpler.
CloudTrail logs who is accessing which service and when each service was accessed. Again, if you have to follow governance policies for compliance reasons, this service is a must. For example, if your organization stores health-related data, HIPAA will require you to log all access to that data. CloudTrail makes it simple to get the audit data in a central place.
While AWS Config (see previous section) focuses on the what, CloudTrail is more about the who and when. AWS Config will tell you which resource’s config is in violation of a rule. CloudTrail, on the other hand, will help you find the root cause of this violation, see how it was changed over time, and discover who was responsible for these changes.
OpsCompass, a cloud security and management solution that gives you real-time visibility, intelligence, and control, uses CloudTrail logs to get insights into your AWS account—and, in turn, into your hosted infrastructure in the cloud. This way, OpsCompass can gather all the information needed to check your system for compliance. In addition, it suggests actionable tasks to solve them.
The AWS Well-Architected Tool
For over a decade, AWS has worked with thousands of companies all over the world, gathering deep knowledge about best practices for using its cloud infrastructure. AWS distilled this knowledge into numerous white papers under the AWS Well-Architected Framework.
To ease the work required to apply these best practices, AWS created the AWS Well-Architected Tool, a managed service that automatically analyzes your AWS infrastructure and checks its compliance to the rules of the Well-Architected Framework.
AWS License Manager
Before software as a service (SaaS) was a widespread business model, software licensing was the main method of software distribution. If you ran software without a license from its creators, you would be in violation of their contract and could be liable.
AWS License Manager is a managed service that scans your EC2 instances for the software running on them, then checks if you have the licenses required to run that software. It can also prevent software from starting when you run out of licenses and help you avoid violating your contracts by accident.
AWS Organizations and AWS Control Tower
As your company starts using multiple AWS services, one AWS account likely won’t be enough. Multiple accounts can help you structure your billing—and they provide much stronger isolation than you’d get if you deploy all your infrastructure in one account. Also, AWS accounts come with free tier usage and limits on the maximum number of resources they can manage. So if your company outgrows those limits, you’ll probably need multiple accounts to get by.
AWS Organizations is a service that helps you manage multiple AWS accounts centrally. It allows you to monitor costs and resource usage, so you don’t get surprise bills from maverick accounts created by a single person in your company.
AWS Control Tower works in tandem with AWS Organizations to ease the work related to creating multiple accounts and managing user permissions. Additionally, it allows you to define the rules that these accounts have to follow. It also monitors the accounts, so you’ll see when an account doesn’t behave according to the rules you define.
AWS Budgets
AWS Budgets is a managed service that helps you keep track of infrastructure spending. It allows you to set budgets and alerts you when necessary so you don’t accidentally spend too much. This is important because when building infrastructure that is effectively rented from a cloud provider, you have to keep an eye on the costs, especially if you automate the provisioning of these resources. One zero too many could significantly increase provisioning and eat a month’s budget in a day.
Sadly, there is currently no way to cap your spend on AWS. However, AWS Budgets at least gives you a heads up if things get out of control. Also, AWS is usually very accommodating with respect to rogue services. For example, if you accidentally accumulated a $1,000 bill, you can write a support ticket and AWS may cancel the charge.
Summary
Deploying your infrastructure miles away in data centers owned by another company can be scary, especially if you’re used to managing your own servers. But getting rid of the undifferentiated heavy lifting in your development and operational processes is the key to staying competitive.
From technical monitoring to auditing, account management, and budgeting, AWS’s service portfolio has you covered. As we discussed in this post, the many services it offers can help you stay on top of all things monitoring—and some can even teach you how to use AWS services efficiently. This means that if you use the right monitoring services, you can explore the AWS ecosystem without a negative developer experience or the fear of massive bills.
House of Brick brings a holistic approach to cloud given decades of enterprise system experience. As an AWS Advanced Consulting Partner, HoB has leveraged the many migration programs, tools like Database Migration Service and Schema Conversion Tool to help customers. In addition, OpsCompass uses AWS services like CloudTrail to give you deeper insights into your infrastructure. It shows you compliance issues—and how to resolve them. It even goes so far as to prepare you for compliance audits, so you have all the data that auditors require in one place. Best of all, OpsCompass works with multiple cloud providers. This way, you only need to consult one source of truth, independently of how your deployments are spread around the world.
Interested in learning more? Check out our free trial.
And, see how House of Brick Solves Customers’ Enterprise System Challenges with AWS.
