Case Study - Automated Log Aggregation on AWS
Stratus10 assisted Peachjar in implementing an automated log aggregation solution that encompassed pushing logs from their EKS clusters, aggregating them in Cloudwatch, and visualizing them in Elasticsearch Service. The solution is reusable across Peachjar’s environments, scales with their compute layer, and enables granular monitoring and alerting capabilities across their platform.
Peachjar is a medium-size educational technology company founded in 2011 that produces eLearning, communication, and other software solutions targeted at increasing parent and student engagement in the learning environment. Peachjar unites schools, parents and communities in a joint mission to elevate student outcomes. Its cloud-based communication platform improves and streamlines school-to-home and community-to-school-to-home communication by distributing critical information and resources directly to parents as interactive digital flyers.
Peachjar required a solution for granular monitoring and alerting of the service-oriented platform running on EKS. They have several services that are hosted in the cluster, and they need to be able to monitor and alert on specific information about each service. Their existing logging solution lacked uniformity, some logs were only available by accessing their cluster directly, and it failed to meet their data retention requirements.
Peachjar needed a homogenous solution that would work across their containerized environment. The logs needed to be centralized and dashboards needed to show both performance metrics and logging/tracing data. The Peachjar team needed to be able to manage the verbosity of logging at the cluster level, and they needed to be able to retain a copy of the logs indefinitely.
Peachjar hosts their application through AWS EKS, and the new ContainerInsights solution provides a perfect foundation for the log aggregation they were looking for. Its integration directly into Cloudwatch for both metrics and logging provided an out-of-the-box solution that meets Peachjar’s retention and data control requirements, and Cloudwatch’s integration with Elasticsearch Service provided the advanced visualization layer Peachjar required.
Stratus10 was the perfect fit to deliver the log aggregation solution for Peachjar. Our expertise across AWS’ suite of services ensured that Peachjar’s solution was delivered using the best combination of tools available. Stratus10’s experience in delivering complex containerized solutions across the AWS ecosystem resulted in a smooth delivery of the solution that met all of the customers scaling, cost, and performance targets.
Stratus10 provided a 4-component solution comprised of Cloudwatch, Cloudwatch Container Insights, Lambda, and Elasticsearch Service.
Container Insights was deployed onto the EKS cluster. This service uses FluentD to collect container-level performance and operational metrics and all of the container logging. The logs and metrics are pushed to AWS Cloudwatch with an indefinite retention policy ensuring Peachjar never loses access to the original logs.
From Cloudwatch, a Log Stream to a Lambda function automatically pushes the logs and the metrics into Elasticsearch Service. Stratus10 recommended pushing the logs for each K8 service into their own index in Elasticsearch Service, giving Peachjar the ability to query all log output for a given service, or aggregate multiple indexes into essentially a logstream of their entire cluster.
The Elasticsearch Service was deployed onto the Peachjar VPC using a private endpoint, and provided more advanced visualization capabilities than Cloudwatch has natively. Delivering the solution through Cloudformation ensured Peachjar could easily repeat the deployment across their environments.
Results and Benefits:
The log aggregation solution delivered by Stratus10 met all of Peachjar’s goals. Peachjar is able to monitor the operational performance of their application at the pod, service, and node levels. They have the ability to generate alerts and perform automated remediation on their cluster when application performance does not meet expectations, and they are able to granularly visualize this performance over time. Finally, developers and system engineers can easily find logs, reducing the time it takes Peachjar to debug issues and giving them the ability to more tightly control access onto their cluster.
"Stratus10 helped us overcome complicated VPN management and procurement for our staff members to our AWS account, as well as help us centralize our application and server logs and make them easily searchable!"
- Wesley Van Rensburg, CTO Peachjar
Stratus10 is an Advanced AWS Partner Network (APN) Consulting Partner and an AWS DevOps Competency Partner helping companies migrate to the cloud or if they are already on AWS we help them implement best practices. We specialize in DevOps automation, Application Modernization, migration services, and cost optimization to help our clients take full advantage of the latest technologies AWS has to offer.
Use case: Solutions Architecture
Date: February 2021