DevOps Digest #36: Kubernetes 1.22, NSA released k8s hardening guidance, DigitalOcean managed MongoDB

💡 Усі статті, обговорення, новини про DevOps — в одному місці. Приєднуйтесь до DevOps спільноти!

Summary: Lots of releases from Grafana Labs, RDS now supports Graviton instances, Kubernetes Pod Security Admission, and the largest DDOS ever.

NSA and CISA release guidance

AWS

Core services (EC2, S3, VPC, CloudFormation)

1. Added support for hibernation for CentOS 8 (+RedHat + Fedora) for On Demand Nitro-based instances , more types of instances (C5d, M5d, R5d) supports hibernation
2. New VT1 instances for video transcoding (with hardware accelerators Xilinx^® Alveo™ U30), new GPU instances G4ad (advertised as 40% better price-performance)
3. Added an ability to customize reverse DNS for Elastic IP addresses
4. More AWS Local Zones for single millisecond latency for customers in 10 US metro areas
5. Introduced AMI aliases for images published in AWS Marketplace — no need to change AMI ID for every region anymore (can simplify deployment scripts)
6. Amazon EFS (Elastic File System) introduces Intelligent-Tiering for cost optimization
7. Some improvements in routing internal traffic (between subnets in VPC) which may be useful for setup intrusion detection systems
8. CloudFormation introduces the option to skip rollback on failure (may speedup troubleshooting\development)
9. Amazon Virtual Private Cloud (VPC) customers can now use their own Prefix Lists to simplify the configuration of security groups and route tables
10. IPv6 endpoints now available for EC2 Instance Metadata Service, Amazon Time Sync Service and VPC DNS Server (useful if you are switching to IPv6 stack)
11. Support UEFI (boot) for AWS EC2 / EC2 VM Import/Export
12. New memory-optimized type of instances — M6i, a blog post about this
13. 79 new resource types supported by CloudFormation
14. Added possibility to assign prefixes to EC2 instances in VPC
15. New EBS type io2 Block Express (up to 256 000 IOPS and 7500 MB/s)
16. New Public Registry for CloudFormation
17. High-memory instances u-* (up to 12 TB of RAM and 448 CPUs) available on-demand (price is up to 110 $/hour)
18. EC2 enables replacing root volume for quick restoration and troubleshooting

More high level (RDS, MKS)

1. RDS supports more instance types (T4g + new memory-optimized X2g, R5b). Switching to ARM instances (T4g) may improve cost/performance up to 36%.
2. Multiple auth modes for MKS (Managed Kafka) + updates to TLS encryption settings
3. 19 new metrics for MKS
4. Amazon API Gateway supports mutual TLS with certificates from third-party CAs
5. RDS proxy can be created in shared VPC (it’s possible to share VPC between accounts in the same AWS Organization, with some limitations)

Services (API)

1. Amazon Transcribe (speech to text service) can be configured to auto remove personal identifiable information
2. Amazon Rekognition improves the accuracy of celebrity recognition, adds new attributes

CI/CD services

1. Amazon CodeGuru Reviewer enhances security findings generated by GitHub Action by adding severity fields and CWE tags

Instructions in GitHub Marketplace on how to set this up can be found here.

1. AWS CodeBuild supports arm instances (may be used for reducing cost)
2. CodeGuru introduced inconsistency detection in source code
3. AWS Copilot (CLI for creating ECS/Fargate services) now supports Pub/Sub architectures
4. Compare options for application profile in CodeGuru
5. Elastic Beanstalk supports Capacity Rebalancing for EC2 Spot instances
6. CodeGuru adds recommendation support for Python
7. CodeBuild support publicly viewable build results
8. CDK Pipelines available (helps with defining Pipeline-as-a-Code)
9. DevOps Monitoring Dashboard adds support for CodeBuild and CodePipeline metrics

Container-related (EKS, ECS)

1. ECR adds an ability to replicate individual repositories to other regions and accounts
2. EKS (managed Kubernetes) can register nodes outside AWS (in public preview from 2021-09-08)
3. EKS supports Multus (meta-CNI plugin)
4. EBS supports idempotent volume creation (useful in CSI driver)
5. Amazon VPC CNI plugin increases pods per node limits by using new capability of assigning CIDR prefixes to ENI
6. EKS and EKS Distro support Kubernetes v1.21
7. AWS App Runner — new service (integrated with AWS CodeBuild) for containers, also released integration with JetBrains IDEs
8. EKS add-ons now supports CoreDNS and kube-proxy

Other

1. AWS IQ (freelance market) allows registering freelancers/consulting firms located in the UK and France, more details here.
2. New offering ‘Build on AWS’ in AWS Activate — looks like a curated managed list of CloudFormation solutions.
3. Amazon SES (Simple Email Service) supports emails up to 40MB
4. Announcing custom widgets for CloudWatch dashboards
5. Improvements in IAM Access Analyzer (generates fine-grained IAM policy from CloudTrail)
6. New exam for Certified SysOps Administrator includes Exam Labs tasks
7. GA for AWS Proton
8. GA for AWS Location Service
9. GA for Lambda Extensions (sidecar pattern, more documentation is here)
10. GA CloudFormation Guard 2.0

GCP

Google Cloud had massive outage in the australia-southeast2 region for 1 hour and 30 minutes. “The issue was transient voltage at the feeder to the network equipment, causing the equipment to reboot. In order to mitigate the issue, traffic within the australia-southeast2 region was redirected temporarily.”

Backup for GKE — now you can protect your GKE workloads more easily.

Artifact Registry, a service with support container images and non-container artifacts, adds support for Node.js, Python, and Java packages.

Google announced Anthos Config Management — set and enforce consistent configurations and policies for your Kubernetes resources.

Kubernetes

Kubernetes 1.22 released.

PSP replacement, rootless mode, seccomp by default, node swap support (in alpha though) are just a small portion of enhancements that are present in the new k8s release. Both PDB and CronJob are finally graduated to Stable!

Top 10 Kubernetes Application Security Hardening Techniques

Security is a key, and Kubernetes is not an exception. Most important security options that should be included in manifests to make your workloads safe.

Kubernetes Pod Security Admission is the successor of Pod Security Policy (PSP) which is scheduled for deprecation in v1.25. This article is a brief overview of new functionality that you should be aware of.

NSA, CISA release Kubernetes Hardening Guidance.

And already there is a tool for testing if Kubernetes is deployed accordingly to this guidance github.com/armosec/kubescape

Automatic Remediation of Kubernetes Nodes

Cloudflare open-sources Scurio — a replacement of node-problem-detector that has one simple job: synchronize Kubernetes node conditions with currently firing alerts in Alertmanager. Here they describe how this tool helps automatically detect and remediate Kubernetes workers with ancient docker issue.

Observability & Incident management

Both Grafana 8.0 and 8.1 were released since the last digest. New panels and visualizations, real-time streaming, improved log navigation, and lots of performance optimizations. Check it out!

Loki 2.3

15x query speed on recent data, deletion/retention improvements, recording rules, and many more.

Introducing Panopticon, A Generic Kubernetes State Metrics Exporter

Quite an interesting attempt to create some generic state metrics exporter as a counterpart to kube-state-metrics. Panopticon differs in a way it can work with custom resources.

An Introduction to Distributed Tracing

Yet another article stated the importance of distributed tracing. If you have reached a point where you’re running more than five interdependent services at once, that means you definitely need distributed tracing.

Alerting on SLOs like Pros

Shortlist of recommendations from SoundCloud on how to properly build SLO. Based on the famous Google SRE book it can serve as a reminder or TLDR if you haven’t read this book.

Interesting

GA for DigitalOcean Managed MongoDB

New DBaaS offering from DigitalOcean, built in partnership with and certified by MongoDB Inc. Pricing starts from $15/month, or a highly available three-node replica set for $45/month.

Email Authenticity 101: DKIM, DMARC, and SPF

Old but gold. All you wanted to know about email security.

Facebook, Google, Isovalent, Microsoft and Netflix Launch eBPF Foundation as Part of the Linux Foundation

The Linux Foundation announced that it is hosting the eBPF Foundation. It should help eBPF extend its powerful capabilities and grow beyond Linux.

17.2M rps DDoS attack

The biggest DDOS attack so far! The attack traffic originated from more than 20,000 bots in 125 countries around the world. Based on the bots’ source IP addresses, almost 15% of the attack originated from Indonesia and another 17% from India and Brazil combined.

Comparison of July DDOS attack with overall Q2 rps

Над дайджестом работали: