DevOps Digest #35: DevOps Days Kyiv, AWS rocks, say goodbye to k8s PSP

💡 Усі статті, обговорення, новини про DevOps — в одному місці. Приєднуйтесь до DevOps спільноти!

Summary: Kubernetes 1.21, ArgoCD 2.0, Kafka can now work without Zookeeper and 2 new subsea cables through the Pacific Ocean.

Note: This is our first digest in English. We’re constantly experimenting with formats, materials, sections, etc. And this case is not an exception. Please let us know what do think about it.

DevOps Days comes back to Kyiv! Seth Vargo, Joel Tosi, Mandi Wells, Joe Beda, and lots of folks from ukrops community will talk about DevOps culture, tools, methodologies. Don’t miss the opportunity! Participation is free.

Amazon Web Services

AWS News

Serial console for troubleshooting network and boot issues are available (only for new generations of EC2 instances based on AWS Nitro System).

Announcing AWS WAF Bot Control for visibility and control over common and pervasive bots, the price started from 10 USD/month.

Support for Gateway Load Balancer now available in more regions. GWLB is a new type of LB introduced by Amazon at the end of 2020. Link to the news. Official documentation about GWLB. Related feature is VPC Ingress Routing, introduced in 2019.

S3 Object Lambda — new integration for configurable transformation S3 objects on the fly is continuing to grow. Now you can add your own code to S3 GET requests to modify and process data as it’s returned to the application. Link to the news + short video about the use case on the same page.

Integration with Agones improved by releasing Amazon GameLift FleetIQ adapter. See here for more details. (May be interesting for DevOps in the game dev industry).

AWS Fault Injection Simulator is generally available from March 2021. The list of supported services includes EC2, ECS, EKS, RDS, also it can inject errors in some internal API calls.

New type of EC2 instance X2gd is available. Memory optimized, based on Graviton2 ARM CPU, 16GB per CPU core.

Lower cost storage classes for EFS are available. About 50% cost reduction for single-AZ. Storage class.

Ethereum’s nodes can be deployed via Amazon Managed Blockchain service. This might be useful for Ethereum-based applications.

General availability of integration AWS Amplify with Flatter (for cross-platform development)

Amazon RDS for MySQL and MariaDB support replication filtering (databases+tables).

Amazon EKS clusters now support user authentication with OIDC compatible identity providers. You may set up now login to EKS via Google/Keycloak/Microsoft,

AWS MKS (Managed Kafka Cluster) news: can scale up and down without downtime and supports recent Apache Kafka version 2.6.1, automatic update is also supported.

Red Hat OpenShift Service on AWS Now GA. Great step ahead for both Red Hat and AWS.

AWS Courses/Certificates

New AWS Cloud Technical Essentials course launched as available for free in Coursera and edX in March 2021.

Amazon S3 Cost Optimization course recently released (60 mins).

Misc

Free Trial: Until June 30th, 2021, all new and existing AWS customers can try the AWS Graviton2 ARM-based t4g.micro (1 GB, 2 vCPU, burstable) instances free for up to 750 hours per month.

Google Cloud Platform

Do you use the Slurm job scheduler? If so, Google announced the newest set of features for Slurm running on Google Cloud. It’s support for Terraform, the HPC VM Image, etc.

Authentication for your data in Cloud Storage — now can make the rules with authentication controls.

From now in the marketplace, you can find Python Package Index (PyPI) dataset. Dataset provides statistics and metadata for all package downloads. More about it here.

Logs are important to troubleshoot and keep applications healthy. Now Google announced a set of open-source JSON dashboards. They can help you quickly analyze logging volumes, logs-based metrics, info about your logging exports across multiple projects. News about it here.

If you use BigQuery- you can speed up small queries with BI Engine. For example, The New York Times uses the SQL interface to BI Engine and speed up their Data Reporting Engine in this way. If you want to know about it more — read it here.

From now you can expand global reach and deliver high performance with the latest release of Google Cloud’s full lifecycle API management platform — Apigee X. More about it here.

Kubernetes

Kubernetes 1.21 released! Memory manager, PDB/CronJob graduation to Stable, deprecation of PSP, and lots more explained in detail by Sysdig. More about the future of PSP can be found here.

Top Kubernetes health metrics you must monitor. Yet simple but worth reading article about essential metrics to gather.

Must-have solutions, practices and tools when it comes to Kubernetes security. Great article, although PodSecurityPolicy is deprecated now.

Argo CD major v2.0-rc1 release : notifications, application sets, image updater, UI changes. Meanwhile, the stable version has already released.

Linkerd 2.10. Now Linkerd control plane is modular and extensible with bare minimum necessary to run: down to 200mb at startup, from ~500mb in Linkerd 2.9!

Flux moves from Sandbox to Incubation. Flux has increased its end user base by 2.75x since joining the CNCF Sandbox and expanded its community by 2x to 4x. It is used by more than 80 organizations in production.

Stateful

Kafka Without ZooKeeper ! It’s finally here: Kafka with Quorum Controller on Raft protocol but slightly modified. The first benchmarks are impressive O_O

Summary numbers for a cluster running 2 million partitions

GA of Redis 6.2. More than 25 new commands and extensions to existing ones, performance and memory optimizations, improved Streams API.

Interesting

2 new subsea cables to connect Singapore, Indonesia, and North America. Called Echo and Bifrost these cables, sponsored by Facebook and Google, will increase overall transpacific capacity by 70 percent.

2 new subsea cables — Echo and Bifrost

Grafana 7.5 released. Improved pie chart panel, better support for Cloudwatch and Loki, and many more!

Hashicorp announced Vault 1.7. This release focuses mostly on improving Vault’s core workflows like Autopilot for Integrated Storage and extending support by adding a new Aerospike storage backend and Snowflake secrets engine.

Slack moves to Envoy. Slack’s journey shifting their WebSockets traffic from HAProxy to Envoy. Great explanation of this choice but they still use Chef).