DEVOPS WEEKLY ISSUE #510 - 4th October 2020
Lots of higher level content this week, from service registries to open source compliance policies and dealing with least privilege to attribute based access control.
env0 sponsors Devops Weekly
Check out how to set up Continuous Deployment of a Python3 application into a VMware vSphere environment, all automated with Terraform!
https://www.env0.com/l/setting-up-cd-with-vmware
About env0: use Terraform to let your team manage their own environments in AWS, Azure and Google. Governed by your policies and with complete visibility & cost management.
News
Open source has led to companies sharing undifferentiated code, but work on processes and policies is rarely done in the open. Given lots of companies are subject to the same compliance regimes, this is definitely wasteful. So I’m super interested to see this set of HIPAA policies.
https://policy.datica.com/
https://github.com/catalyzeio/policies
This post looks at how to embrace devops at the same time as maintain least privilege controls in cloud environments.
https://ermetic.com/whats-new/blog/the-three-ways-of-devops/
A look at the concept of emergence in system design and management and how emergent behaviour can be both positive and negative.
https://medium.com/@JonHall_/complex-adaptive-systems-ii-thinking-about-emergence-and-itsm-afec6c3305a3
A post on why moving from a monolithic to microservices architecture is time consuming, due to assumptions and coupling that need to be broken down.
https://medium.com/@kentbeck_7670/monolith-services-theory-practice-617e4546a879
A look at building a service registry from jsonnet files in a Git repository. Interesting ides about the benefits of standard service descriptions and building tools on top of that to make managing infrastructure and applications easier.
https://blog.lawrencejones.dev/service-registry/
Scaling managing permissions is hard. This post looks at one proposed approach, using attribute based access control, and some its problems.
https://onecloudplease.com/blog/security-september-still-early-days-for-abac
Balancing defaults between getting started quickly and scaling is often tricky. A good post on configuring sidecars to limit the memory consumed by large Istio clusters.
https://karlstoney.com/2020/10/03/istio-at-scale-sidecar/
A walkthrough of configuring a high-availability Kubernetes cluster on Azure using Terraform, looking at the various services needed for a production setup, like Calico and Active Directory.
https://codersociety.com/blog/articles/terraform-azure-kubernetes
Tools
Couler provides a DSL for creating and managing workflows on different workflow engines, including Argo Workflows, Tekton Pipelines, and Apache Airflow.
https://github.com/couler-proj/couler
Pinot is a real-time distributed OLAP datastore, built to deliver scalable real-time analytics with low latency. It can ingest from batch data sources (such as Hadoop HDFS, Amazon S3, Azure ADLS, Google Cloud Storage) as well as stream data sources (such as Apache Kafka).
https://pinot.apache.org/
https://github.com/apache/incubator-pinot