DEVOPS WEEKLY ISSUE #494 - 14th June 2020

Lots of posts this week on common challenges for growing teams and organisations. Test speed, monorepo performance, strategy patterns, data platform migration and more.

News

A look at using Nomad for dynamic scheduling across 200+ edge locations, in particular for running local management tooling.
https://blog.cloudflare.com/how-we-use-hashicorp-nomad/

A look at a recent paper, and some of the maths behind it, for setting effective service level objectives that take into account backend performance and user impact.
https://billduncan.org/the-tail-at-scale/
https://billduncan.org/the-tail-at-scale-revisited/

Cloud Native is more than just technology and tools. This post explores modern patterns for strategy and transformation, including how to make small investments pay off, build psychological safety, reducing the cost of experiments and more. See the webinar from the authors of these patterns below too.
https://www.container-solutions.com/a-pattern-language-for-strategy-chapter-ii

Lots of teams are and organisations are adopting monorepos, but one of the challenges at scale is Git repository performance. This post explores work from one team to speed things up.
https://dropbox.tech/application/speeding-up-a-git-monorepo-at-dropbox-with–200-lines-of-code

Hard won lessons from implementing Istio, pointing out both the good and the bad.
https://zwischenzugs.com/2020/05/05/riding-the-tiger-lessons-learned-implementing-istio/

As organisations grow, the speed of tests invariably becomes a challenge. This post looks at using dynamic analysis to map files to tests, and then to selectively run a subset of the tests when changes are made.
https://engineering.shopify.com/blogs/engineering/spark-joy-by-running-fewer-tests

A well reasoned argument for why one organisation doesn’t use Kubernetes. Looking at operational overhead and security challenges.
https://blog.coinbase.com/container-technologies-at-coinbase-d4ae118dcb6c

A case study looking at a large data platform migration, including moving from Hive to Spark, and building a developer-friendly system.
https://databricks.com/blog/2020/06/10/accelerating-developers-by-ditching-the-data-center.html

This post explores how a single SQL statement can drastically improve performance. The basics of SQL are well understood but a good reminder how powerful databases like postgreSQL are.
https://jlongster.com/how-one-word-postgresql-performance

Events

A free webinar with Jamie Dobson to introduce a pattern language for strategy and execution. This language was developed after analysing five years of studying successful (and unsuccessful) Cloud Native transformations. Perfect for executives, leaders and engineers.
https://info.container-solutions.com/strategy-webinar-with-jamie-dobson?utm_source=devopsns&utm_medium=newsletter&utm_campaign=patterns

Jobs

King is looking for new members for the infrastructure engineering teams to help develop, manage and expand our software based networking setup across datacenters and (Google) cloud. Please take a look at the open role for networking engineer. We’re also still looking for both database and streaming data engineers, if that is more your style.
https://careers.king.com/jobs/job/14780-site-reliability-engineer-technology/
https://careers.king.com/jobs/job/15353-database-site-reliability-engineer-technology/
https://careers.king.com/jobs/job/15270-network-site-reliability-engineer-technology/

Tools

KIP is a virtual kubelet implementation which allows for launching pods in ephemeral EC2 or GCP instances, including spot instances in EC2.
https://github.com/elotl/kip
https://itnext.io/cloud-bursting-with-virtual-kubelet-and-kip-kloud-instance-provider-4b86a479ce38