DEVOPS WEEKLY ISSUE #277 - 17th April 2016
Several stories of running data storage technologies this week from the venerable MySQL to the cutting edge Kafka. Lots of hard earned wisdom in tales of how people scale particular technologies, thanks to anyone who has ever written up what worked for them (and what didn’t) when using a particular technology.
Sponsor
Built for DevOps – The VictorOps platform provides seamless incident management to improve uptime and make on-call suck less.
Start your free trial today: http://try.victorops.com/DevOps-Weekly
Sponsored
6 Ways DevOps helped PrepSportswear move from Monolith to MicroServices
Learn how PrepSportswear, the largest direct-to-consumer marketplace for personalized fan gear, moved from a team operating a monolithic app using a waterfall development methodology to a modern IT organization using practices from Agile development, DevOps methodologies, and a Service-Oriented Architecture approach. The Impact? Their most successful online holiday shopping season in company history!
http://ow.ly/4mJXxt
News
Education is critical in any discipline, but tricky in a fast moving area like the one many of us work in. This post picks out several areas where education and industry are diverging, including agile, lean, web-scale operations, IT service management and more.
http://www.lean4it.com/2016/04/pivoting-it-education.html
Some good practical implementation details for anyone rolling out continuous deployment. I particularly liked the guiding principles section focused on social considerations.
http://engineering.instagram.com/posts/1125308487520335/continuous-deployment-at-instagram/
Content Security Policy is a useful way of combatting cross-site scripting, clickjacking and other code injection attacks. But you need to write the policy. This post delves into some of the details.
https://diogomonica.com/2015/12/30/deep-into-csp/
I’m pretty happy to see the Open Container Initiative standards effort taking a look at a specification for container images. This post explains what and a little of the why.
https://coreos.com/blog/oci-image-specification.html
A good high-level overview of microservices as an architectural pattern, including useful observations about synchronous HTTP being a bad default communication protocol, service isolation, location transparency and more.
http://jonasboner.com/bla-bla-microservices-bla-bla/
I’ve been predicting an explosion of interest in container build for a while. Using the Nix build and packaging system is an interesting approach, with a number of advantages as laid out in this post.
http://lethalman.blogspot.co.uk/2016/04/cheap-docker-images-with-nix_15.html
The concept of a pipeline as the way for managing software delivery has been around for a while, but how do containers change the steps in your pipeline? This post does a good job of stepping through various options.
http://www.infoq.com/articles/continuous-deployment-containers
MySQL is hugely popular, but how many people really know how to monitor it? This post delves into all the details including key statistics, available metrics, query performance, connections and much more.
https://www.datadoghq.com/blog/monitoring-mysql-performance-metrics/
Another MySQL post, this one an experience report of scaling MySQL on EC2. Details of sharding strategies, migrating data, zero-downtime operations and more.
https://nylas.com/blog/growing-up-with-mysql/
Nice writeup of a logging architecture based on Logstash, Kibana and Elasticsearch, along with Rsyslog, Kafka and taking advantage of some AWS features.
https://www.bashton.com/blog/2016/elk-on-ark/
And another Kafka post, this one a tale of scaling and monitoring Kafka in production. Also includes a discussion of why Kafka makes sense for some architectures, in particular it’s combination of high throughput and persistence.
http://www.confluent.io/blog/how-we-monitor-and-run-kafka-at-scale-signalfx
I’ve not had chance to read the new SRE book just yet but here is a set of comprehensive notes from the book. Might be enough to interest you to buy, or decide it covers ground you are already familiar with.
http://danluu.com/google-sre-book/
A quick overview of why blameless reviews are useful for learning from mistakes in teams.
http://www.slideshare.net/joelchippindale/learning-through-blameless-reviews
Tools
Airflow is a system to programmatically author, schedule and monitor data pipelines. The idea is to describe pipelines as directed acyclic graphs using Python. Airflow also comes with some great looking visualisation tools too.
https://github.com/airbnb/airflow
Habitus is a build tool for Docker images. It builds on Dockerfile, allowing for multi-stage builds as well as simple secrets management and environment variable injection.
http://www.habitus.io/
Napalm, or Network Automation and Programmability Abstraction Layer with Multivendor support, is a suite of tools for driving network devices with code, in this case Python. SUpport is already impressive, with JunOS, EOS, NXOS, IOS and more on the list.
https://github.com/napalm-automation/napalm
Built for DevOps – The VictorOps platform provides seamless incident management to improve uptime and make on-call suck less.
Start your free trial today: http://try.victorops.com/DevOps-Weekly