Devops Diary

Online resizing LVM guest-partitions on OpenNebula/libvirt guests

Today I found out howto resize guest partitions on OpenNebula (or in general with libvirt, which OpenNebula uses underneath for KVM virtualization).

I’m using a LVM storage for virtual machines. So resizing them is pretty easy.

Note: Replace the XX below with your virtual machines ID, as you can find out with e.g. onevm list or virsh list

host ~ $ sudo lvresize -L 50G /dev/volgrp/lv-one-XX
  Extending logical volume lv-one-XX to 50.00 GiB
  Logical volume lv-one-XX successfully resized

Then, you need to tell libvirt about the new size. This can be done with the following command: Note: The —path argument is the device name of the partition used by the guest. You can find it out by using virsh domblklist one-XX

host ~ $ sudo virsh blockresize one-XX --path vdX --size 50G
Block device 'vdX' is resized

Now simply login your guest machine and adapt the filesystem (no reboot required)

guest ~ $ sudo resize2fs /dev/vdX   # For ext filesystems
guest ~ $ sudo xfs_growfs /dev/vdX  # For xfs filesystems

Unfortunately, the new size will not display in the SIZE attribute of the image in OpenNebula (as seen with oneimage show XX). Apparently this is only set when creating the image.

As far as I know, this only works when using the virtio or scsi block drivers. In OpenNebula you can e.g. set DEV_PREFIX="vd" in your image configuration to use virtio.

Many thanks to Humble Chirammal for pointing out virsh blockresize.

Nested if workaround for Nginx to allow a specific ip address access to a disabled site

When doing maintenance on a web application, you probably have a custom 503 site, showing your customers that the servers are currently lying on the operating table.

At the dynamic ridesharing service flinc, we touch a certian file on our reverse proxies (e.g. using capistrano deploy:web:disable) when maintenance begins. Nginx then serves a static “we’ve disabled the site for maintenance” site, instead of the actual content.

But wouldn’t it be nice to test your web application before going live for your customers? It sure would. Unfortunately, this is not as simple as a task as you might think, because you cannot nest if directives in an Nginx location and if is evil.

iptables-ng cookbook for chef

Today, I released iptables-ng, a cookbook to maintain iptables rules on different machines using chef.

But why another cookbook? There are two fairly often used around

Well, I wanted a tool which can do all the following:

  • Configure iptables rules in a consistent and nice way for all distributions
  • Be configured by using LWRPs only
  • Be configured by using node attributes only
  • Respect the way the currently used distribution stores their rules
  • Provide a good-to-read and good-to-maintain way of deploying complex iptables rulesets
  • Provide a way of specifying the order of the iptables rules, in case needed
  • Only run iptables-restore once during a chef run, and only if something was actually changed
  • Support both, ipv6 as well as ipv4
  • Be able to assemble iptables rules from different recipes (and even cookbooks), so you can set your iptables rule where you actually configure the service

ipswitch - migrate IP addresses without downtime

When doing quick maintenance tasks on a server, you can use the following approach to keep your site available:

  • Failover the backnet IP address of the host to another host
  • Use arping to tell the network that this IP was switched
  • Remove the IP from the host that needs maintenance

In case you do not have a full high-availability setup available, you can use ipswitch, a small tool I wrote to assist with this kind of simple failover tasks.

You can install it using

$ gem install ipswitch

apt-get cleanup commands

Just a short post about some useful cleanup commands for Debian and Ubuntu systems. There are (to my knowledge) no build in task solving the following things

  • Remove old kernels (while keeping the currently running and the latest)
  • Purge removed packages (especially after autoremoving unneeded dependencies)

Howto use chef with ssl

By default, the connections between the chef-client and the chef-server are not secured. This is a short post on howto encrypt and verify your connections.

As of chef-11 (unlike chef-10), SSL is enabled by default. But (naturally, as Opscode cannot create trusted certificates for your domain) the certificates are not verified. This essentially means that the connection is not secure at all.

Unless you only use chef in a trusted network, you should invest some time in securing your clients connections.

Chef deploy_revision and Capistrano git_style

One thing that was annoying me for a long time, was that, using Capistrano deployment, you cannot spawn a new vanilla virtual machine, and bring it to a fully up-and-running state with just one Chef command.

make deploy_revision compatible with Capistrano, so deployments can happen with Capistrano, until we’ve decided to fully migrate to Chef, or to stick with the push deployment

About me

Hi, I’m Chris.

I’m the Devops behind the infrastructure of the dynamic ridesharing service flinc, and the financial startup vaamo. I currently work as a freelancer, and you can find me on

You can contact me via email: (sorry, you need Javascript to see my email) [gpg]