Storage replication using DRDB

Storage on a personal computer, or laptop, is always tied to the physical machine. It would be nice to have it synchronised between all of my computers. Linux has this possibility with a Distributed Replicated Block Device (DRDB). This blog explains how to set this up, at least for a lab setting.

Kernel configuration and tools

Make sure that the drdb module is available in the kernel, and emerge sys-cluster/drbd-utils.

Create block devices on two computers

In our case we have two computers, one called 'hp' and another one called 'asrock'. There are no free block devices available on these comptuers, so we create a couple using loop devices. Loop files are not really supported for DRBD, but it works for a lab setup...

On both hp and asrock:

[all] # cd /root
[all] # mkdir nocow
[all] # chattr +C nocow
[all] # cd nocow
[all] # dd if=/dev/zero of=blockfile bs=1M count=5120
[all] # losetup /dev/loop0 blockfile

This creates 5 GB files in /root/nocow called blockfile. The chattr +C command is to prevent copy-on-write in case the file system is BTRFS. The losetup command creates the loop devices '/dev/loop0'.

Configure DRBD and a resource group

On both computers create a file /etc/drbd.d/global_common.conf:

global {
  usage-count no;
}
common {
  net {
    protocol C;
  }
}

Which tells DRBD to not report the version and to use synchronous replication.

On both computers create a file called /etc/drbd.d/r0.res to define resource group r0:

resource r0 {
  on hp {
    device    /dev/drbd0;
    disk      /dev/loop0;
    address   192.168.178.57:7789;
    meta-disk internal;
  }
  on asrock {
    device    /dev/drbd0;
    disk      /dev/loop0;
    address   192.168.178.105:7789;
    meta-disk internal;
  }
}

Initialize the resource group by executing on both computers:

[all] # drbdadm create-md r0

Execute on the primary, in this case on 'hp':

(hp) # drbdadm primary --force r0

Start the DRBD deamon on both computers:

[all] # /etc/init.d/drbd start

The devices now start to synchronize. This can be observed with the following command on one or the other computer::

(hp) # cat /proc/drbd
version: 8.4.11 (api:1/proto:86-101)
srcversion: 3F1488B7CBF933E2A527384
 0: cs:SyncSource ro:Secondary/Secondary ds:UpToDate/Inconsistent C r-----
    ns:166480 nr:0 dw:0 dr:166480 al:0 bm:0 lo:0 pe:5 ua:0 ap:0 ep:1 wo:f oos:5077096
        [>....................] sync'ed:  3.3% (4956/5116)M
        finish: 0:11:19 speed: 7,452 (6,132) K/sec

The cluster is now operational, even while it synchronizes.

Start using the disk, on the primary side

To prove that it all works, the following can be executed:

(hp) # mkfs -t ext4 /dev/drbd0
(hp) # mkdir /mnt/drbd0
(hp) # mount /dev/drbd0 /mnt/drbd0
(hp) # echo "xxxxx" >> /mnt/drbd0/xxxxx
(hp) # ls /mnt/drbd0/xxxxx
lost+found  xxxxx

This puts an ext4 file system on the disk, mounts it, and writes something to it.

Change roles to prove that it works

Unmount the disk on the primary side, demote the resource group to secondary, and upgrade the resource as primary on the other computer and mount it there:

(hp) # umount /mnt/drbd0
(hp) # drbdadm secondary r0
(asrock) # drbdadm primary r0
(asrock) # mount /dev/drbd0 /mnt/drbd0
(asrock) # ls /mnt/drbd0/xxxxx
lost+found  xxxxx

Pages