A major reason enterprise customers see high latency and poorer than expected performance when implementing flash technology is that the flash partition is not aligned on a sector boundary that allows the flash device to access its data efficiently. When creating a Logical Volume (LVM), things can even get more complicated. Proper partition alignment is critical to performance when implementing flash in your enterprise.
An aligned partition is one that starts on a sector number that’s evenly divisible by 4k, or 8k, or a starting sector that is divisible by eight. Aligned input-out (IO) operations will start at sector 8 for 4k alignments, 16 for 8k alignments, and so forth, with sector 2048 for 1M alignments.
If a flash partition is unaligned – its IO operations start at a sector number not divisible by eight – the device will perform two IOs over adjacent blocks instead of one. These extra IOs will degrade performance of the flash device. In our testing, we have seen up to 4x performance gains by properly aligning the flash device.
Out with the old … in with the new
There are many articles, websites, and Linux system administrators best practice documents describing how to create a logical volume (LVM) – an abstraction of a number of flash devices into a single storage volume that enables dynamic volume resizing and makes it easier to replace, re-partition and back up individual devices in Linux. However, most of these practices were developed before the advent of PCIe® flash devices. I have worked with customers who have used these old practices of creating LVMs and some of them are seeing very poor performance when implementing flash in their environments.
My conversations with customers and documents I’ve read on creating LVMs have revealed that the first step in creating a LVM – to create a physical volume (PV) – needs refinement. The reason is the PV create process can use a raw device, a partitioned device, or a mix. I would suggest getting into the habit of aligning all flash devices on a physical sector boundary so that all PVs are aligned. The PV command is typically specified as either “pvcreate /dev/sdX,” which allocates the whole device (non-partitioned) to the PV, or “pvcreate /dev/sdX1,” which uses a partition to create the PV. If the PV is created using a mix of raw devices and partitioned devices, or multiple partitioned devices, is there alignment over all the PVs? Maybe! Maybe not! That’s the problem!
Aligning for higher speed
I recommend a new approach to creating LVMs when using flash technology. My suggestion is to align each of the flash devices on a 1M boundary before creating the PV. Here are the steps to help make sure that you are using boundary-aligned devices when creating a LVM:
- Create a partition on each flash device starting with sector 2048, which will align on a 1M boundary. There are many ways to accomplish this. The method I use is:
- When creating the PV, instead of using “pvcreate /dev/sdX,” use:
- From this point, you can follow the standard steps to finish creating the LVM:
echo “2048,,8e” | sfdisk – uS /dev/sdX – force
pvcreate /dev/sdX1
vgcreate
lvcreate
Implementing flash in the enterprise is a great way to produce low latencies while providing high IOPs and throughput. By following these steps, you will successfully set up an LVM over multiple flash devices that are aligned on a proper boundary to get the best performance.
Leave A Comment