My first blog in this series, How to maximize performance of PCIe flash for enterprise applications running on Linux, describes the steps for aligning PCIe® flash devices. This blog covers the next stage of setting up the PCIe flash device when using the Linux® operating system: creating a RAW device or a file system.
At this point, one or more PCIe flash cards have been partitioned on a sector boundary. Depending on their use, these partitioned devices are either set up as a single RAW device or as part of a logical volume or RAID array.
Next step is to determine how these devices will be used. Most administrators will create file systems on these partitions. Some Oracle administrators will use them as RAW devices and assign them to Automatic Storage Management (ASM). Still others, those looking for the best performance possible from the device will stick to a RAW device. For many years, the recommendation was not to use RAW devices because the complexity of managing them outweighed their small potential gains in performance.
ASM uses RAW devices but makes administration of these devices much easier. More on ASM in Part 3 of this series.
Building a file system
Next is to build a file system on the RAW device, LVM or RAID. But first we need to determine the best type of file system to use. There are many to choose from including:
- EXT-2
- EXT-3
- EXT-4
- XFS
- BTRFS
- ZFS
To keep this brief, I will only go over EXT-4. This type of file system is the most current and provides the latest enhancements for increasing capacity, disabling journaling and many other capabilities, though XFS can be a higher performance alternative.
To create an EXT-4 file system, use this command:
mkfs.ext4 /dev/sdX
You can now turn off or on certain features of the EXT-4 file system by using “tune2fs. Here are a couple of examples of using tune2fs:
- To list all file system features for /dev/sdX1, use this tune2fs command:
tune2fs –l /dev/sdX1 | grep ‘Filesystem features’
- To disable journaling on /dev/sdX1, use this tune2fs command:
tune2fs -O ^has_journal /dev/sdX1
Mounting the file system
The next step is to mount the file system and assign the owner:group to the mount point. There are also many tuning options that can be added to the mount command when using PCIe flash cards. The mount options I use are:
- NOATIME
- NODIRATIME
- MAX_BATCH_TIME=0
- NOBARRIER
- DISCARD
The mount command for /dev/sda1 to /u01 would be:
mount –o noatime,nodiratime,max_batch_time=0, nobarrier,discard /dev/sda1 /u01
To make these mount points persistent over reboots, add them to the mount entries in ‘/etc/fstab’. Finally, you need to give a user rights for reading and writing to this mount point, and to assign ownership to /u01 – for example, assigning ownership of /u01 to the oracle userid and to the dba group. To do this, use the “chown” command:
chown oracle:dba /u01
The PCIe flash device is now ready to be used.
This post is part of a four-part series:
Part 1 describes how to align the PCIe devices.
Part 2 describes how to create RAW devices or file systems.
Part 3 describes how to use Oracle ASM when deploying PCIe flash cards.
Part 4 describes how to persist assignment to dynamically changing Nytro PCIe devices.
Leave A Comment