Jeff Stonacek, Principal Architect

Introduction

If you are reading this post then you probably already know about HugePages in Linux. If not, then read this previous blog post, which describes HugePages.

In newer versions of Linux, specifically RHEL/OEL 6.x and SLES 11, transparent HugePages were introduced in an attempt to improve memory management. Transparent HugePages allow the kernel to dynamically allocate HugePages as needed. This differs from traditional HugePages that needed to be allocated in a contiguous block, typically at boot time. While dynamic allocation is a valid practice on systems that have an abundance of memory and are allocating HugePages in small increments, typical Oracle database environments do not behave that way. Oracle database servers typically have large memory requirements and allocate a HugePages at a demanding rate.

Leaving transparent HugePages enabled on an Oracle database server can cause performance issues, or worse, stability problems. Therefore, it is recommended to disable transparent HugePages for Oracle database servers.

Editing the kernel line in the grub.conf can file disable transparent HugePages. Below is an example:

title Oracle Linux Server Unbreakable Enterprise Kernel (3.8.13-
16.2.1.el6uek.x86_64)
        root (hd0,0)
        kernel /vmlinuz-3.8.13-16.2.1.el6uek.x86_64 ro root=/dev/mapper/rootvg-
rootlv transparent_hugepage=never
        rd_NO_LUKS rd_LVM_LV=rootvg/rootlv LANG=en_US.UTF-8 rd_NO_MD 
rd_LVM_LV=rootvg/swaplv
        SYSFONT=latarcyrheb-sun16 KEYBOARDTYPE=pc KEYTABLE=us rd_NO_DM rhgb quiet numa=off
initrd /initramfs-3.8.13-16.2.1.el6uek.x86_64.img

The problem with this method is that the change has to be made every time the kernel is upgraded.

Tuned

A better solution is to disable transparent HugePages with Tuned. Tuned is a daemon that starts at boot time and modifies the system based on a configuration file. Tuned continually monitors the system and makes modifications dynamically. If not already done, install Tuned as shown below.

yum install tuned
Loaded plugins: product-id, subscription-manager
This system is not registered to Red Hat Subscription Management. You can use 
subscription-manager to register.
dvd 
| 3.9 kB   00:00 ...
Setting up Install Process
Resolving Dependencies
--> Running transaction check
---> Package tuned.noarch 0:0.2.19-13.el6 will be installed
--> Finished Dependency Resolution

Dependencies Resolved

==============================================================================================================================
Package                       Arch                    Version 
Repository                    Size
==============================================================================================================================
Installing:
tuned                         noarch                   0.2.19-13.el6 
dvd                           94 k

Transaction Summary
==============================================================================================================================
Install 1 Package(s)

Total download size: 94 k
Installed size: 222 k
Is this ok [y/N]: y
Downloading Packages:
Running rpm_check_debug
Running Transaction Test
Transaction Test Succeeded
Running Transaction
  Installing : tuned-0.2.19-13.el6.noarch 
1/1
  Verifying : tuned-0.2.19-13.el6.noarch 
1/1

Installed:
  tuned.noarch 0:0.2.19-13.el6

Complete!

After installing Tuned, modify the appropriate Tuned profile to disable THP. To do this we will copy an existing profile and modify it.

cd /etc/tune-profiles
cp -r enterprise-storage enterprise-storage-no-thp
cd enterprise-storage-no-thp

Next, edit the ktune.sh script and set transparent HugePages to never.

vi ktune.sh

#!/bin/sh

. /etc/tune-profiles/functions

start() {
        set_cpu_governor performance
        set_transparent_hugepages never
        disable_disk_barriers
        multiply_disk_readahead 4

        return 0
}

stop() {
       restore_cpu_governor
       restore_transparent_hugepages
       enable_disk_barriers
       restore_disk_readahead

       return 0
}

process $@

Finally, enable the Tuned profile.

tuned-adm profile enterprise-storage-no-thp

Switching to profile 'enterprise-storage-no-thp'
Applying deadline elevator: dm-0 dm-1 sda                     [ OK ]
Applying ktune sysctl settings:
/etc/ktune.d/tunedadm.conf:                                   [ OK ]
Calling '/etc/ktune.d/tunedadm.sh start':                     [ OK ]
Applying sysctl settings from /etc/sysctl.conf
Starting tuned:                                               [ OK ]

# chkconfig --list | grep tuned
tuned           0:off 1:off 2:off 3:on 4:on 5:on 6:off

# cat /sys/kernel/mm/transparent_hugepage/enabled
always madvise [never]

Conclusion

In this blog post we discussed how to disable transparent HugePages using Tuned. The benefit of using Tuned is that settings are persistent after a kernel upgrade.