With the arrival of “cheap” Intel/AMD based servers with 64-bit CPU’s and large amount of memory, I think the DBA managing Oracle on Linux/Windows has to be aware of kernel features that allows better handling of otherwise, vast amount of standard memory pages (4k on Linux and Windows). Lately I was doing some research of the available articles on Metalink about Linux and Windows approaches to handle larger than default (4k) memory pages.

While finding documentation about HugePages feature on Linux was easy, the lack of the same information for Windows equivalent, Large Pages mechanism, was somehow a frustrating experience – I hope Oracle will refresh some of the existing notes that covers NT architecture with the information about Large Pages. The concept behind the HugePages & Large Page are very similar, but as we know, usually the devil is hiding in the details.

Below are references that I used during my research.

Linux – HugePages

Metalink Notes:

• Note:361323.1 “HugePages on Linux: What It Is… and What It Is Not…”
• Note:361468.1 “HugePages on 64-bit Linux”
• Note:401749.1 “Shell Script to Calculate Values Recommended HugePages / HugeTLB Configuration”

Bigpages Linux kernel feature was replaced by HugePages in 2.6 backported to 2.4), for those of you who are “stuck” on older kernel version (such as RHAS 2.1), two Metalink notes:

• Note:275318.1 “The Bigpages Feature on Linux”
• Note:261889.1 “Bigpages vs. Hugetlb on RedHat Linux”

Other references:

• Kevin Closson blog: Oracle11g Automatic Memory Management – Part I. Linux Hugepages Support.

Windows Server 2003 (x86, EMT64)

---

Oracle introduced support for Large Pages in Windows 2003 (SP1) in Oracle 10g R1.

I would expect to find the necessary information in the following two Metalink notes:

• Note:46001.1 “Oracle Database and the Windows NT memory architecture, Technical Bulletin”
• Note:46053.1 “Windows NT Memory Architecture Overview”

…unfortunately not.

The Chapter 7, Tuning Windows to Optimize Oracle Database in Oracle Database Platform Guide 10g Release 2 (10.2) for Microsoft Windows (x64) is much more helpful. This is all you need to setup Oracle 10g/11g to use Large Pages on Windows 2003.

Microsoft official documentation:

• MS Windows Internals, by Mark E. Russinovich and David Salomon, Fourth Edition
• MSDN- Large Page Support (targeted at developers)
• The Memory Manager in Windows Server 2003 and Windows Vista (ppt)

Other sources:

• AMD: Supersizing Java: Large Pages on the Opteron Processor, Part 1
• AMD: Supersizing Java: Large Pages on the Opteron Processor, Part 2
• VMware: Large Page Performance

My personal side notes about “Large Pages” on Windows 2003:

• Which Windows versions has support for Large Pages? I’m not 100% sure I know the answer. According to Wikipedia the Large Pages support is available on Windows 2003 SP1, but not on Windows XP/Vista. I don’t think this is entirely correct, since Windows XP is mentioned in Windows Internals book on page 383 (see reference above)
• don’t use Oracle LOCK_SGA in conjunction with Large Page, instead refer to PRE_PAGE_SGA parameter, if you wish to pre-allocate large pages in memory at instance startup
• user account under which process that wants to use Large Pages is running, needs “Lock Pages in Memory” system privilege. Oracle process by default runs under System account which already has this privilege…[this is false, by default no user account has this privilege, hence adding “System” account (or dedicated user account, if used!) is mandatory]
• not running Oracle service under dedicated user (such as local user oracle) means that service will run under account SYSTEM. If we authorize SYSTEM account to use “Lock Pages in Memory” privilege, we’re giving this privilege to several other services. This brings some security/stability concerns to the table. Shouldn’t we mandatory run Oracle service under some user account, rather than SYSTEM, if we’re going to use Large Pages?
• memory pages allocated via Large Page are pinned in memory and doesn’t swap to the pagefile, make sure you’re not allocating too much memory (for example, over allocating SGA) and thus taking away the memory from the regular VM space that is needed for client sessions (Oracle shadow “processes”)
• each Large Page must consist of contiguous memory, so for example if the size of the page is extremely large it can happen that due to the memory fragmentation, the process can not allocate all requested large pages (the potential for the problem rise with the number of different running processes and the frequency of starting/stopping the process that is using Large Pages)
• once allocated, the memory pinned as Large Page is not released until the process stops
• Windows settings that control Large Page behavior are stored under registry key:
  HKLM\SYSTEM\CurrentControlSet\Control\Session Manager\Memory
• according to “Windows Internals” book, the default large page sizes depends on architecture used:
  x86 = 4MB (2MB if PAE is enabled)
  x64 = 2MB
  IA64 = 16MB
Advertisement

Related