Chris Peck BSc (Hons) - www.chrispeck.co.uk

Home About Website Design Work University Final Year Project Education Employment Life Links News and Updates Contact Print

Internet links

Microsoft Microsoft MSDN David R. Overbeck’s Memory management supersite Windows Network & .net Magazine Windows IT Library

08210 Networked Computer Systems

Coursework: Paging within Windows NT

This is a report into the Memory Management functions for the Microsoft Windows NT operating system (hereon in referred to as Windows NT), and how the various memory management functions provide the various functionality found within Windows NT.

All operating systems contain a number of managerial functions, and this is no different with Windows NT, which uses its Memory Management functions to provide efficient functionality for users and programmers alike.

The memory management in Windows NT is based upon the architecture of the Intel 80386 processor (whose memory management architecture is mimicked by the later Intel 80486 and Pentium series). The Intel 80386 processor was a thirty-two bit processor, and hence has an address bus of thirty-two bits. These 32-bits can be used by the operating system to create virtual memory.

Windows NT can manipulate this to create four gigabytes of virtual memory - i.e. four gigabytes of physical (real) memory can be referenced, or 2³² bytes. This allows applications to recognise up to four gigabytes of memory, determined from the 32-bit pointers, which have a range of 00000000 to FFFFFFFF. The upper portion of this is reserved for use by the system, while the remaining two gigabytes is designated for application use. As a result, each application has its own private address space from which it may access the lower portion.

This large virtual memory can, therefore, be provided by:
1. Moving data from the physical RAM to the Hard Drive
2. Referencing physical memory addresses to virtual memory addresses
And can be provided by the virtual memory management scheme using the 32-bit linear address.

Not all memory is allocated to the virtual, page-referenced memory, and can be split into three parts:
1. Non-paged pool: Programs with a high priority, i.e. kernel and system components. This is typically only two to three megabytes
2. Paged pool: The most frequent accessed memory
3. cached memory

Windows NT memory management also allows for memory mapping, which permits multiple processes, even device drivers and user applications, to share the same physical memory. This results in very fast data transfers between cooperating processes or between a driver and an application. Memory mapping can be used to dramatically enhance real-time performance.

By using these techniques, programs can be designed and written to exploit large amounts of memory without being concerned with the amount or type of memory available.

The memory can be accessed using Shared Memory, which uses the memory mapped mechanism to allocate memory to the relevant process by making them named shared member objects. Processes can call these objects by using a handle. Any procedure can call this, as long as it has security object permissions.

However, demand paging is preferred as it results in less of a hindrance on performance than pure memory swap. This allows programs that need the most memory and processor attention receive it.

Paging moves only those parts of memory which need to be moved, managing memory divided into equal-sized 4kB (4096 byte) pages. The virtual memory manager uses paging to move the required data between the physical memory and the hard drive. When the requested data is not available in physical RAM, the desired page is swapped by the virtual memory manager.

Fig1: Diagram of a Page Table Directory (source: Microsoft MSDN)

By using the 32-bit linear address, the required page can be located. The upper 10 bits are used as a page directory index, to create a logical mapping for pages. A specific register in the processor, register CR3 (or the Page Directory Base Register (PDBR)), is always set to point to the current page table directory, which itself is a 4096 bytes page. The page table directory is an array of pointers to page tables, of which there are 1024 entries of 4 bytes each. Each page table contains 1024 page table entry (PTE) entries of 4 bytes (or 32 bits) each. See Fig 1 for a diagram of the page directory.

Fig 2: Diagram of uses of the 32 bits of a Page Table Entry (PTE) (source: Microsoft MSDN)

Of the 32bits for each PTE, twenty bits (bits 26 to 7) refer to the physical location to which the page is referencing, with the remaining twelve bits used for page attributes and security. These attributes include access permissions to denote whether a page is read-write or read-only, a supervisor bit which denotes whether the page can be accessed from the user-mode code or from the kernel-mode code only, i.e. whether the user can access the code or just the operating system. A diagram of the 32bits of a PTE can be found at Fig 2. The accessed bit and the dirty bit define the current status of the page, with the accessed bit set by the processor when the page is accessed and the dirty bit set when the page is written to. The transition bit is used to set when the page is being placed back into physical memory. Several bits are also used by the operating system to provide copy-on-write protection and to set invalid pages. The final bit is the valid bit, set when the page is a valid page to a reference and is in physical memory. If this is not set, the page will be invalid and return a page fault exception.

When a page fault exception occurs, the system will attempt to load the page from the hard drive into physical memory. This is known as the pager process, which verifies the address is valid to access (i.e. that there has been some memory allocated for this process), and that it can be loaded. Once loaded, control is returned to the operating system and the instruction that caused the page fault is restarted.

As this is the least efficient use of the paging function, it makes sense to try to keep as many relevant pages in memory, while reducing the number of non-relevant pages in memory.

In order to do this, the pages are separated into eight different categories. These categories are:

Active: pages loaded into memory which are valid pages, i.e. have a relevant page table entry in the page table directory.
Transition: used for pages where an I/O operation, e.g. reading from or committing to a disk, is being performed. This is a temporary state as while the operation is being performed, the page is neither active, nor on a paging list.
Modified: those pages which have recently been active and have been modified but have not yet been committed to disk
Modified no write: used for transaction logging by the operating system
Standby: holds pages which were previously modified, as if the pages were set to free, if they were required again by the operating system, a page fault would occur.
Free: pages which have are no longer active, and so can be overwritten, but have yet to be cleared.
Zeroed: Free pages that have been cleared
Bad: pages which have caused hardware errors or general parity issues, and as such cannot be used for any process. These never become free or zeroed.

The pages and their relevant states are kept track of by the Paging Database in Windows NT. This is managed by the virtual memory manager, which reduces the number of allocated pages set to free in order for the pages to be re-allocated, and also ensures aging pages can be swapped properly. Windows NT uses a first in, first out replacement algorithm to define the oldest pages so they can be cleared in order for new pages to be created.

The virtual memory manager also controls modified pages, the writing of modified pages to disk, the swapping of pages and the clearing of pages marked free so that they can be over-written. This is achieved by Windows NT by using a number of “threads”, each of which has a different task with a differing priority.

Through the use of the Windows NT Paging Database and the Virtual Memory Manager, Windows NT is able to keep track of the page table entries and their current state, in order to ensure pages are used efficiently, and the required pages are available when required.

Through the use of paging, Windows NT is able to run applications and programs more efficiently, by using virtual memory. With a large amount of virtual memory available for the operating system, programs can be written to exploit large amounts of memory without being concerned with the type of memory or the amount available. These memory functions combined create an efficient, dependable operating system.

Bibliography

The Virtual-Memory Manager in Windows NT (Randy Kath)
Inside Windows NT, 2nd Edition (David Solomen)
Inside Windows NT (Helen Custer)
Microsoft (http://www.microsoft.com)
Microsoft MSDN (http://msdn.microsoft.com)
David R. Overbeck’s Memory management supersite (http://tyrant112.tripod.com/overbeck/Mem.htm)
Windows Network & .net Magazine (http://www.winntmag.com)
Windows IT Library (http://www.windowsitlibrary.com)