The reason for this is that the implementation of the C runtime (CRT) library’s memory management routines on HP-UX does not shrink the heap of a process. This effectively means that if you were to allocate a 512MB block of memory and then release that memory again, the operating system would still report your process as using 512MB of memory. Because of this, long-lived processes (such as services) that may use large amounts of memory to do processing at some point eventually dominate the memory usage of the system, even though they are actually using only a small part of this expanded heap on average. More specifically, the OS is basically reporting that there are 20 instances of a single service each using 1GB of RAM! Shared memory, on the other hand, obviously uses memory on the system, but since SHM isn’t tied to any specific process, it won’t cause the participating processes to display a huge amount of memory usage.

The other alternative is of course to change the way the service does processing, but this involves code changes in both the front and back-end parts of the software, so a shared memory allocator would be the path of least resistance in this case.

“But wait!”, you say, “Wouldn’t that involve changing the back-end code to make use of the new allocator anyway?” The answer to this is that, yes, it could, but there’s an easier option. The dynamic loader on UNIX uses a neat little environment variable called LD_PRELOAD which causes the shared libraries specified in it to be loaded before any other shared libraries. This allows you to easily override the CRT’s memory management routines, since yours get loaded first. This means that the amount of code that needs to be changed go from every memory allocation call to, effectively, none at all.

So we can have the manager allocate a big block of shared memory, and have the services communicate their requests for memory to the manager using UNIX domain sockets. The manager then attempts to service the request by reserving a block of shared memory for that process and passing the address back to the requesting process. The service can then map the shared memory segment into it’s own address space, use the memory for whatever it needs it for, and then tells the manager to release the memory again. Easy off bang!

Not quite, though.

HP-UX has support for both 32-bit and 64-bit operation. In 32-bit mode, the maximum size of a shared memory segment is 1GB, while in 64-bit mode, a shared memory segment may be up to 4TB in size. The systems we’re working on run a 64-bit kernel, so the maximum size of a segment isn’t the problem. The problem is that the processes themselves are 32-bit executables, and to understand why, I’ll need to talk about how HP-UX defines the memory space for a 32-bit executable.

32-bit mode implies a maximum addressable range of 2³² bits, or 4GB of memory. In HP-UX, this 4GB virtual address space is divided into four quadrants of 1GB each. The first two quadrants are used for the process’ text and data segments. The third and fourth quadrants are used as a global shared memory space, with the final 250MB of the fourth quadrant being reserved for system I/O (my guess is that this is used for buffers when doing I/O operations). The global adjective means that any shared memory in the system will be (potentially) visible in any other process. Since shared memory does utilise access control when a process requests to map a segment into its address space, there aren’t any security concerns regarding that, though. What this does mean, though, is that the total amount of shared memory available in 32-bit processes is limited to 1.75GB. And by “32-bit processes” I mean all 32-bit processes currently running in the system.

Luckily, HP-UX has a feature known as memory windows. To use memory windows, the process to be executed is run using a wrapper called setmemwindow, which will execute the given command in a given memory window. The effect of this is that instead of the process having the normal 1GB global Quadrant 3, it receives a separate 1GB Quadrant 3 which is then shared by all processes belonging to the same memory window.

The idea I had is to use this feature by partitioning the services that require the shared memory pool into groups, and having them and their respective shared memory manager all belong to the same memory window. This frees up the global Q3 for other processes running on the system that might also use SHM, and allows multiple shared memory managers to execute in parallel with their own set of managed services.

The current status of my investigation, barring finding out about the above technical details, is that I’ve completed a basic manager process capable of reserving and releasing blocks of the shared memory segment to requesting processes, as well as a small test program that communicates requests to the manager. The next step is to write two programs that will perform random memory allocations, one using the SHM manager and another using the standard CRT, to get an idea of what the performance impact of the additional management layer will be.

Hopefully someone will find this useful, since I had to do quite a bit of Google-trawling to find out about all of this. If anyone wants a few of the references I used, feel free to leave a comment, and I’ll see if I can set you up.