System V Shared Memory in Linux

Shared Memory

Shared memory is one of the three inter process communication (IPC) mechanisms available under Linux and other Unix-like systems. The other two IPC mechanisms are the message queues and semaphores. In case of shared memory, a shared memory segment is created by the kernel and mapped to the data segment of the address space of a requesting process. A process can use the shared memory just like any other global variable in its address space.

In the inter process communication mechanisms like the pipes, fifos and message queues, the work involved in sending data from one process to another is like this. Process P1 makes a system call to send data to Process P2. The message is copied from the address space of the first process to the kernel space during the system call for sending the message. Then, the second process makes a system call to receive the message. The message is copied from the kernel space to the address space of the second process. The shared memory mechanism does away with this copying overhead. The first process simply writes data into the shared memory segment. As soon as it is written, the data becomes available to the second process. Shared memory is the fastest mechanism for inter-process communication.

Like message queues and semaphores, shared memory also comes in two flavors, the traditional System V shared memory and the newer POSIX shared memory. In this post, we will look at the System V shared memory calls. Example programs for a server and client processes communicating via the System V shared memory are given near the end of the post.

To use a System V IPC mechanism, we need a System V IPC key. The ftok function, which does the job, is

#include <sys/types.h>
#include <sys/ipc.h>

key_t ftok (const char *pathname, int proj_id);

The pathname is an existing file in the filesystem. The last eight bits of proj_id are used; these must not be zero. A System V IPC key is returned.

#include <sys/ipc.h>
#include <sys/shm.h>

int shmget (key_t key, size_t size, int shmflg);

As the name suggests, shmget gets you a shared memory segment associated with the given key. The key is obtained earlier using the ftok function. If there is no existing shared memory segment corresponding to the given key and IPC_CREAT flag is specified in shmflg, a new shared memory segment is created. Also, the key value could be IPC_PRIVATE, in which case a new shared memory segment is created. size specifies the size of the shared memory segment to be created; it is rounded up to a multiple of PAGE_SIZE. If shmflg has IPC_CREAT | IPC_EXCL specified and a shared memory segment for the given key exists, shmget fails and returns -1, with errno set to EEXIST. The last nine bits of shmflg specify the permissions granted to owner, group and others. The execute permissions are not used. If shmget succeeds, a shared memory identifier is returned. On error, -1 is returned and errno is set to the relevant error.

#include <sys/types.h>
#include <sys/shm.h>

void *shmat (int shmid, const void *shmaddr, int shmflg);

With shmat, the calling process can attach the shared memory segment identified by shmid. The process can specify the address at which the shared memory segment should be attached with shmaddr. However, in most cases, we do not care at what address system attaches the shared memory segment and shmaddr can conveniently be specified as NULL. shmflg specifies the flags for attaching the shared memory segment. If shmaddr is not null and SHM_RND is specified in shmflg, the shared memory segment is attached at address rounded down to the nearest multiple of SHMLBA, where SHMLBA stands for Segment low boundary address. The idea is to attach at an address which is a multiple of SHMLBA. On most Linux systems, SHMLBA is the same as PAGE_SIZE. Another flag is SHM_RDONLY, which means the shared memory segment should be attached with read only access.

On success, shmat returns pointer to the attached shared memory segment. On error,
(void *) -1 is returned, with errno set to the cause of the error.

#include <sys/types.h>
#include <sys/shm.h>

int shmdt (const void *shmaddr);

shmdt detaches a shared memory segment from the address space of the calling process. shmaddr is the address at which the shared memory segment was attached, being the value returned by an earlier shmat call. On success, shmdt returns 0. On error, shmdt returns -1 and errno is set to indicate the reason of error.

#include <sys/ipc.h>
#include <sys/shm.h>

int shmctl (int shmid, int cmd, struct shmid_ds *buf);

The shmctl call is for control operations of a System V shared memory segment identified by the shmid identifier, returned by an earlier shmget call. The data structure for shared memory segments in the kernel is,

struct shmid_ds { 
    struct ipc_perm shm_perm;    /* Ownership and permissions */
    size_t          shm_segsz;   /* Size of segment (bytes) */ 
    time_t          shm_atime;   /* Last attach time */
    time_t          shm_dtime;   /* Last detach time */
    time_t          shm_ctime;   /* Last change time */
    pid_t           shm_cpid;    /* PID of creator */ 
    pid_t           shm_lpid;    /* PID of last shmat(2)/shmdt(2) */
    shmatt_t        shm_nattch;  /* No. of current attaches */
    ...        
};         

Where. the struct ipc_perm is,

struct ipc_perm { 
    key_t          __key;    /* Key supplied to shmget(2) */
    uid_t          uid;      /* Effective UID of owner */
    gid_t          gid;      /* Effective GID of owner */
    uid_t          cuid;     /* Effective UID of creator */ 
    gid_t          cgid;     /* Effective GID of creator */ 
    unsigned short mode;     /* Permissions + SHM_DEST and
                                SHM_LOCKED flags */
    unsigned short __seq;    /* Sequence number */
};         

The cmd parameter in shmctl specifies the command. The important commands are IPC_STAT, IPC_SET and IPC_RMID. The IPC_STAT command copies the data in the kernel data structure shmid_ds for the shared memory into the location pointed by the parameter buf. With the IPC_SET command, we can set some of the fields in the shmid_ds structure in the kernel for the shared memory segment. The fields that can be modified are shm_perm.uid, shm_perm.gid and the least significant 9 bits of shm_perm.mode. The command, IPC_RMID, marks a shared memory segment for removal from the system. The shared memory segment is actually removed after the last process detaches it from its address space.

The example has a server process called spooler which prints strings received from clients. The spooler is a kind of consumer process which consumes strings. The spooler creates a shared memory segment and attaches it to its address space. The client processes attach the shared memory segment and write strings in it. The client processes are producers; they produce strings. The spooler takes strings from the shared memory segment and writes the strings on the terminal. The spooler prints strings in the order they are written in the shared memory segment by the clients. So, effectively, there are n concurrent processes producing strings at random times and the spooler prints them nicely in the chronological order. Both the client and server programs are in C. The software architecture is like this.

The spooler code is,

/*
 *
 *       spooler.c: Print strings in the shared memory segment
 *                  (Consumer process)
 */

#include <stdio.h>
#include <stdlib.h>
#include <sys/types.h>
#include <sys/ipc.h>
#include <sys/shm.h> 
#include <sys/sem.h>

// Buffer data structures
#define MAX_BUFFERS 10
#define SHARED_MEMORY_KEY "/tmp/shared_memory_key"
#define SEM_MUTEX_KEY "/tmp/sem-mutex-key"
#define SEM_BUFFER_COUNT_KEY "/tmp/sem-buffer-count-key"
#define SEM_SPOOL_SIGNAL_KEY "/tmp/sem-spool-signal-key"
#define PROJECT_ID 'K'

struct shared_memory {
    char buf [MAX_BUFFERS] [256];
    int buffer_index;
    int buffer_print_index;
};

void error (char *msg);

int main (int argc, char **argv)
{
    key_t s_key;
    union semun  
    {
        int val;
        struct semid_ds *buf;
        ushort array [1];
    } sem_attr;
    int shm_id;
    struct shared_memory *shared_mem_ptr;
    int mutex_sem, buffer_count_sem, spool_signal_sem;
    
    printf ("spooler: hello world\n");
    //  mutual exclusion semaphore
    /* generate a key for creating semaphore  */
    if ((s_key = ftok (SEM_MUTEX_KEY, PROJECT_ID)) == -1)
        error ("ftok");
    if ((mutex_sem = semget (s_key, 1, 0660 | IPC_CREAT)) == -1)
        error ("semget");
    // Giving initial value. 
    sem_attr.val = 0;        // locked, till we finish initialization
    if (semctl (mutex_sem, 0, SETVAL, sem_attr) == -1)
        error ("semctl SETVAL");
    
    // Get shared memory 
    if ((s_key = ftok (SHARED_MEMORY_KEY, PROJECT_ID)) == -1)
        error ("ftok");    
    if ((shm_id = shmget (s_key, sizeof (struct shared_memory), 
         0660 | IPC_CREAT)) == -1)
        error ("shmget");
    if ((shared_mem_ptr = (struct shared_memory *) shmat (shm_id, NULL, 0)) 
         == (struct shared_memory *) -1) 
        error ("shmat");
    // Initialize the shared memory
    shared_mem_ptr -> buffer_index = shared_mem_ptr -> buffer_print_index = 0;

    // counting semaphore, indicating the number of available buffers.
    /* generate a key for creating semaphore  */
    if ((s_key = ftok (SEM_BUFFER_COUNT_KEY, PROJECT_ID)) == -1)
        error ("ftok");
    if ((buffer_count_sem = semget (s_key, 1, 0660 | IPC_CREAT)) == -1)
        error ("semget");
    // giving initial values
    sem_attr.val = MAX_BUFFERS;    // MAX_BUFFERS are available
    if (semctl (buffer_count_sem, 0, SETVAL, sem_attr) == -1)
        error ("semctl SETVAL");

    // counting semaphore, indicating the number of strings to be printed.
    /* generate a key for creating semaphore  */
    if ((s_key = ftok (SEM_SPOOL_SIGNAL_KEY, PROJECT_ID)) == -1)
        error ("ftok");
    if ((spool_signal_sem = semget (s_key, 1, 0660 | IPC_CREAT)) == -1)
        error ("semget");
    // giving initial values
    sem_attr.val = 0;    // 0 strings are available initially.
    if (semctl (spool_signal_sem, 0, SETVAL, sem_attr) == -1)
        error ("semctl SETVAL");
    
    // Initialization complete; now we can set mutex semaphore as 1 to 
    // indicate shared memory segment is available
    sem_attr.val = 1;
    if (semctl (mutex_sem, 0, SETVAL, sem_attr) == -1)
        error ("semctl SETVAL"); 

    struct sembuf asem [1];

    asem [0].sem_num = 0;
    asem [0].sem_op = 0;
    asem [0].sem_flg = 0;

    while (1) {  // forever
        // Is there a string to print? P (spool_signal_sem);
        asem [0].sem_op = -1;
        if (semop (spool_signal_sem, asem, 1) == -1)
	    perror ("semop: spool_signal_sem");
    
        printf ("%s", shared_mem_ptr -> buf [shared_mem_ptr -> buffer_print_index]);

        /* Since there is only one process (the spooler) using the 
           buffer_print_index, mutex semaphore is not necessary */
        (shared_mem_ptr -> buffer_print_index)++;
        if (shared_mem_ptr -> buffer_print_index == MAX_BUFFERS)
           shared_mem_ptr -> buffer_print_index = 0;

        /* Contents of one buffer has been printed.
           One more buffer is available for use by producers.
           Release buffer: V (buffer_count_sem);  */
        asem [0].sem_op = 1;
        if (semop (buffer_count_sem, asem, 1) == -1)
	    perror ("semop: buffer_count_sem");
    }
}

// Print system error and exit
void error (char *msg)
{
    perror (msg);
    exit (1);
}

And, the client code is,

/*
 *
 *       client.c: Write strings for printing in the shared memory segment
 *                 (Producer process)
 */

#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <sys/types.h>
#include <unistd.h>
#include <sys/ipc.h>
#include <sys/shm.h> 
#include <sys/sem.h>

// Buffer data structures
#define MAX_BUFFERS 10
#define SHARED_MEMORY_KEY "/tmp/shared_memory_key"
#define SEM_MUTEX_KEY "/tmp/sem-mutex-key"
#define SEM_BUFFER_COUNT_KEY "/tmp/sem-buffer-count-key"
#define SEM_SPOOL_SIGNAL_KEY "/tmp/sem-spool-signal-key"
#define PROJECT_ID 'K'

struct shared_memory {
    char buf [MAX_BUFFERS] [256];
    int buffer_index;
    int buffer_print_index;
};

void error (char *msg);

int main (int argc, char **argv)
{
    key_t s_key;
    union semun  
    {
        int val;
        struct semid_ds *buf;
        ushort array [1];
    } sem_attr;
    int shm_id;
    struct shared_memory *shared_mem_ptr;
    int mutex_sem, buffer_count_sem, spool_signal_sem;
    
    //  mutual exclusion semaphore
    /* generate a key for creating semaphore  */
    if ((s_key = ftok (SEM_MUTEX_KEY, PROJECT_ID)) == -1)
        error ("ftok");
    if ((mutex_sem = semget (s_key, 1, 0)) == -1)
        error ("semget");
    
    // Get shared memory 
    if ((s_key = ftok (SHARED_MEMORY_KEY, PROJECT_ID)) == -1)
        error ("ftok");    
    if ((shm_id = shmget (s_key, sizeof (struct shared_memory), 0)) == -1)
        error ("shmget");
    if ((shared_mem_ptr = (struct shared_memory *) shmat (shm_id, NULL, 0)) 
         == (struct shared_memory *) -1) 
        error ("shmat");

    // counting semaphore, indicating the number of available buffers.
    /* generate a key for creating semaphore  */
    if ((s_key = ftok (SEM_BUFFER_COUNT_KEY, PROJECT_ID)) == -1)
        error ("ftok");
    if ((buffer_count_sem = semget (s_key, 1, 0)) == -1)
        error ("semget");

    // counting semaphore, indicating the number of strings to be printed.
    /* generate a key for creating semaphore  */
    if ((s_key = ftok (SEM_SPOOL_SIGNAL_KEY, PROJECT_ID)) == -1)
        error ("ftok");
    if ((spool_signal_sem = semget (s_key, 1, 0)) == -1)
        error ("semget");
    
    struct sembuf asem [1];

    asem [0].sem_num = 0;
    asem [0].sem_op = 0;
    asem [0].sem_flg = 0;

    char buf [200];

    printf ("Please type a message: ");

    while (fgets (buf, 198, stdin)) {
        // remove newline from string
        int length = strlen (buf);
        if (buf [length - 1] == '\n')
           buf [length - 1] = '\0';

        // get a buffer: P (buffer_count_sem);
        asem [0].sem_op = -1;
        if (semop (buffer_count_sem, asem, 1) == -1)
	    error ("semop: buffer_count_sem");
    
        /* There might be multiple producers. We must ensure that 
            only one producer uses buffer_index at a time.  */
        // P (mutex_sem);
        asem [0].sem_op = -1;
        if (semop (mutex_sem, asem, 1) == -1)
	    error ("semop: mutex_sem");

	    // Critical section
            sprintf (shared_mem_ptr -> buf [shared_mem_ptr -> buffer_index], "(%d): %s\n", getpid (), buf);
            (shared_mem_ptr -> buffer_index)++;
            if (shared_mem_ptr -> buffer_index == MAX_BUFFERS)
                shared_mem_ptr -> buffer_index = 0;

        // Release mutex sem: V (mutex_sem)
        asem [0].sem_op = 1;
        if (semop (mutex_sem, asem, 1) == -1)
	    error ("semop: mutex_sem");
    
	// Tell spooler that there is a string to print: V (spool_signal_sem);
        asem [0].sem_op = 1;
        if (semop (spool_signal_sem, asem, 1) == -1)
	    error ("semop: spool_signal_sem");

        printf ("Please type a message: ");
    }
 
    if (shmdt ((void *) shared_mem_ptr) == -1)
        error ("shmdt");
    exit (0);
}

// Print system error and exit
void error (char *msg)
{
    perror (msg);
    exit (1);
}

We can compile and run the spooler as below.

$ gcc spooler.c -o spooler
$ >/tmp/shared_memory_key
$ >/tmp/sem-mutex-key
$ >/tmp/sem-buffer-count-key
$ >/tmp/sem-spool-signal-key
$ ./spooler
spooler: hello world
(5942): hello spooler
(5980): Hello spooler
(6012): Hello world

And compiling and running a client from another terminal,

$ gcc client.c -o client
$ ./client
Please type a message: Hello spooler
...

A screenshot of spooler running with three clients is shown below.

1. System V Semaphores
2. POSIX Shared Memory in Linux