Multiplexing with select() and poll()

Multiplexing allows a program to monitor multiple file descriptors (including sockets) simultaneously, waiting until one or more of them becomes ready for I/O operations. This approach is essential for building efficient servers that can handle multiple clients with a single thread.

The Need for Multiplexing

When developing networked applications, especially servers, you often need to handle multiple connections concurrently. There are several approaches to this problem:

Process per client: Fork a new process for each client
Thread per client: Create a new thread for each client
Multiplexing: Use a single thread to handle multiple clients

The first two approaches can become resource-intensive as the number of clients increases. Multiplexing offers a more scalable solution by allowing a single thread to efficiently manage many connections.

The select() Function

The select() function is one of the oldest and most portable methods for multiplexing I/O. It allows a program to monitor multiple file descriptors, waiting until one or more of them becomes ready for some class of I/O operation.

Function Signature

int select(int nfds, fd_set *readfds, fd_set *writefds,
           fd_set *exceptfds, struct timeval *timeout);

Parameters:

nfds: The highest-numbered file descriptor in any of the sets, plus 1
readfds: Set of file descriptors to check for readability
writefds: Set of file descriptors to check for writability
exceptfds: Set of file descriptors to check for exceptional conditions
timeout: Maximum time to wait, or NULL to wait indefinitely

Returns:

The number of ready file descriptors on success
0 if the timeout expired
-1 on error

fd_set Macros

The fd_set type represents a set of file descriptors. Several macros are provided to manipulate these sets:

void FD_ZERO(fd_set *set);           // Clear all file descriptors from the set
void FD_SET(int fd, fd_set *set);    // Add a file descriptor to the set
void FD_CLR(int fd, fd_set *set);    // Remove a file descriptor from the set
int FD_ISSET(int fd, fd_set *set);   // Check if a file descriptor is in the set

Example: TCP Echo Server Using select()

#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>
#include <sys/types.h>
#include <sys/socket.h>
#include <netinet/in.h>
#include <arpa/inet.h>
#include <sys/select.h>
#include <errno.h>
 
#define PORT 8080
#define MAX_CLIENTS 30
#define BUFFER_SIZE 1024
 
int main() {
    int server_fd, client_fds[MAX_CLIENTS];
    fd_set read_fds, master_fds;
    struct sockaddr_in server_addr, client_addr;
    socklen_t client_len = sizeof(client_addr);
    char buffer[BUFFER_SIZE];
    int max_fd, activity, i, valread;
    
    // Initialize client_fds array
    for (i = 0; i < MAX_CLIENTS; i++) {
        client_fds[i] = 0;
    }
    
    // Create server socket
    server_fd = socket(AF_INET, SOCK_STREAM, 0);
    if (server_fd < 0) {
        perror("Socket creation failed");
        exit(EXIT_FAILURE);
    }
    
    // Set socket options to reuse address
    int opt = 1;
    if (setsockopt(server_fd, SOL_SOCKET, SO_REUSEADDR, &opt, sizeof(opt)) < 0) {
        perror("setsockopt failed");
        exit(EXIT_FAILURE);
    }
    
    // Initialize server address
    memset(&server_addr, 0, sizeof(server_addr));
    server_addr.sin_family = AF_INET;
    server_addr.sin_addr.s_addr = INADDR_ANY;
    server_addr.sin_port = htons(PORT);
    
    // Bind socket
    if (bind(server_fd, (struct sockaddr *)&server_addr, sizeof(server_addr)) < 0) {
        perror("Bind failed");
        exit(EXIT_FAILURE);
    }
    
    // Listen for connections
    if (listen(server_fd, 5) < 0) {
        perror("Listen failed");
        exit(EXIT_FAILURE);
    }
    
    printf("Server listening on port %d...\n", PORT);
    
    // Initialize the file descriptor sets
    FD_ZERO(&master_fds);
    FD_SET(server_fd, &master_fds);
    max_fd = server_fd;
    
    // Main loop
    while (1) {
        // Copy the master set to the read set
        read_fds = master_fds;
        
        // Wait for activity on any of the sockets
        activity = select(max_fd + 1, &read_fds, NULL, NULL, NULL);
        
        if (activity < 0) {
            perror("select failed");
            exit(EXIT_FAILURE);
        }
        
        // Check if the server socket has activity (new connection)
        if (FD_ISSET(server_fd, &read_fds)) {
            int new_client = accept(server_fd, (struct sockaddr *)&client_addr, &client_len);
            
            if (new_client < 0) {
                perror("Accept failed");
                exit(EXIT_FAILURE);
            }
            
            printf("New connection from %s:%d\n", 
                   inet_ntoa(client_addr.sin_addr), 
                   ntohs(client_addr.sin_port));
            
            // Add the new client to an empty slot
            for (i = 0; i < MAX_CLIENTS; i++) {
                if (client_fds[i] == 0) {
                    client_fds[i] = new_client;
                    break;
                }
            }
            
            if (i == MAX_CLIENTS) {
                printf("Too many clients, connection rejected\n");
                close(new_client);
            } else {
                // Add the new socket to the master set
                FD_SET(new_client, &master_fds);
                
                // Update the maximum file descriptor
                if (new_client > max_fd) {
                    max_fd = new_client;
                }
                
                printf("Client added to slot %d\n", i);
            }
        }
        
        // Check client sockets for activity
        for (i = 0; i < MAX_CLIENTS; i++) {
            int client_fd = client_fds[i];
            
            if (client_fd > 0 && FD_ISSET(client_fd, &read_fds)) {
                // Read data from the client
                valread = read(client_fd, buffer, BUFFER_SIZE - 1);
                
                if (valread <= 0) {
                    // Client disconnected or error
                    getpeername(client_fd, (struct sockaddr *)&client_addr, &client_len);
                    printf("Client disconnected: %s:%d\n", 
                           inet_ntoa(client_addr.sin_addr), 
                           ntohs(client_addr.sin_port));
                    
                    // Close the socket and remove from the master set
                    close(client_fd);
                    FD_CLR(client_fd, &master_fds);
                    client_fds[i] = 0;
                } else {
                    // Data received, echo it back
                    buffer[valread] = '\0';
                    printf("Received from client %d: %s\n", i, buffer);
                    
                    // Echo back to the client
                    send(client_fd, buffer, valread, 0);
                }
            }
        }
    }
    
    return 0;
}

Limitations of select()

While select() is portable and widely supported, it has several limitations:

Limited Scalability: The fd_set is typically implemented as a bit array with a fixed maximum size (FD_SETSIZE, often 1024)
Inefficiency: The entire fd_set must be copied between user space and kernel space on each call
Linear Scanning: After select() returns, the program must scan all file descriptors to find which ones are ready
No Information About Events: select() only indicates readiness, not what event occurred

The poll() Function

The poll() function provides similar functionality to select() but addresses some of its limitations. It uses an array of pollfd structures instead of bit sets, which allows for a larger number of file descriptors.

Function Signature

int poll(struct pollfd *fds, nfds_t nfds, int timeout);

Parameters:

fds: Array of pollfd structures
nfds: Number of file descriptors in the array
timeout: Timeout in milliseconds, -1 to wait indefinitely, 0 for immediate return

Returns:

The number of ready file descriptors on success
0 if the timeout expired
-1 on error

pollfd Structure

struct pollfd {
    int   fd;       /* File descriptor */
    short events;   /* Events to look for */
    short revents;  /* Events that occurred */
};

Common event flags:

POLLIN: Data available for reading
POLLOUT: Writing now will not block
POLLERR: Error condition
POLLHUP: Hang up (connection closed)
POLLNVAL: Invalid request (fd not open)

Example: TCP Echo Server Using poll()

#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>
#include <sys/types.h>
#include <sys/socket.h>
#include <netinet/in.h>
#include <arpa/inet.h>
#include <poll.h>
#include <errno.h>
 
#define PORT 8080
#define MAX_CLIENTS 30
#define BUFFER_SIZE 1024
 
int main() {
    int server_fd;
    struct sockaddr_in server_addr, client_addr;
    socklen_t client_len = sizeof(client_addr);
    char buffer[BUFFER_SIZE];
    int activity, i, valread;
    
    // Create server socket
    server_fd = socket(AF_INET, SOCK_STREAM, 0);
    if (server_fd < 0) {
        perror("Socket creation failed");
        exit(EXIT_FAILURE);
    }
    
    // Set socket options to reuse address
    int opt = 1;
    if (setsockopt(server_fd, SOL_SOCKET, SO_REUSEADDR, &opt, sizeof(opt)) < 0) {
        perror("setsockopt failed");
        exit(EXIT_FAILURE);
    }
    
    // Initialize server address
    memset(&server_addr, 0, sizeof(server_addr));
    server_addr.sin_family = AF_INET;
    server_addr.sin_addr.s_addr = INADDR_ANY;
    server_addr.sin_port = htons(PORT);
    
    // Bind socket
    if (bind(server_fd, (struct sockaddr *)&server_addr, sizeof(server_addr)) < 0) {
        perror("Bind failed");
        exit(EXIT_FAILURE);
    }
    
    // Listen for connections
    if (listen(server_fd, 5) < 0) {
        perror("Listen failed");
        exit(EXIT_FAILURE);
    }
    
    printf("Server listening on port %d...\n", PORT);
    
    // Initialize the pollfd array
    struct pollfd fds[MAX_CLIENTS + 1];
    int nfds = 1;
    
    // Add the server socket to the pollfd array
    fds[0].fd = server_fd;
    fds[0].events = POLLIN;
    
    // Initialize the rest of the pollfd array
    for (i = 1; i <= MAX_CLIENTS; i++) {
        fds[i].fd = -1;
    }
    
    // Main loop
    while (1) {
        // Wait for activity on any of the sockets
        activity = poll(fds, nfds, -1);
        
        if (activity < 0) {
            perror("poll failed");
            exit(EXIT_FAILURE);
        }
        
        // Check if the server socket has activity (new connection)
        if (fds[0].revents & POLLIN) {
            int new_client = accept(server_fd, (struct sockaddr *)&client_addr, &client_len);
            
            if (new_client < 0) {
                perror("Accept failed");
                exit(EXIT_FAILURE);
            }
            
            printf("New connection from %s:%d\n", 
                   inet_ntoa(client_addr.sin_addr), 
                   ntohs(client_addr.sin_port));
            
            // Add the new client to an empty slot
            for (i = 1; i <= MAX_CLIENTS; i++) {
                if (fds[i].fd < 0) {
                    fds[i].fd = new_client;
                    fds[i].events = POLLIN;
                    
                    // Update nfds if necessary
                    if (i >= nfds) {
                        nfds = i + 1;
                    }
                    
                    break;
                }
            }
            
            if (i > MAX_CLIENTS) {
                printf("Too many clients, connection rejected\n");
                close(new_client);
            } else {
                printf("Client added to slot %d\n", i);
            }
        }
        
        // Check client sockets for activity
        for (i = 1; i < nfds; i++) {
            if (fds[i].fd >= 0 && (fds[i].revents & POLLIN)) {
                // Read data from the client
                valread = read(fds[i].fd, buffer, BUFFER_SIZE - 1);
                
                if (valread <= 0) {
                    // Client disconnected or error
                    getpeername(fds[i].fd, (struct sockaddr *)&client_addr, &client_len);
                    printf("Client disconnected: %s:%d\n", 
                           inet_ntoa(client_addr.sin_addr), 
                           ntohs(client_addr.sin_port));
                    
                    // Close the socket and mark the slot as empty
                    close(fds[i].fd);
                    fds[i].fd = -1;
                    
                    // Compact the array if this was the last client
                    if (i == nfds - 1) {
                        while (i > 1 && fds[i-1].fd == -1) {
                            i--;
                        }
                        nfds = i + 1;
                    }
                } else {
                    // Data received, echo it back
                    buffer[valread] = '\0';
                    printf("Received from client %d: %s\n", i, buffer);
                    
                    // Echo back to the client
                    send(fds[i].fd, buffer, valread, 0);
                }
            }
        }
    }
    
    return 0;
}

Advantages of poll() over select()

No Fixed Limit: The number of file descriptors is not limited by FD_SETSIZE
More Information: The revents field indicates what events occurred
Efficiency: No need to rebuild the entire set for each call
Separation of Input and Output: The events field specifies what to monitor, while revents indicates what happened

Limitations of poll()

Still O(n) Scanning: The program must still scan all file descriptors to find which ones are ready
Inefficiency for Large Sets: For very large numbers of file descriptors, the overhead of copying the entire array between user space and kernel space can be significant
Limited Event Types: The event types are fixed and cannot be extended

Comparison: select() vs poll()

Feature	select()	poll()
Portability	Very high (POSIX, Windows)	High (POSIX)
Maximum FDs	Limited by FD_SETSIZE	Unlimited
Event Information	Limited (ready/not ready)	More detailed (POLLIN, POLLOUT, etc.)
Timeout Specification	struct timeval (microsecond precision)	int (millisecond precision)
Efficiency for Large Sets	Poor	Moderate
API Complexity	Moderate	Moderate

Best Practices for Multiplexing

Use Non-blocking Sockets: Combine multiplexing with non-blocking sockets to prevent any single operation from blocking the entire program
Handle Partial Operations: Send and receive operations may complete partially, so track how much data has been processed
Implement Timeouts: Use the timeout parameter to prevent indefinite waiting
Buffer Management: Implement efficient buffer management for data that cannot be processed immediately
Error Handling: Check for and handle all possible error conditions
Consider More Efficient Alternatives: For high-performance applications, consider using epoll (Linux), kqueue (BSD/macOS), or IOCP (Windows)

Example: Handling Partial Writes

When sending large amounts of data with non-blocking sockets, the send() function may not send all the data at once. Here's how to handle partial writes:

ssize_t send_all(int sockfd, const void *buf, size_t len) {
    const char *ptr = (const char *)buf;
    size_t remaining = len;
    ssize_t sent;
    
    while (remaining > 0) {
        sent = send(sockfd, ptr, remaining, 0);
        
        if (sent < 0) {
            if (errno == EAGAIN || errno == EWOULDBLOCK) {
                // Socket would block, add to write set and return
                return len - remaining;
            } else {
                // Real error
                return -1;
            }
        } else if (sent == 0) {
            // Connection closed
            return len - remaining;
        }
        
        ptr += sent;
        remaining -= sent;
    }
    
    return len;
}

Conclusion

Multiplexing with select() and poll() allows a single thread to efficiently handle multiple connections, making it possible to build scalable server applications without the overhead of creating a thread or process for each client.

While both functions have their limitations, they are widely supported and provide a good foundation for understanding I/O multiplexing. For high-performance applications handling thousands of connections, more advanced mechanisms like epoll() (Linux), kqueue (BSD/macOS), or IOCP (Windows) may be more appropriate.

In the next section, we'll explore advanced multiplexing with epoll(), which addresses many of the limitations of select() and poll() for Linux-based systems.

Test Your Knowledge

Take a quiz to reinforce what you've learned

Exam Preparation

Access short and long answer questions for written exams

Share this page

WhatsApp Twitter/X

Test Your Knowledge

Exam Preparation

On this page