Python Socket Programming

Python’s socket module is part of the standard library and can be used to build network-based applications. This tutorial explains the fundamentals of Python socket programming, including creating a basic client-server setup, serving multiple clients with threads, and understanding how TCP and UDP sockets differ. It also covers establishing connections, transmitting and receiving data, and creating network applications that manage connections and errors correctly. The examples require Python 3.10 or newer.

Key Takeaways

• Python’s socket module provides a lightweight, cross-platform interface to operating-system socket APIs such as Berkeley sockets and Winsock, meaning many Python socket functions closely reflect the underlying OS operations.
• TCP servers use bind(), listen(), and accept(), while TCP clients use connect(). UDP sockets instead rely on recvfrom() and sendto() and do not need a connection-establishment phase.
• Set SO_REUSEADDR before calling bind() to reduce Address already in use errors during development.
• Prefer sendall() over send() when transmitting larger payloads so partial sends do not cause incomplete messages.
• Creating one thread for every client is straightforward, but it becomes inefficient once concurrency reaches several hundred connections. For larger workloads, use asyncio or selectors.
• Place recv() and send() operations inside try/except handling for ConnectionResetError, BrokenPipeError, and socket.timeout, and make sure sockets are closed from a finally block.

Prerequisites

• Python 3.10 or newer installed on the local system. The message-framing example uses the bytes | None union type syntax that became available in Python 3.10.
• Basic familiarity with Python concepts such as variables, functions, loops, and try/except statements.
• Two terminal windows or tabs available at the same time so the server and client programs can run as separate processes.

What Is a Socket in Python

A socket is a file descriptor, or handle, representing one endpoint of a network connection. Python’s socket module provides a thin interface over the socket APIs supplied by the operating system, including Berkeley sockets and Winsock. It exposes operations such as bind(), listen(), accept(), connect(), send(), and recv().

How the Python socket Module Maps to the OS Network Stack

Two operating-system behaviors are responsible for many common Python socket problems. First, recv() returns only the bytes currently available, up to the requested maximum, rather than guaranteeing a specific number of bytes. Second, ports may remain reserved in the TIME_WAIT state for as long as roughly 60 seconds after a server closes. Partial-read issues and “Address already in use” errors generally originate from these behaviors.

When socket.socket(AF_INET, SOCK_STREAM) is called, the operating system creates a socket file descriptor associated with the IPv4 TCP stack. bind() assigns the socket a local address and port. listen() turns it into a passive listening socket. accept() waits until the TCP three-way handshake has finished and then returns a new connected socket. sendall() places bytes into the kernel’s sending buffer. recv(n) retrieves as many as n bytes from the receive buffer. If fewer bytes are currently available, fewer bytes are returned. This behavior is normal and is the reason message-framing techniques are necessary.

TCP vs UDP: Choosing the Right Socket Type

The most frequently used socket types are SOCK_STREAM for TCP and SOCK_DGRAM for UDP. Unix domain sockets combine AF_UNIX with SOCK_STREAM to provide inter-process communication between programs on the same host without sending traffic through the network stack.

Setting Up a Basic TCP Server and Client

A TCP server binds itself to an address, waits for a client connection, and then exchanges data inside a loop. The following two scripts form a functioning server-client pair that can be executed on the same machine.

Python Socket Server

Save the following program as socket_server.py. It binds the server to 127.0.0.1, the loopback interface, so only programs running on the same machine can connect. To listen on every available network interface instead, use ” or ‘0.0.0.0’.

socket_server.py

import socket


def server_program():
    host = '127.0.0.1'  # loopback address used for local testing
    port = 5000  # use a port number above 1024

    server_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)  # create an IPv4 TCP socket
    server_socket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)  # permit immediate port reuse after shutdown
    server_socket.bind((host, port))  # associate the socket with the host and port

    server_socket.listen(5)  # allow up to 5 queued connection requests
    conn, address = server_socket.accept()  # wait for and accept a connection
    print("Connection from: " + str(address))
    while True:
        raw = conn.recv(1024)  # receive at most 1024 bytes; larger messages may require several recv() calls
        if not raw:
            break
        try:
            data = raw.decode('utf-8')
        except UnicodeDecodeError:
            print(f"Received non-UTF-8 data from {address}, skipping")
            continue
        print("from connected user: " + str(data))
        data = input(' -> ')
        conn.sendall(data.encode())  # transmit the response to the client

    conn.close()  # close the client connection
    server_socket.close()  # close the server's listening socket


if __name__ == '__main__':
    server_program()

A UnicodeDecodeError from raw.decode(‘utf-8’) can have two common causes: the received bytes may genuinely not be UTF-8, or a valid multibyte character may have been divided between two recv() operations. The except block handles both situations by silently discarding those bytes. If message boundaries or non-ASCII text are important, use the send_msg and recv_msg functions described in the framing section below. Those helpers collect an entire message before decoding it.

There are two separate socket objects in this example. server_socket is the listening socket. It remains active throughout the server’s lifetime and is responsible only for accepting new connections. conn is the connected socket created by accept(). It represents one specific client and is the socket used for reading and writing data. Every client receives its own conn object, while server_socket is not used directly with send() or recv().

The input() function inside the loop makes the example server interactive. In an actual application, replace input() with the relevant application logic. Keeping input() there causes the server to stop receiving additional data until the person operating the terminal enters a response.

Python Socket Client

Save the client program as socket_client.py. It connects to the same loopback address and port used by the server. The client does not need to call bind(), because the operating system automatically selects an ephemeral local port when connect() runs.

socket_client.py

import socket


def client_program():
    host = '127.0.0.1'  # loopback address used for local testing
    port = 5000  # port used by the socket server

    client_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)  # create an IPv4 TCP socket
    client_socket.connect((host, port))  # establish a connection with the server

    message = input(" -> ")  # collect user input

    while message.lower().strip() != 'bye':
        client_socket.sendall(message.encode())  # transmit the message
        raw = client_socket.recv(1024)  # receive at most 1024 bytes; larger messages may require several recv() calls
        if not raw:
            break
        try:
            data = raw.decode('utf-8')
        except UnicodeDecodeError:
            print("Received non-UTF-8 data from server, skipping")
            continue

        print('Received from server: ' + data)  # display the response in the terminal

        message = input(" -> ")  # request another message

    client_socket.close()  # close the connection


if __name__ == '__main__':
    client_program()

Python Socket Programming Output

Start the server program first. Then launch the client program from another terminal. Enter a message in the client terminal and press Enter to transmit it to the server.

The server terminal displays:

Output:

python3 socket_server.py
Connection from: ('127.0.0.1', 57822)
from connected user: Hi
 -> Hello
from connected user: How are you?
 -> Good
from connected user: Awesome!
 -> Ok then, bye!

The client terminal displays:

Output:

python3 socket_client.py
 -> Hi
Received from server: Hello
 -> How are you?
Received from server: Good
 -> Awesome!
Received from server: Ok then, bye!
 -> Bye

The server operates on port 5000, but the client also receives its own port number from the operating system during connect(). In this example, port 57822 was assigned to the client side.

Sending and Receiving Complete Messages

TCP provides a continuous byte stream rather than individual messages. If sendall() transmits 500 bytes, the first recv(1024) operation on the receiving side could return 200 bytes, all 500 bytes, or another quantity depending on buffering and network timing. A single sendall() operation therefore does not necessarily correspond to a single recv() operation.

A common solution is length-prefix framing. Before transmitting the payload, send a fixed four-byte header containing the payload size as a big-endian unsigned integer. The receiver first reads exactly four bytes, decodes the length, and then repeatedly calls recv() until the specified number of payload bytes has been collected. This technique works with both binary and text-based protocols and introduces only four additional bytes for each message.

The following helper functions implement that approach. send_msg adds the header before the payload and sends both through one sendall() call. recv_msg creates a bytearray buffer and relies on the private _recv_exactly helper to collect bytes across as many recv() operations as the kernel requires.

framing.py

import socket
import struct


def send_msg(sock, data: bytes) -> None:
    header = struct.pack('>I', len(data))  # create a 4-byte big-endian length header
    sock.sendall(header + data)


def recv_msg(sock) -> bytes | None:
    raw_len = _recv_exactly(sock, 4)
    if raw_len is None:
        return None  # the connection ended before the header was received
    msg_len = struct.unpack('>I', raw_len)[0]
    return _recv_exactly(sock, msg_len)


def _recv_exactly(sock, n: int) -> bytes | None:
    buf = bytearray()
    while len(buf) < n:
        chunk = sock.recv(n - len(buf))
        if not chunk:
            return None  # the remote endpoint closed the connection
        buf.extend(chunk)
    return bytes(buf)

Use send_msg and recv_msg instead of direct sendall and recv calls whenever message boundaries are important. The earlier echo server uses plain recv and sendall to keep the example simple. For structured data, those operations can be replaced with send_msg and recv_msg.

Handling Multiple Clients with Threading

A server designed for only one client blocks while waiting on accept() and can process only one connection at a time. To serve multiple clients concurrently, Python’s threading module can create a separate thread for every accepted connection.

Spawning a Thread per Client Connection

The complete server below uses handle_client() to manage an individual connection and server_program() to continuously accept new connections. Every accepted connection launches a daemon thread so the worker threads do not keep the Python interpreter running after the primary loop ends.

threaded_server.py

import socket
import threading


def handle_client(conn, addr):
    print(f"Connection from: {addr}")
    try:
        while True:
            data = conn.recv(1024)
            if not data:  # an empty bytes value indicates that the client closed the connection
                break
            conn.sendall(data)  # send the received data back to the client
    except (ConnectionResetError, BrokenPipeError) as e:
        print(f"Connection error with {addr}: {e}")
    finally:
        conn.close()


def server_program():
    host = '127.0.0.1'  # loopback address used for local testing
    port = 5000

    server_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)  # create an IPv4 TCP socket
    server_socket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)  # permit immediate reuse after shutdown
    server_socket.bind((host, port))
    server_socket.listen(5)  # permit up to 5 queued connection attempts
    print(f"Server listening on {host}:{port}")

    try:
        while True:
            conn, addr = server_socket.accept()
            thread = threading.Thread(target=handle_client, args=(conn, addr), daemon=True)
            thread.start()
            print(f"Active connections: {threading.active_count() - 1}")
    except KeyboardInterrupt:
        print("Server shutting down")
    finally:
        server_socket.close()


if __name__ == '__main__':
    server_program()

threading.active_count() reports how many threads are currently alive, including the main thread. Subtracting one therefore gives the current number of active client-handling threads.

Pressing Ctrl-C causes a KeyboardInterrupt. The outer try/except catches that exception, allowing the listening socket to be closed properly before the program terminates.

Thread Safety Considerations for Shared State

If several threads access and modify shared information, such as a list of connected clients, a message history, or a counter, protect that data with threading.Lock(). Without a lock, operations from different threads can overlap and leave the shared state inconsistent.

import threading

client_list = []
lock = threading.Lock()

def handle_client(conn, addr):
    with lock:
        client_list.append(addr)
    # process the connection...

Use with lock: instead of manually invoking lock.acquire() followed by lock.release(). The with statement ensures that the lock is released even when an exception occurs inside the protected section.

Limitations of the Thread-per-Client Model

On Linux, each thread can reserve as much as 8 MB of virtual address space for its stack, although the resident memory used by an idle thread is generally below 100 KB. With high connection rates, scheduler overhead and context switching usually become more significant than the raw memory requirement. These costs increase as concurrent connections reach the hundreds. Servers expected to manage hundreds or thousands of simultaneous connections can instead use selectors.DefaultSelector for I/O multiplexing or asyncio.start_server() for coroutine-based concurrency. Both approaches are discussed later in the Python Socket Libraries section.

Error Handling and Socket Exceptions

Socket applications commonly fail in several predictable ways. ConnectionResetError can occur when a remote process terminates unexpectedly. BrokenPipeError can occur when data is written to a socket that has already been closed by the other endpoint. socket.timeout occurs when a blocking operation takes longer than the configured timeout. The following patterns handle all three conditions without allowing the server to crash.

Common Exceptions: ConnectionResetError, BrokenPipeError, socket.timeout

• ConnectionResetError: occurs when the remote endpoint unexpectedly terminates the connection, such as when a client process is killed during an active session.
• BrokenPipeError: occurs during send() or sendall() when the receiving endpoint has already closed its socket.
• socket.timeout: occurs when a blocking operation runs longer than the timeout configured through settimeout().

Using Try-Except Blocks Around recv and send

Place the receiving and sending loop inside try/except handling that covers all three exceptions:

while True:
    try:
        data = conn.recv(1024)
        if not data:
            break
        conn.sendall(data)
    except ConnectionResetError:
        print("Client disconnected unexpectedly")
        break
    except BrokenPipeError:
        print("Send failed: client has closed the connection")
        break
    except socket.timeout:
        print("Connection timed out")
        break

Checking for an empty bytes value with if not data: break is different from exception handling. When recv() returns b”, it normally means that the remote endpoint performed a clean shutdown. It is not itself an error.

Closing Sockets Safely with a finally Block

Put conn.close() inside a finally block so that the socket is always closed, regardless of whether the loop ends normally or because of an exception:

conn, addr = server_socket.accept()
try:
    while True:
        data = conn.recv(1024)
        if not data:
            break
        conn.sendall(data)
finally:
    conn.close()

Sockets that are not closed correctly consume file descriptors. Linux commonly defaults to a limit of 1024 open file descriptors for each process. A server processing many connections over time can eventually reach this limit unless every completed connection is properly closed.

Socket Configuration and Options

Three settings are relevant to most servers. SO_REUSEADDR helps prevent address-already-in-use errors after a restart. settimeout() prevents inactive connections from blocking indefinitely. setblocking(False) provides the starting point for implementing non-blocking I/O.

Setting SO_REUSEADDR to Avoid Address Already in Use Errors

After a server stops, the operating system can retain the port in TIME_WAIT for approximately 60 seconds so delayed packets belonging to the earlier connection can disappear. Attempting to bind the same port while it remains in that state can produce:

OSError: [Errno 98] Address already in use

The issue can be addressed with one line placed after creating the socket and before calling bind():

server_socket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)

This instructs the operating system to permit immediate reuse of the same address and port combination. It is a standard setting for servers that may be restarted during development or deployment.

Configuring Socket Timeouts with settimeout

socket.settimeout(seconds) affects blocking operations performed through that socket. If an operation does not finish within the specified interval, socket.timeout is raised. Passing None restores ordinary blocking behavior without a timeout.

The following example configures a timeout on the client before attempting the connection:

import socket

client_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)  # create an IPv4 TCP socket
client_socket.settimeout(5)
try:
    client_socket.connect(('127.0.0.1', 5000))
except socket.timeout:
    print("Connection timed out after 5 seconds")

Non-Blocking Sockets and setblocking(False)

Calling sock.setblocking(False) changes recv(), send(), accept(), and connect() so that they immediately raise BlockingIOError whenever the requested action would otherwise block. This behavior forms the basis of I/O multiplexing. One or more non-blocking sockets can be registered with selectors.DefaultSelector, which then indicates which sockets are ready for reading or writing without blocking the others.

import selectors
import socket

sel = selectors.DefaultSelector()
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)  # create an IPv4 TCP socket
sock.setblocking(False)
sel.register(sock, selectors.EVENT_READ, data=None)

For a complete server based on this multiplexing technique, refer to the Python selectors documentation.

Building a UDP Server and Client in Python

UDP sockets can exchange messages without first establishing a connection. The following echo server and client communicate through UDP and use port 5001 so they do not conflict with the earlier TCP examples.

Key Differences from TCP Socket Setup

UDP uses recvfrom() rather than recv(). recvfrom() returns both the received data and the sender’s address. sendto() accepts the data along with the destination address. A UDP server does not need listen(), accept(), or connect().

UDP Server Code Example

udp_server.py

import socket


def udp_server():
    host = '127.0.0.1'  # loopback address used for local testing
    port = 5001

    server_socket = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)  # create an IPv4 UDP socket
    server_socket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)  # permit immediate reuse after shutdown
    server_socket.bind((host, port))
    print(f"UDP server listening on {host}:{port}")

    while True:
        data, addr = server_socket.recvfrom(4096)
        print(f"Received from {addr}: {data.decode()}")
        server_socket.sendto(data, addr)  # return the received data to its sender


if __name__ == '__main__':
    udp_server()

UDP Client Code Example

udp_client.py

import socket


def udp_client():
    host = '127.0.0.1'  # loopback address used for local testing
    port = 5001

    client_socket = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)  # create an IPv4 UDP socket
    client_socket.sendto(b"Hello, UDP server!", (host, port))
    data, addr = client_socket.recvfrom(4096)
    print(f"Response from server: {data.decode()}")
    client_socket.close()


if __name__ == '__main__':
    udp_client()

Launch python3 udp_server.py first and then run python3 udp_client.py from another terminal. Since UDP is connectionless, the client can immediately send a datagram without performing a handshake and receive the echoed response through a single recvfrom() operation.

Python Socket Libraries and When to Use Them Instead

Working with raw sockets provides complete control, but it also requires building much of the surrounding infrastructure yourself. Before using raw socket operations directly, determine whether one of the following abstractions already meets the requirements.

socketserver Module for Simpler Server Boilerplate

socketserver.TCPServer manages bind(), listen(), and accept() internally. A server can subclass BaseRequestHandler and define a handle() method that contains the logic for each connection:

import socketserver


class EchoHandler(socketserver.BaseRequestHandler):
    def handle(self):
        data = self.request.recv(1024)
        self.request.sendall(data)


if __name__ == '__main__':
    with socketserver.TCPServer(('127.0.0.1', 5002), EchoHandler) as server:
        server.serve_forever()

socketserver works well for straightforward request-response servers where detailed control over individual socket options is unnecessary. To process connections with multiple threads, combine it with socketserver.ThreadingMixIn.

asyncio for High-Concurrency Use Cases

asyncio.start_server() creates a non-blocking TCP server that operates through the event loop. Instead of allocating an operating-system thread for each connection, it uses coroutines and can therefore manage thousands of simultaneous connections from one thread. Async socket patterns are the standard approach for high-concurrency applications in Python 3.11 and newer:

import asyncio


async def handle_client(reader, writer):
    data = await reader.read(1024)
    writer.write(data)
    await writer.drain()
    writer.close()
    await writer.wait_closed()


async def main():
    server = await asyncio.start_server(
        handle_client, '127.0.0.1', 5003
    )
    async with server:
        await server.serve_forever()


if __name__ == '__main__':
    asyncio.run(main())

Every connection is processed by a coroutine. As a result, thousands of coroutines can operate concurrently without the memory usage and scheduling overhead associated with allocating one thread to every client. The Python asyncio documentation provides a complete reference.

When Raw Sockets Are Still the Right Choice

Use the socket module directly when you need to:

• Create a custom protocol over TCP or UDP when no higher-level library represents the required message format.
• Study network programming close to the operating-system level because raw sockets provide the Berkeley sockets API with very little abstraction.
• Configure socket options directly, including SO_RCVBUF, SO_SNDBUF, or TCP_NODELAY, when those controls are not exposed by a higher-level library.

Concurrency model comparison:

FAQ

1. What Is the Default Buffer Size for socket.recv() in Python, and How Do I Choose the Right Value?

recv() does not provide a default buffer size. The size argument must be supplied and determines the maximum number of bytes that one call can read. Frequently used values include 1024 and 4096 bytes. A 4096-byte buffer is a common option for interactive or line-based protocols. For larger transfers, including file downloads, a larger value such as 65536 bytes can be used while repeatedly reading until all expected data has arrived. Increasing the buffer size can reduce the number of system calls, although it also increases memory usage for each connection.

2. Why Do I Get an “Address Already in Use” Error When Restarting My Server?

When a server process stops, the operating system can keep its port in the TIME_WAIT state for approximately 60 seconds so delayed packets from the previous connection can expire. Calling bind() again during this period can result in OSError: [Errno 98] Address already in use. Configure SO_REUSEADDR immediately after creating the socket and before calling bind() to avoid this problem:

server_socket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)

This allows the operating system to reuse the address even when another socket has used it recently.

3. What Is the Difference Between socket.send() and socket.sendall() in Python?

send() can transmit fewer bytes than the total amount supplied and returns the number of bytes actually sent. When working with larger payloads, the application must repeatedly call send() while moving the buffer position forward until every byte has been transmitted. sendall() performs this repetition internally and only raises an exception when transmission fails. For ordinary application code, sendall() is generally preferable because it prevents part of a message from being silently omitted due to a partial send.

4. How Do I Handle Multiple Clients Simultaneously in a Python Socket Server?

There are two main approaches. One option is to create a threading.Thread for every accepted connection. Another is to use selectors.DefaultSelector for non-blocking I/O multiplexing. Threading is easier to implement and is suitable for light or moderate workloads. When large numbers of clients need to connect concurrently, asyncio with asyncio.start_server() provides a more scalable alternative, as demonstrated in the earlier libraries section.

5. What Is the Backlog Parameter in socket.listen()?

The backlog argument passed to listen() specifies the maximum number of pending connection attempts that the operating system will queue before additional connections are rejected. It does not set the maximum number of established connections; it only controls the queue of connections waiting to be accepted. A backlog value of 5 is suitable for development, while production systems can use a value such as 128 or socket.SOMAXCONN.

6. When Should I Use UDP Instead of TCP for Python Socket Programming?

UDP is appropriate when minimizing latency is more important than guaranteeing delivery. Examples include DNS queries, video streaming, online games, and telemetry systems. UDP does not promise that packets will arrive, arrive in the original order, or arrive at all. Applications that require acknowledgments or ordered, reliable delivery should use TCP.

7. How Do I Set a Timeout on a Python Socket Connection?

Call socket.settimeout(seconds) on the socket before invoking connect() or recv():

client_socket.settimeout(5)

If the operation does not finish within five seconds, socket.timeout is raised. Passing None returns the socket to blocking mode without a timeout.

8. What Happens If the Server Calls recv() but the Client Has Already Closed the Connection?

When the remote endpoint closes a connection cleanly, recv() returns an empty bytes object, b”. This is not an exception. It is the normal indication that the other endpoint performed a graceful shutdown. The receiving loop should check the returned value, stop when b” is encountered, and then close the server-side socket. If the client connection disappears unexpectedly because of a terminated process or network failure, recv() can raise ConnectionResetError instead.

Conclusion

This tutorial introduced Python socket programming from the fundamentals upward. It explained how the socket module corresponds to the operating system’s Berkeley sockets API, how to create a TCP server and client for a single connection, how to process several clients with threading, how to configure options such as SO_REUSEADDR and socket timeouts, how non-blocking I/O works, how to implement a UDP echo server, and how to decide between raw sockets and higher-level options such as socketserver and asyncio.

Using these concepts, you can create custom TCP and UDP protocols, troubleshoot common socket problems including ConnectionResetError and BrokenPipeError, and select a concurrency approach that matches the workload. The functional examples in this tutorial run directly with Python 3.x and provide a foundation that can be extended with authentication, TLS encryption through ssl.SSLContext.wrap_socket(), or custom message framing.