1.5.Networking concepts

All messages following the initial handshake are associated with a ‘capability’. Any number of capabilities can be used concurrently on a single RLPx connection.

A capability is identified by a short ASCII name (max eight characters) and version number. The capabilities supported on either side of the connection are exchanged in the Hello message belonging to the ‘p2p’ capability which is required to be available on all connections.

While the framing layer supports a capability-id, the current version of RLPx doesn’t use that field for multiplexing between different capabilities. Instead, multiplexing relies purely on the message ID.

Each capability is given as much of the message-ID space as it needs. All such capabilities must statically specify how many message IDs they require. On connection and reception of the Hello message, both peers have equivalent information about what capabilities they share (including versions) and are able to form consensus over the composition of message ID space.

Message IDs are assumed to be compact from ID 0x10 onwards (0x00-0x0f is reserved for the “p2p” capability) and given to each shared (equal-version, equal-name) capability in alphabetic order. Capability names are case-sensitive. Capabilities which are not shared are ignored. If multiple versions are shared of the same (equal name) capability, the numerically highest wins, others are ignored.

A connection is a reliable, bidirectional communication channel between two libp2p peers that provides security and the ability to open multiple logically independent streams.

Security in this context means that all communications (after an initial handshake) are encrypted, and that the identity of each peer is cryptographically verifiable by the other peer.

Streams are reliable, bidirectional channels that are multiplexed over a libp2p connection. They must support backpressure, which prevents receivers from being flooded by data from eager senders. They can also be “half closed”, meaning that a stream can be closed for writing data but still open to receiving data and vice versa.

Support for multiple streams ensures that a single connection between peers can support a wide variety of interactions, each with their own protocol. This is especially helpful if connections are difficult to establish due to NAT traversal issues or other connectivity barriers.

Connections take place over an underlying transport, for example TCP sockets, websockets, or various protocols layered over UDP.

While some transport protocols like QUIC have “built in” security and stream multiplexing, others such as TCP need to have those capabilities layered on top of the “raw” transport connection.

When the base capabilities of security and stream multiplexing are not natively supported by the underlying transport protocol, a connection upgrade process occurs to augment the raw transport connection with the required features.

libp2p peers can both initiate connections to other peers and accept incoming connections. We use the term dial to refer to initiating outbound connections, and listen to refer to accepting inbound connections.

One of libp2p’s core design goals is to be adaptable to many network environments, including those that don’t yet exist. To provide this flexibility, the connection upgrade process supports multiple protocols for connection security and stream multiplexing and allows peers to select which to use for each connection.

The process of selecting protocols is called protocol negotiation. In addition to its role in the connection upgrade process, protocol negotiation is also used whenever a new stream is opened over an existing connection. This allows libp2p applications to route application-specific protocols to the correct handler functions.

Each protocol supported by a peer is identified using a unique string called a protocol id. While any string can be used, the conventional format is a path-like structure containing a short name and a version number, separated by / characters. For example: /yamux/1.0.0 identifies version 1.0.0 of the yamux stream multiplexing protocol. multistream-select itself has a protocol id of /multistream/1.0.0.

Including a version number in the protocol id simplifies the case where you want to concurrently support multiple versions of a protocol, perhaps a stable version and an in-development version. By default, libp2p will route each protocol id to its handler function using exact literal matching of the protocol id, so new versions will need to be registered separately. However, the handler function receives the protocol id negotiated for each new stream, so it’s possible to register the same handler for multiple versions of a protocol and dynamically alter functionality based on the version in use for a given stream.

Clients MUST use exact equality when negotiating protocol versions to use and MAY use the version to give priority to higher version numbers.

Clients MUST support multistream-select 1.0 and MAY support multiselect 2.0 when the specification solidifies. Once all clients have implementations for multiselect 2.0, multistream-select 1.0 MAY be phased out.

libp2p is designed to support a variety of transport protocols, including those that do not natively support the core libp2p capabilities of security and stream multiplexing. The process of layering capabilities onto “raw” transport connections is called “upgrading” the connection.

Because there are many valid ways to provide the libp2p capabilities, the connection upgrade process uses protocol negotiation to decide which specific protocols to use for each capability. The protocol negotiation process uses multistream-select as described in the Protocol Negotiation section.

When raw connections need both security and multiplexing, security is always established first, and the negotiation for stream multiplexing takes place over the encrypted channel.

Once we’ve established a libp2p connection to another peer, new streams are multiplexed over the connection using the native facilities of the transport, or the stream multiplexer negotiated during the upgrade process if the transport lacks native multiplexing. Either peer can open a new stream to the other over an existing connection.

When a new stream is opened, a protocol is negotiated using multistream-select. The protocol negotiation process for new streams is very similar to the one used for upgrading connections. However, while the security and stream multiplexing modules for connection upgrades are typically libp2p framework components, the protocols negotiated for new streams can be easily defined by libp2p applications.

Streams are routed to application-defined handler functions based on their protocol id string. Incoming stream requests will propose a protocol id to use for the stream using multistream-select, and the peer accepting the stream request will determine if there are any registered handlers capable of handling the protocol. If no handlers are found, the peer will respond to the proposal with "na".