Peers & Peer IDs

Cryptography

A Peer Identity is a unique reference to a specific peer within the overall peer-to-peer network.

As well as serving as a unique identifier for each peer, a Peer ID is a verifiable link between a peer and its public cryptographic key.

Each libp2p peer controls a private key, which it keeps secret from all other peers. Every private key has a corresponding public key, which is shared with other peers.

Together, the public and private key (or "key pair") allow peers to establish secure communication channels with each other.

Conceptually, a Peer ID is a cryptographic hash of a peer’s public key. When peers establish a secure channel, the hash can be used to verify that the public key used to secure the channel is the same one used to identify the peer.

The Peer ID spec goes into detail about the byte formats used for libp2p public keys and how to hash the key to produce a valid Peer ID.

https://docs.libp2p.io/concepts/fundamentals/peers/

---

Peer IDs are the cornerstone of authentication and should always be generated in a deterministic manner. We suggest NEVER using randomly generated peer_id's to ensure signatures can be authenticated for secure communication channels within the subnet.

Peer IDs in the subnet must match the peer IDs registered on-chain.

Each subnet validator node should generate three peer IDs: a main peer ID for communication, a bootstrap peer ID, and a client peer ID. When registering a subnet on-chain, all peer IDs are required.

Peer IDs

Main Peer ID

The main peer ID is used for the validator node and all communications. This will be used to authenticate proof of stake from the Hypertensor blockchain to the subnet, and for authentication between communication of other peers in the subnet.

Bootstrap Peer ID

The bootstrap peer ID is used specifically for the bootstrap node. This bootstrap peer ID is also tied to the same subnet node on-chain and is also used for proof-of-stake.

What is a bootstrap or bootnode?

When a new node joins a decentralized network, it needs to connect to nodes that are already on the network in order to then discover new peers. These entry points into the network are called bootstrap nodes (or bootnodes). Subnets should have a public list of bootstrap nodes in their documentation for other nodes to connect to.

Bootstrap nodes do not validate anything, but are an entry point for others to connect to.

The main validator nodes that run application logic can be bootstrap nodes, although it is suggested not to use these nodes as entry points and not make these IPs and ports known to the general public.

Client Peer ID

The client peer ID is designed for use as a client, such as for hosting a frontend for an inference subnet.

---

Generating Peer IDs Deterministically

Each of these examples will save the private key file to the given --path, --boostrap_path, --client_path.

See keygen.py in the mesh template for the full example.

Ed25519

import argparse
import asyncio
import hashlib
import os

import multihash
from cryptography.hazmat.primitives import serialization
from cryptography.hazmat.primitives.asymmetric import ed25519, rsa

from mesh.p2p.p2p_daemon import P2P
from mesh.p2p.p2p_daemon_bindings.datastructures import PeerID
from mesh.proto import crypto_pb2
from mesh.utils.logging import get_logger

logger = get_logger(__name__)

def generate_ed25519_private_key(path: str):
    private_key = ed25519.Ed25519PrivateKey.generate()

    raw_private_key = private_key.private_bytes(
        encoding=serialization.Encoding.Raw,
        format=serialization.PrivateFormat.Raw,
        encryption_algorithm=serialization.NoEncryption()
    )

    public_key = private_key.public_key().public_bytes(
        encoding=serialization.Encoding.Raw,
        format=serialization.PublicFormat.Raw,
    )

    combined_key_bytes = raw_private_key + public_key

    protobuf = crypto_pb2.PrivateKey(key_type=crypto_pb2.KeyType.Ed25519, data=combined_key_bytes)

    with open(path, "wb") as f:
        f.write(protobuf.SerializeToString())

    os.chmod(path, 0o400)
    with open(path, "rb") as f:
        data = f.read()
        key_data = crypto_pb2.PrivateKey.FromString(data).data
        private_key = ed25519.Ed25519PrivateKey.from_private_bytes(key_data[:32])
        public_key = private_key.public_key().public_bytes(
            encoding=serialization.Encoding.Raw,
            format=serialization.PublicFormat.Raw,
        )

        combined_key_bytes = private_key.private_bytes_raw() + public_key

        encoded_public_key = crypto_pb2.PublicKey(
            key_type=crypto_pb2.Ed25519,
            data=public_key,
        ).SerializeToString()

        encoded_digest = b"\x00$" + encoded_public_key

        peer_id = PeerID(encoded_digest)

        peer_id_to_bytes = peer_id.to_bytes()

        assert peer_id == peer_id_to_bytes

    return encoded_digest

def main():
    parser = argparse.ArgumentParser(formatter_class=argparse.ArgumentDefaultsHelpFormatter)
    parser.add_argument("--path", type=str, required=False, default="private_key.key", help="File location of private key. ")
    parser.add_argument("--bootstrap_path", type=str, required=False, default="bootstrap_private_key.key", help="File location of bootstrap private key. ")

    args = parser.parse_args()

    path = args.path
    bootstrap_path = args.bootstrap_path

    encoded_digest = generate_ed25519_private_key(path)
    bootstrap_encoded_digest = generate_ed25519_private_key(bootstrap_path)

    peer_id = PeerID(encoded_digest)
    bootstrap_peer_id = PeerID(bootstrap_encoded_digest)
    logger.info(f"Peer ID {peer_id}")
    logger.info(f"Bootstrap Peer ID (Optional usage) {bootstrap_peer_id}")

    """
    Test identity
    """
    async def test_identity(identity_path: str):
        p2p = await P2P.create(identity_path=identity_path)
        p2p_peer_id = p2p.peer_id

        await p2p.shutdown()

        return p2p_peer_id

    p2p_peer_id = asyncio.run(test_identity(path))
    assert peer_id.__eq__(p2p_peer_id), "Generated Peer ID and subnet Peer ID are not equal"
    p2p_bootstrap_peer_id = asyncio.run(test_identity(bootstrap_path))
    assert bootstrap_peer_id.__eq__(p2p_bootstrap_peer_id), "Generated Bootstrap Peer ID and subnet Peer ID are not equal"

if __name__ == "__main__":
    main()

RSA

import argparse
import asyncio
import hashlib
import os

import multihash
from cryptography.hazmat.primitives import serialization
from cryptography.hazmat.primitives.asymmetric import ed25519, rsa

from mesh.p2p.p2p_daemon import P2P
from mesh.p2p.p2p_daemon_bindings.datastructures import PeerID
from mesh.proto import crypto_pb2
from mesh.utils.logging import get_logger

logger = get_logger(__name__)

def generate_rsa_private_key(path: str):
    # Generate the RSA private key
    private_key = rsa.generate_private_key(
        public_exponent=65537,
        key_size=2048,
    )

    # Serialize the private key to DER format
    private_key = private_key.private_bytes(
        encoding=serialization.Encoding.DER,
        format=serialization.PrivateFormat.TraditionalOpenSSL,
        encryption_algorithm=serialization.NoEncryption()
    )

    protobuf = crypto_pb2.PrivateKey(key_type=crypto_pb2.KeyType.RSA, data=private_key)

    with open(path, "wb") as f:
        f.write(protobuf.SerializeToString())

    with open(path, "rb") as f:
        data = f.read()
        key_data = crypto_pb2.PrivateKey.FromString(data).data

        private_key = serialization.load_der_private_key(key_data, password=None)

        encoded_public_key = private_key.public_key().public_bytes(
            encoding=serialization.Encoding.DER,
            format=serialization.PublicFormat.SubjectPublicKeyInfo,
        )

        encoded_public_key = crypto_pb2.PublicKey(
            key_type=crypto_pb2.RSA,
            data=encoded_public_key,
        ).SerializeToString()

        encoded_digest = multihash.encode(
            hashlib.sha256(encoded_public_key).digest(),
            multihash.coerce_code("sha2-256"),
        )
    return encoded_digest

def main():
    parser = argparse.ArgumentParser(formatter_class=argparse.ArgumentDefaultsHelpFormatter)
    parser.add_argument("--path", type=str, required=False, default="private_key.key", help="File location of private key. ")
    parser.add_argument("--bootstrap_path", type=str, required=False, default="bootstrap_private_key.key", help="File location of bootstrap private key. ")

    args = parser.parse_args()

    path = args.path
    bootstrap_path = args.bootstrap_path

    encoded_digest = generate_rsa_private_key(path)
    bootstrap_encoded_digest = generate_rsa_private_key(bootstrap_path)

    peer_id = PeerID(encoded_digest)
    bootstrap_peer_id = PeerID(bootstrap_encoded_digest)
    logger.info(f"Peer ID {peer_id}")
    logger.info(f"Bootstrap Peer ID (Optional usage) {bootstrap_peer_id}")

    async def test_identity(identity_path: str):
        p2p = await P2P.create(identity_path=identity_path)
        p2p_peer_id = p2p.peer_id

        await p2p.shutdown()

        return p2p_peer_id

    p2p_peer_id = asyncio.run(test_identity(path))
    assert peer_id.__eq__(p2p_peer_id), "Generated Peer ID and subnet Peer ID are not equal"
    p2p_bootstrap_peer_id = asyncio.run(test_identity(bootstrap_path))
    assert bootstrap_peer_id.__eq__(p2p_bootstrap_peer_id), "Generated Bootstrap Peer ID and subnet Peer ID are not equal"

if __name__ == "__main__":
    main()