Run Server

Getting Started

Ensure your .env file in the root directory has the accurate RPC URL for DEV_RPC.

DEV_RPC="wss://rpc.hypertensor.org:443"

Server

Running A Server On Linux

https://github.com/hypertensor-blockchain/subnet-llm/tree/main?tab=readme-ov-file#connect-your-gpu-and-increase-petals-capacity

Running A Server on Linux With An AMD GPU

https://github.com/bigscience-workshop/petals/wiki/Running-on-AMD-GPU

Run A Server on Windows (WSL)

https://github.com/bigscience-workshop/petals/wiki/Run-Petals-server-on-Windows

Note

  • This will not run with the default Windows OS. Ensure to use WSL if on Windows.

  • Running a relay server will receive fewer rewards, currently by 33%, than a direct server. Relay servers are slower and negatively impact performance.

    • Relay servers may be unresponsive at times and cause a broken consensus.

    • Testnet v1.0 will be testing the peer consensus mechanism with relay servers. Relay servers may be removed from the peer consensus mechanism, thus unable to receive rewards, once live testing is completed or during live testing.

Run Server

Run the server

python -m petals_tensor.cli.run_server [model_path] --public_ip [public_ip] --port [port]

  • model_path: The HuggingFace model path.

  • --public_ip: The public IP of the server for other peers to connect to.

  • --port: The port of the server for other peers to connect to (open port before running the command).

Example

python -m petals_tensor.cli.run_server bigscience/bloom-560m --public_ip 127.0.0.1 --port 31330

Specify --initial_peers Example

If the initial peers found in the petals_tensor/constants.py file are not online, you can specify initial peers.

python -m petals_tensor.cli.run_server bigscience/bloom-560m --public_ip 127.0.0.1 --port 31330 --initial_peers /ip4/127.0.0.1/tcp/31330/p2p/12D3KooWHkT1TjZRFKsw4e4xnAgoHMD7oenTPYZTv4kegid3w5Kb /ip4/127.0.0.1/udp/31330/quic/p2p/12D3KooWHkT1TjZRFKsw4e4xnAgoHMD7oenTPYZTv4kegid3w5Kb

Note

This should begin loading blocks (not to be confused with the blockchain blocks) within 5-10 minutes or less. If it doesn't output logs such as Loaded bigscience/bloom-560m block 1 within 5-10 minutes, try restarting your server or computer. Once these logs begin appearing, it can take up to 30 minutes to complete.

PreviousIntroductionNextPropose Subnet

Last updated