Run Server

Running a server is starting a model validator to enter as a peer to contribute to hosting machine learning models. Once you have completed this you will then add your model validator to interface with Hypertensor to generate rewards.

Server

Running A Petals Tensor Server On Linux

https://github.com/hypertensor-blockchain/petals_tensor/tree/main?tab=readme-ov-file#connect-your-gpu-and-increase-petals-capacity

Running A Petals Tensor Server on Linux With An AMD GPU

https://github.com/bigscience-workshop/petals/wiki/Running-on-AMD-GPU

Run A Petals Tensor Server on Windows (WSL)

https://github.com/bigscience-workshop/petals/wiki/Run-Petals-server-on-Windows


Note

  • This will not run with the default Windows OS. Ensure to use WSL if on Windows.

  • Running a relay server will receive fewer rewards, currently by 33%, than a direct server. Relay servers are slower and negatively impact performance.

    • Relay servers may be unresponsive at times and cause a broken consensus.

    • Testnet v1.0 will be testing the peer consensus mechanism with relay servers. Relay servers may be removed from the peer consensus mechanism, thus unable to receive rewards, once live testing is completed or during live testing.


Run Server

Run the server

python -m petals_tensor.cli.run_server [model_path] --public_ip [public_ip] --port [port] --tcp_public_ip [tcp_public_ip] --tcp_port [tcp_port]
  • model_path: The HuggingFace model path.

  • --public_ip: The public IP of the server for other peers to connect to.

  • --port: The port of the server for other peers to connect to (open port before running the command).

  • --tcp_public_ip: The IP for the blockchain to call.

  • --tcp_port: The port for the blockchain to call.

Example

python -m petals_tensor.cli.run_server petals-team/StableBeluga2 --public_ip 127.0.0.1 --port 31330 --tcp_public_ip 127.0.0.1 --tcp_port 3000

Specify --initial_peers Example

If the initial peers found in the petals/constants.py file are not online, you can specify initial peers.

python -m petals_tensor.cli.run_server petals-team/StableBeluga2 --public_ip 127.0.0.1 --port 31330 --tcp_public_ip 127.0.0.1 --tcp_port 3000 python3 -m petals.cli.run_server bigscience/bloom-560m --initial_peers /ip4/127.0.0.1/tcp/31330/p2p/12D3KooWHkT1TjZRFKsw4e4xnAgoHMD7oenTPYZTv4kegid3w5Kb /ip4/127.0.0.1/udp/31330/quic/p2p/12D3KooWHkT1TjZRFKsw4e4xnAgoHMD7oenTPYZTv4kegid3w5Kb

Note

--tcp_public_ip and --tcp_port are not used in Hypertensor Testnet v1.0. These are designated for the later versions for validating peer model hosting called by the Hypertensor blockchain off-chain workers.


Note

This should begin loading blocks (not to be confused with the blockchain blocks) within 5-10 minutes or less. If it doesn't output logs such as Loaded bigscience/bloom-560m block 1 within 5-10 minutes, try restarting your server or computer. Once these logs begin appearing, it can take up to 30 minutes to complete.

Concluding

Once you have successfully started your node and contributed your GPU to the AI model, the substrate model validator data should automatically update the model and peer data, such as the model ID, model path, initialization block, peer ID, port, IP, and model peer initialization block. This data is stored locally in the root. This data can be queried through run_get_peer and run_get_network_config. You can then begin to add your model validator data from the CLI.

Last updated