Lighthouse Book

Documentation for Lighthouse users and developers.

Doc Status Chat Badge

Lighthouse is an Ethereum 2.0 client that connects to other Ethereum 2.0 clients to form a resilient and decentralized proof-of-stake blockchain.

We implement the specification as defined in the ethereum/eth2.0-specs repository.

Topics

You may read this book from start to finish, or jump to some of these topics:

Prospective contributors can read the Contributing section to understand how we develop and test Lighthouse.

About this Book

This book is open source, contribute at github.com/sigp/lighthouse/book.

The Lighthouse CI/CD system maintains a hosted version of the master branch at lighthouse-book.sigmaprime.io.

Command-Line Interface (CLI)

Lighthouse a collection of CLI applications. The two primary binaries are:

  • beacon_node: the largest and most fundamental component which connects to the p2p network, processes messages and tracks the head of the beacon chain.
  • validator_client: a lightweight but important component which loads a validators private key and signs messages using a beacon_node as a source-of-truth.

There are also some ancillary binaries:

  • account_manager: generates cryptographic keys.
  • lcli: a general-purpose utility for troubleshooting Lighthouse state transitions (developer tool).

Installation

Presently, we recommend building Lighthouse using the $ cargo build --release --all command and executing binaries from the <lighthouse-repository>/target/release directory.

Documentation

Each binary supports the --help flag, this is the best source of documentation.

$ ./beacon_node --help
$ ./validator_client --help
$ ./account_manager --help
$ ./lcli --help

Beacon Node

The beacon_node CLI has two primary tasks:

  • Resuming an existing database with $ ./beacon_node.
  • Creating a new testnet database using $ ./beacon_node testnet.

Creating a new database

Use the $./beacon_node testnet command (see testnets for more information).

Resuming from an existing database

Once a database has been created, it can be resumed by running $ ./beacon_node.

Presently, this command will fail if no existing database is found. You must use the $ ./beacon_node testnet command to create a new database.

Testnets

The Lighthouse CLI has a testnet sub-command to allow creating or connecting to Eth2 beacon chain testnets.

For detailed documentation, use the --help flag on the CLI:

$ ./beacon_node testnet --help
$ ./validator_client testnet --help

Examples

All examples assume a working development environment and commands are based in the target/release directory (this is the build dir for cargo).

Start a beacon node given a validator count and genesis_time

To start a brand-new beacon node (with no history) use:

$ ./beacon_node testnet -f quick 8 <GENESIS_TIME>

Where GENESIS_TIME is in unix time.

Notes:

  • This method conforms the "Quick-start genesis" method in the ethereum/eth2.0-pm repository.
  • The -f flag ignores any existing database or configuration, backing them up before re-initializing.
  • 8 is the validator count and 1567222226 is the genesis time.
  • See $ ./beacon_node testnet quick --help for more configuration options.

Start a beacon node given a genesis state file

A genesis state can be read from file using the testnet file subcommand. There are three supported formats:

  • ssz (default)
  • json
  • yaml

Start a new node using /tmp/genesis.ssz as the genesis state:

$ ./beacon_node testnet --spec minimal -f file ssz /tmp/genesis.ssz

Notes:

  • The -f flag ignores any existing database or configuration, backing them up before re-initializing.
  • See $ ./beacon_node testnet file --help for more configuration options.
  • The --spec flag is required to allow SSZ parsing of fixed-length lists. Here the minimal eth2 specification is chosen, allowing for lower validator counts. See eth2.0-specs/configs for more info.

Start an auto-configured validator client

To start a brand-new validator client (with no history) use:

$ ./validator_client testnet -b insecure 0 8

Notes:

  • The -b flag means the validator client will "bootstrap" specs and config from the beacon node.
  • The insecure command dictates that the interop keypairs will be used.
  • The 0 8 indicates that this validator client should manage 8 validators, starting at validator 0 (the first deposited validator).
  • The validator client will try to connect to the beacon node at localhost. See --help to configure that address and other features.
  • The validator client will operate very unsafely in testnet mode, happily swapping between chains and creating double-votes.

Exporting a genesis file

Genesis states can downloaded from a running Lighthouse node via the HTTP API. Three content-types are supported:

  • application/json
  • application/yaml
  • application/ssz

Using curl, a genesis state can be downloaded to /tmp/genesis.ssz:

$ curl --header "Content-Type: application/ssz" "localhost:5052/beacon/state/genesis" -o /tmp/genesis.ssz

Advanced

Below are some CLI commands useful when working with testnets.

Specify a boot node by multiaddr

You can specify a static list of multiaddrs when booting Lighthouse using the --libp2p-addresses command.

Example:

$ ./beacon_node --libp2p-addresses /ip4/192.168.0.1/tcp/9000

Specify a boot node by ENR (Ethereum Name Record)

You can specify a static list of Discv5 addresses when booting Lighthouse using the --boot-nodes command.

Example:

$ ./beacon_node --boot-nodes -IW4QB2Hi8TPuEzQ41Cdf1r2AUU1FFVFDBJdJyOkWk2qXpZfFZQy2YnJIyoT_5fnbtrXUouoskmydZl4pIg90clIkYUDgmlwhH8AAAGDdGNwgiMog3VkcIIjKIlzZWNwMjU2azGhAjg0-DsTkQynhJCRnLLttBK1RS78lmUkLa-wgzAi-Ob5

Avoid port clashes when starting nodes

Starting a second Lighthouse node on the same machine will fail due to TCP/UDP port collisions. Use the -b (--port-bump) flag to increase all listening ports by some n.

Example:

Increase all ports by 10 (using multiples of 10 is recommended).

$ ./beacon_node -b 10

Start a testnet with a custom slot time

Lighthouse can run at quite low slot times when there are few validators (e.g., 500 ms slot times should be fine for 8 validators).

Example:

The -t (--slot-time) flag specifies the milliseconds per slot.

$ ./beacon_node testnet -t 500 recent 8

Note: bootstrap loads the slot time via HTTP and therefore conflicts with this flag.

Simple Local Testnet

With a functional development environment, starting a local multi-node testnet is easy:

  1. Start the first node: $ ./beacon_node testnet -f recent 8
  2. Start a validator client: $ ./validator_client testnet -b insecure 0 8
  3. Start more nodes with $ ./beacon_node -b 10 testnet -f bootstrap http://localhost:5052
    • Increment the -b value by 10 for each additional node.

Detailed Instructions

First, setup a Lighthouse development environment and navigate to the target/release directory (this is where the binaries are located).

Starting a beacon node

Start a new node (creating a fresh database and configuration in ~/.lighthouse), using:

$ ./beacon_node testnet -f recent 8

Notes:

  • The -f flag ignores any existing database or configuration, backing them up before re-initializing.
  • 8 is number of validators with deposits in the genesis state.
  • See $ ./beacon_node testnet recent --help for more configuration options, including minimal/mainnet specification.

Starting a validator client

In a new terminal window, start the validator client with:

$ ./validator_client testnet -b insecure 0 8

Notes:

  • The -b flag means the validator client will "bootstrap" specs and config from the beacon node.
  • The insecure command uses predictable, well-known private keys. Since this is just a local testnet, these are fine.
  • The 0 8 indicates that this validator client should manage 8 validators, starting at validator 0 (the first deposited validator).
  • The validator client will try to connect to the beacon node at localhost. See --help to configure that address and other features.

Adding another beacon node

You may connect another (non-validating) node to your local network using the lighthouse bootstrap command.

In a new terminal window, run:

$ ./beacon_node -b 10 testnet -r bootstrap

Notes:

  • The -b (or --port-bump) increases all the listening TCP/UDP ports of the new node to 10 higher. Your first node's HTTP server was at TCP 5052 but this one will be at 5062.
  • The -r flag creates a new data directory with a random string appended (avoids data directory collisions between nodes).
  • The default bootstrap HTTP address is http://localhost:5052. The new node will download configuration via HTTP before starting sync via libp2p.
  • See $ ./beacon_node testnet bootstrap --help for more configuration.

APIs

The Lighthouse beacon_node provides two APIs for local consumption:

Security

These endpoints are not designed to be exposed to the public Internet or untrusted users. They may pose a considerable DoS attack vector when used improperly.

HTTP API

Swagger Badge

The Lighthouse API is documented in Open API format and is available at SwaggerHub: Lighthouse REST API.

By default, a Lighthouse beacon_node exposes a HTTP server on localhost:5052.

The following CLI flags control the HTTP server:

  • --no-api: disable the HTTP server.
  • --api-port: specify the listen port of the server.
  • --api-address: specify the listen address of the server.

Examples

In addition to the complete Open API docs (see above), some examples are provided below.

Examples assume there is a Lighthouse node exposing a HTTP API on localhost:5052. Responses are JSON.

Get the node's beacon chain head

$ curl localhost:5052/beacon/head

{"slot":0,"block_root":"0x827bf71805540aa13f6d8c7d18b41b287b2094a4d7a28cbb8deb061dbf5df4f5","state_root":"0x90a78d73294bc9c7519a64e1912161be0e823eb472012ff54204e15a4d717fa5"}%

Get the node's finalized checkpoint

$ curl localhost:5052/beacon/latest_finalized_checkpoint

{"epoch":0,"root":"0x0000000000000000000000000000000000000000000000000000000000000000"}%

Get the node's ENR

$ curl localhost:5052/network/enr

"-IW4QFyf1VlY5pZs0xZuvKMRZ9_cdl9WMCDAAJXZiZiuGcfRYoU40VPrYDLQj5prneJIz3zcbTjHp9BbThc-yiymJO8HgmlwhH8AAAGDdGNwgiMog3VkcIIjKIlzZWNwMjU2azGhAjg0-DsTkQynhJCRnLLttBK1RS78lmUkLa-wgzAi-Ob5"%

Get a list of connected peer ids

$ curl localhost:5052/network/peers

["QmeMFRTWfo3KbVG7dEBXGhyRMa29yfmnJBXW84rKuGEhuL"]%

Get the node's peer id

$ curl localhost:5052/network/peer_id

"QmRD1qs2AqNNRdBcGHUGpUGkpih5cmdL32mhh22Sy79xsJ"%

Get the list of listening libp2p addresses

Lists all the libp2p multiaddrs that the node is listening on.

$ curl localhost:5052/network/listen_addresses

["/ip4/127.0.0.1/tcp/9000","/ip4/192.168.1.121/tcp/9000","/ip4/172.17.0.1/tcp/9000","/ip4/172.42.0.1/tcp/9000","/ip6/::1/tcp/9000","/ip6/fdd3:c293:1bc::203/tcp/9000","/ip6/fdd3:c293:1bc:0:9aa9:b2ea:c610:44db/tcp/9000"]%

Pretty-print the genesis state and state root

Returns the genesis state and state root in your terminal, in YAML.

$ curl --header "Content-Type: application/yaml" "localhost:5052/beacon/state?slot=0"

Websocket API

By default, a Lighthouse beacon_node exposes a websocket server on localhost:5053.

The following CLI flags control the websocket server:

  • --no-ws: disable the websocket server.
  • --ws-port: specify the listen port of the server.
  • --ws-address: specify the listen address of the server.

All clients connected to the websocket server will receive the same stream of events, all triggered by the BeaconChain. Each event is a JSON object with the following schema:

{
    "event": "string",
    "data": "object"
}

Events

The following events may be emitted:

Beacon Head Changed

Occurs whenever the canonical head of the beacon chain changes.

{
    "event": "beacon_head_changed",
    "data": {
        "reorg": "boolean",
        "current_head_beacon_block_root": "string",
        "previous_head_beacon_block_root": "string"
    }
}

Beacon Finalization

Occurs whenever the finalized checkpoint of the canonical head changes.

{
    "event": "beacon_finalization",
    "data": {
        "epoch": "number",
        "root": "string"
    }
}

Beacon Block Imported

Occurs whenever the beacon node imports a valid block.

{
    "event": "beacon_block_imported",
    "data": {
        "block": "object"
    }
}

Beacon Block Rejected

Occurs whenever the beacon node rejects a block because it is invalid or an error occurred during validation.

{
    "event": "beacon_block_rejected",
    "data": {
        "reason": "string",
        "block": "object"
    }
}

Beacon Attestation Imported

Occurs whenever the beacon node imports a valid attestation.

{
    "event": "beacon_attestation_imported",
    "data": {
        "attestation": "object"
    }
}

Beacon Attestation Rejected

Occurs whenever the beacon node rejects an attestation because it is invalid or an error occurred during validation.

{
    "event": "beacon_attestation_rejected",
    "data": {
        "reason": "string",
        "attestation": "object"
    }
}

Contributing to Lighthouse

Chat Badge

Lighthouse welcomes contributions. If you are interested in contributing to the Ethereum ecosystem, and you want to learn Rust, Lighthouse is a great project to work on.

To start contributing,

  1. Setup a development environment.
  2. Browse through the open issues (tip: look for the good first issue tag).
  3. Comment on an issue before starting work.
  4. Share your work via a pull-request.

If you have questions, please reach out via Discord.

Ethereum 2.0

Lighthouse is an implementation of the Ethereum 2.0 specification, as defined in the ethereum/eth2.0-specs repository.

We recommend reading Danny Ryan's (incomplete) Phase 0 for Humans before diving into the canonical spec.

Rust

Lighthouse adheres to Rust code conventions as outlined in the Rust Styleguide.

Please use clippy and rustfmt to detect common mistakes and inconsistent code formatting:

$ cargo clippy --all
$ cargo fmt --all --check

Panics

Generally, panics should be avoided at all costs. Lighthouse operates in an adversarial environment (the Internet) and it's a severe vulnerability if people on the Internet can cause Lighthouse to crash via a panic.

Always prefer returning a Result or Option over causing a panic. For example, prefer array.get(1)? over array[1].

If you know there won't be a panic but can't express that to the compiler, use .expect("Helpful message") instead of .unwrap(). Always provide detailed reasoning in a nearby comment when making assumptions about panics.

TODOs

All TODO statements should be accompanied by a GitHub issue.


# #![allow(unused_variables)]
#fn main() {
pub fn my_function(&mut self, _something &[u8]) -> Result<String, Error> {
  // TODO: something_here
  // https://github.com/sigp/lighthouse/issues/XX
}
#}

Comments

General Comments

  • Prefer line (//) comments to block comments (/* ... */)
  • Comments can appear on the line prior to the item or after a trailing space.

# #![allow(unused_variables)]
#fn main() {
// Comment for this struct
struct Lighthouse {}
fn make_blockchain() {} // A comment on the same line after a space
#}

Doc Comments

  • The /// is used to generate comments for Docs.
  • The comments should come before attributes.

# #![allow(unused_variables)]
#fn main() {
/// Stores the core configuration for this Lighthouse instance.
/// This struct is general, other components may implement more
/// specialized config structs.
#[derive(Clone)]
pub struct LighthouseConfig {
    pub data_dir: PathBuf,
    pub p2p_listen_port: u16,
}
#}

Rust Resources

Rust is an extremely powerful, low-level programming language that provides freedom and performance to create powerful projects. The Rust Book provides insight into the Rust language and some of the coding style to follow (As well as acting as a great introduction and tutorial for the language).

Rust has a steep learning curve, but there are many resources to help. We suggest:

Development Environment Setup

Linux, MacOS & Windows

  1. Install Rust and Cargo with rustup.
    • Use the stable toolchain (it's the default).
  2. Install build dependencies using your package manager.
    • clang, protobuf, libssl-dev, cmake
  3. Clone the github.com/sigp/lighthouse repository.
  4. Run $ make to build Lighthouse.
  5. Run $ make test to run the test suite
    • If you experience any failures, please reach out on discord.
    • Developers use $ make test-full to ensure you have the full set of test vectors.
  • The beacon_node, validator_client and other binaries are created in target/release directory.
  • First-time compilation may take several minutes.

Windows

Perl may also be required to build Lighthouse. You can install Strawberry Perl, or alternatively if you're using the Chocolatey package manager for Windows, use the following choco install command: choco install strawberryperl.

Additionally, the dependency protoc-grpcio v0.3.1 is reported to have issues compiling in Windows. You can specify a known working version by editing version in protos/Cargo.toml section to protoc-grpcio = "<=0.3.0".

Contiguous Integration (CI) and Testing

Lighthouse uses a self-hosted Gitlab CI server to run tests and deploy docs.

For security reasons, CI will only be run automatically for Lighthouse maintainers. Contributors without maintainer privileges will need to have CI triggered for them prior to a PR being merged.

You can see the full set of tests we run in the gitlab-ci.yml file. The following two commands should complete successfully before CI can pass:

$ cargo test --all --all-features
$ cargo fmt --all --check

Note: Travis CI is also used, however it does not run the full test suite.

Ethereum 2.0 Spec Tests

The ethereum/eth2.0-spec-tests repository contains a large set of tests that verify Lighthouse behaviour against the Ethereum Foundation specifications.

These tests are quite large (100's of MB), so we don't download them by default. Developers should ensure they have downloaded these tests using the Makefile in tests/ef_tests.

Failures in these tests should prevent CI from passing.