v1.5.0, the Doppelganger Protection feature is available for the Validator
Client. Taken from the German doppelgänger, which translates literally to "double-walker", a
"doppelganger" in Eth2 refers to another instance of a validator running in a separate validator
process. As detailed in Slashing Protection, running the same validator twice will inevitably
result in slashing.
The Doppelganger Protection (DP) feature in Lighthouse imperfectly attempts to detect other instances of a validator operating on the network before any slashable offences can be committed. It achieves this by staying silent for 2-3 epochs after a validator is started so it can listen for other instances of that validator before starting to sign potentially slashable messages.
Note: Doppelganger Protection is not yet interoperable, so if it is configured on a Lighthouse validator client, the client must be connected to a Lighthouse beacon node. Because Infura uses Teku, Lighthouse's Doppelganger Protection cannot yet be used with Infura's Eth2 service.
There are two important initial considerations when using DP:
The mechanism is best-effort and imperfect. Even if another validator exists on the network, there is no guarantee that your Beacon Node (BN) will see messages from it. It is feasible for doppelganger protection to fail to detect another validator due to network faults or other common circumstances.
DP should be considered a last-line-of-defence that might save a validator from being slashed due to operator error (i.e. running two instances of the same validator). Users should never rely upon DP and should practice the same caution with regards to duplicating validators as if it did not exist.
Remember: even with doppelganger protection enabled, it is not safe to run two instances of the same validator.
DP works by staying silent on the network for 2-3 epochs before starting to sign slashable messages. Staying silent and refusing to sign messages will cause the following:
- 2-3 missed attestations, incurring penalties and missed rewards.
- 2-3 epochs of missed sync committee contributions (if the validator is in a sync committee, which is unlikely), incurring penalties and missed rewards (post-Altair upgrade only).
- Potentially missed rewards by missing a block proposal (if the validator is an elected block proposer, which is unlikely).
The loss of rewards and penalties incurred due to the missed duties will be very small in dollar-values. Generally, they will equate to around one US dollar (at August 2021 figures) or about 2% of the reward for one validator for one day. Since DP costs so little but can protect a user from slashing, many users will consider this a worthwhile trade-off.
The 2-3 epochs of missed duties will be incurred whenever the VC is started (e.g., after an update or reboot) or whenever a new validator is added via the VC HTTP API.
If you understand that DP is imperfect and will cause some (generally, non-substantial) missed
duties, it can be enabled by providing the
lighthouse vc --enable-doppelganger-protection
When enabled, the validator client will emit the following log on start up:
INFO Doppelganger detection service started service: doppelganger
Whilst DP is active, the following log will be emitted (this log indicates that one validator is staying silent and listening for validators):
INFO Listening for doppelgangers doppelganger_detecting_validators: 1, service: notifier
When a validator has completed DP without detecting a doppelganger, the following log will be emitted:
INFO Doppelganger protection complete validator_index: 42, msg: starting validator, service: notifier
If a doppelganger is detected, logs similar to those below will be emitted (these logs indicate that
the validator with the index
42 was found to have a doppelganger):
CRIT Doppelganger(s) detected doppelganger_indices: , msg: A doppelganger occurs when two different validator clients run the same public key. This validator client detected another instance of a local validator on the network and is shutting down to prevent potential slashable offences. Ensure that you are not running a duplicate or overlapping validator client, service: doppelganger INFO Internal shutdown received reason: Doppelganger detected. INFO Shutting down.. reason: Failure("Doppelganger detected.")
Observing a doppelganger is a serious problem and users should be very alarmed. The Lighthouse DP system tries very hard to avoid false-positives so it is likely that a slashing risk is present.
If a doppelganger is observed, the VC will shut down. Do not restart the VC until you are certain there is no other instance of that validator running elsewhere!
The steps to solving a doppelganger vary depending on the case, but some places to check are:
- Is there another validator process running on this host?
- Unix users can check
ps aux | grep lighthouse
- Windows users can check the Task Manager.
- Unix users can check
- Has this validator recently been moved from another host? Check to ensure it's not running.
- Has this validator been delegated to a staking service?
Yes, probably. If you don't have a clear and well considered reason not to use DP, then it is a good idea to err on the safe side.
DP takes 2-3 epochs, which is approximately 12-20 minutes.
To avoid false positives from restarting the same VC, Lighthouse will wait until the next epoch before it starts detecting doppelgangers. Additionally, a validator might not attest till the end of the next epoch. This creates a 2 epoch delay, which is just over 12 minutes. Network delays or issues might lengthen this time more.
This means your validator client might take up to 20 minutes to detect a doppelganger and shut down.
🙅 Absolutely not. 🙅 DP is imperfect and cannot be relied upon. The Internet is messy and lossy, there's no guarantee that DP will detect a duplicate validator before slashing conditions arise.