Skip to main content
For how op-conductor works and the guarantees it provides, see OP Conductor. The flags and RPC methods used below are catalogued in the configuration and RPC reference. At OP Labs, op-conductor is deployed as a kubernetes statefulset because it requires a persistent volume to store the raft log. This guide describes setting up conductor on an existing network without incurring downtime. You can utilize the op-conductor-ops tool to confirm the conductor status between the steps.

​
Assumptions

This setup guide has the following assumptions:
  • 3 deployed sequencers (sequencer-0, sequencer-1, sequencer-2) that are all in sync and in the same vpc network
  • sequencer-0 is currently the active sequencer
  • You can execute a blue/green style sequencer deployment workflow that involves no downtime (described below)
  • conductor and sequencers are running in k8s or some other container orchestrator (vm-based deployment may be slightly different and not covered here)

​
Spin up op-conductor

1

Deploy conductor

Deploy a conductor instance per sequencer with sequencer-1 as the raft cluster bootstrap node:
  • suggested conductor configs:
  • sequencer-1 op-conductor extra config:
2

Pause two conductors

Pause sequencer-0 & sequencer-2 conductors with conductor_pause RPC request.
3

Update op-node configuration and switch the active sequencer

Deploy an op-node config update to all sequencers that enables conductor. Use a blue/green style deployment workflow that switches the active sequencer to sequencer-1:
  • all sequencer op-node configs:
4

Confirm sequencer switch was successful

Confirm sequencer-1 is active and successfully producing unsafe blocks. Because sequencer-1 was the raft cluster bootstrap node, it is now committing unsafe payloads to the raft log.
5

Add voting nodes

Add voting nodes to cluster using conductor_AddServerAsVoter RPC request to the leader conductor (sequencer-1)
6

Confirm state

Confirm cluster membership and sequencer state:
  • sequencer-0 and sequencer-2:
    1. raft cluster follower
    2. sequencer is stopped
    3. conductor is paused
    4. conductor enabled in op-node config
  • sequencer-1
    1. raft cluster leader
    2. sequencer is active
    3. conductor is paused
    4. conductor enabled in op-node config
7

Resume conductors

Resume all conductors with conductor_resume RPC request to each conductor instance.
8

Confirm state

Confirm all conductors successfully resumed with conductor_paused
9

Transfer leadership

Trigger leadership transfer to sequencer-0 using conductor_transferLeaderToServer
10

Confirm state

Confirm cluster membership and sequencer state:sequencer-1 and sequencer-2:
  1. raft cluster follower
  2. sequencer is stopped
  3. conductor is active
  4. conductor enabled in op-node config
sequencer-0:
  1. raft cluster leader
  2. sequencer is active
  3. conductor is active
  4. conductor enabled in op-node config
11

Update configuration

Deploy a config change to sequencer-1 conductor to remove the OP_CONDUCTOR_PAUSED: true flag and OP_CONDUCTOR_RAFT_BOOTSTRAP flag.

​
Blue/green deployment

In order to ensure there is no downtime when setting up conductor, you need to have a deployment script that can update sequencers without network downtime. An example of this workflow might look like:
  1. Query current state of the network and determine which sequencer is currently active (referred to as “original” sequencer below). From the other available sequencers, choose a candidate sequencer.
  2. Deploy the change to the candidate sequencer and then wait for it to sync up to the original sequencer’s unsafe head. You may want to check peer counts and other important health metrics.
  3. Stop the original sequencer using admin_stopSequencer which returns the last inserted unsafe block hash. Wait for candidate sequencer to sync with this returned hash in case there is a delta.
  4. Start the candidate sequencer at the original’s last inserted unsafe block hash.
    1. Here you can also execute additional check for unsafe head progression and decide to roll back the change (stop the candidate sequencer, start the original, rollback deployment of candidate, etc.)
  5. Deploy the change to the original sequencer, wait for it to sync to the chain head. Execute health checks.

​
Post-conductor launch deployments

After conductor is live, a similar canary style workflow is used to ensure minimal downtime in case there is an issue with deployment:
  1. Choose a candidate sequencer from the raft-cluster followers
  2. Deploy to the candidate sequencer. Run health checks on the candidate.
  3. Transfer leadership to the candidate sequencer using conductor_transferLeaderToServer. Run health checks on the candidate.
  4. Test if candidate is still the leader using conductor_leader after some grace period (ex: 30 seconds)
    1. If not, then there is likely an issue with the deployment. Roll back.
  5. Upgrade the remaining sequencers, run healthchecks.