Skip to main content
In the introduction to the derivation pipeline, the derivation pipeline is broken down to demonstrate the composition of stages, forming the transformation function from L1 data into L2 payload attributes. What makes kona’s derivation pipeline extensible is that stages are composed using trait-abstraction. That is, each successive stage composes the previous stage as a generic. As such as long as a stage satisfies two rules, it can be swapped into the pipeline seamlessly.
  1. The stage implements the trait required by the next stage.
  2. The stage uses the same trait for the previous stage as the current stage to be swapped out.
Below provides a concrete example, swapping out the L1Retrieval stage.

​
Example

In the current, post-Holocene hardfork DerivationPipeline, the bottom three stages of the pipeline are as follows (from top down). In this set of stages, the IndexedTraversal or PollingTraversal stage sits at the bottom. It implements the L1Retrieval trait called the L1RetrievalProvider. This provides generic methods that allow the L1Retrieval stage to call those methods on the generic previous stage that implements this provider trait. As we go up a level, the same trait abstraction occurs. The L1Retrieval stage implements the provider trait that the FrameQueue stage requires. This trait is the FrameQueueProvider. Now that we understand the trait abstractions, let’s swap out the L1Retrieval stage for a custom DapRetrieval stage. 
// ...
// imports
// ...

// We use the same "L1RetrievalProvider" trait here
// in order to seamlessly use the `IndexedTraversal` or `PollingTraversal`

/// DapRetrieval stage
#[derive(Debug)]
pub struct DapRetrieval<P>
where
    P: L1RetrievalProvider + OriginAdvancer + OriginProvider + SignalReceiver,
{
    /// The previous stage in the pipeline.
    pub prev: P,
    provider: YourDataAvailabilityProvider,
    data: Option<Bytes>,
}

#[async_trait]
impl<P> FrameQueueProvider for DapRetrieval<P>
where
    P: L1RetrievalProvider + OriginAdvancer + OriginProvider + SignalReceiver + Send,
{
    type Item = Bytes;

    async fn next_data(&mut self) -> PipelineResult<Self::Item> {
        if self.data.is_none() {
            let next = self
                .prev
                .next_l1_block()
                .await? // SAFETY: This question mark bubbles up the Eof error.
                .ok_or(PipelineError::MissingL1Data.temp())?;
            self.data = Some(self.provider.get_data(&next).await?);
        }

        match self.data.as_mut().expect("Cannot be None").next().await {
            Ok(data) => Ok(data),
            Err(e) => {
                if let PipelineErrorKind::Temporary(PipelineError::Eof) = e {
                    self.data = None;
                }
                Err(e)
            }
        }
    }
}

// ...
// impl OriginAdvancer for DapRetrieval
// impl OriginProvider for DapRetrieval
// impl SignalReceiver for DapRetrieval
// ..
Notice, the L1RetrievalProvider is used as a trait bound so the IndexedTraversal or PollingTraversal stage can be used seamlessly as the “prev” stage in the pipeline. Concretely, an instantiation of the DapRetrieval stage could be the following. 
DapRetrieval<IndexedTraversal<..>>