Struct JsonlPipeline 

pub struct JsonlPipeline<I>
where
    I: ScrapedItem,
{ /* private fields */ }

JSON Lines output pipeline. A pipeline that writes each scraped item to a JSON Lines (.jsonl) file. Each item is written as a JSON object on a new line.

Implementations

impl<I> JsonlPipeline<I>

where I: ScrapedItem,

pub fn new( file_path: impl AsRef<Path>, ) -> Result<JsonlPipeline<I>, PipelineError>

Creates a new JsonlPipeline that writes to the specified file path.

Errors

Returns an error when the output file cannot be opened or the parent directory cannot be created.

pub fn with_schema_export_config( self, config: SchemaExportConfig, ) -> JsonlPipeline<I>

Applies typed export mapping before values are written.

Trait Implementations

impl<I> Pipeline<I> for JsonlPipeline<I>

where I: ScrapedItem,

fn name(&self) -> &str

Returns the name of the pipeline.

fn process_item<'life0, 'async_trait>( &'life0 self, item: I, ) -> Pin<Box<dyn Future<Output = Result<Option<I>, PipelineError>> + Send + 'async_trait>>

where 'life0: 'async_trait, JsonlPipeline<I>: 'async_trait,

Processes a single scraped item.

fn close<'life0, 'async_trait>( &'life0 self, ) -> Pin<Box<dyn Future<Output = Result<(), PipelineError>> + Send + 'async_trait>>

where 'life0: 'async_trait, JsonlPipeline<I>: 'async_trait,

Called when the spider is closing.

fn get_state<'life0, 'async_trait>( &'life0 self, ) -> Pin<Box<dyn Future<Output = Result<Option<Value>, PipelineError>> + Send + 'async_trait>>

where 'life0: 'async_trait, Self: 'async_trait,

Returns the current state of the pipeline as a JSON value.

fn restore_state<'life0, 'async_trait>( &'life0 self, _state: Value, ) -> Pin<Box<dyn Future<Output = Result<(), PipelineError>> + Send + 'async_trait>>

where 'life0: 'async_trait, Self: 'async_trait,

Restores the pipeline’s state from a JSON value.