xdas.processing.process#
- xdas.processing.process(atom, data_loader, data_writer)[source]#
Execute a chunked processing pipeline.
- Parameters:
atom (callable) – The atomic operation to execute on each chunk of data.
data_loader (DataArrayLoader) – The data loader object that provides the chunks of data.
data_writer (DataArrayWriter) – The data writer object that writes the processed data.
- Returns:
result – The result of the processing pipeline.
- Return type:
Notes
This function executes a chunked processing pipeline by ingesting the data from the data_loader and flushing the processed data through the data_writer. It iterates over the chunks of data provided by the data_loader, applies the atom function to each chunk, and writes the processed data using the data_writer. The progress of the processing is monitored using a Monitor object.