# Data Transforms¶

## Design of Data pipelines¶

Following typical conventions, we use Dataset and DataLoader for data loading with multiple workers. Dataset returns a dict of data items corresponding the arguments of models’ forward method. Since the data flow estimation may not be the same size, we introduce a new DataContainer type in MMCV to help collect and distribute data of different size. See here for more details.

The data preparation pipeline and the dataset is decomposed. Usually a dataset defines how to process the annotations and a data pipeline defines all the steps to prepare a data dict. A pipeline consists of a sequence of operations. Each operation takes a dict as input and also output a dict for the next transform.

Here is a pipeline example for PWC-Net training on FlyingChairs.

train_pipeline = [
dict(
type='ColorJitter',
brightness=0.5,
contrast=0.5,
saturation=0.5,
hue=0.5),
dict(type='RandomGamma', gamma_range=(0.7, 1.5)),
dict(type='RandomFlip', prob=0.5, direction='horizontal'),
dict(type='RandomFlip', prob=0.5, direction='vertical'),
dict(type='RandomAffine',
global_transform=dict(
translates=(0.05, 0.05),
zoom=(1.0, 1.5),
shear=(0.86, 1.16),
rotate=(-10., 10.)
),
relative_transform=)dict(
translates=(0.00375, 0.00375),
zoom=(0.985, 1.015),
shear=(1.0, 1.0),
rotate=(-1.0, 1.0)
),
dict(type='RandomCrop', crop_size=(384, 448)),
dict(type='PackFlowInputs')
]


For each operation, we list the related dict fields that are added/updated/removed. Before pipelines, the information we can directly obtain from the datasets are img1_path, img2_path and flow_fw_path.

LoadImageFromFile

• add: img1, img2, img_shape, ori_shape

LoadAnnotations

Note

FlyingChairs doesn’t provide the ground truth of backward flow, so gt_flow_bw is None. Besides, FlyingChairs’ ground truth is dense, so sparse is False. For some special datasets, such as HD1K and KITTI, their ground truth is sparse, so gt_valid_fw and gt_valid_bw will be added. FlyingChairsOcc and FlyingThing3d contain the ground truth of occlusion, so gt_occ_fw and gt_occ_bw will be added for these datasets. In the pipelines below, we only consider the case of FlyingChairs.

### Pre-processing¶

ColorJitter

• update: img1, img2

RandomGamma

• update: img1, img2

RandomFlip

• update: img1, img2, flow_gt

RandomAffine

• update: img1, img2, flow_gt

RandomCrop

PackFlowInputs
• remove: img1 and img2 (merged into inputs), keys specified by data_keys (like gt_flow_fw, merged into data_sample) keys specified by meta_keys (merged into the metainfo of data_sample), all other keys