# Visualization¶

MMFlow 1.x provides convenient ways for monitoring training status or visualizing data and model predictions.

## Training status Monitor¶

MMFlow 1.x uses TensorBoard to monitor training status.

### TensorBoard Configuration¶

Install TensorBoard following official instructions

pip install future tensorboard


Add TensorboardVisBackend in vis_backend of visualizer in configs/_base_/default_runtime.py:

vis_backends = [dict(type='LocalVisBackend'), dict(type='TensorboardVisBackend')]
visualizer = dict(
type='FlowLocalVisualizer', vis_backends=vis_backends, name='visualizer')


The configuration contains LocalVisBackend, which means the scalars during training will be stored locally as well.

### Examining scalars in TensorBoard¶

Launch training experiment e.g.

python tools/train.py configs/pwcnet/pwcnet_8xb1_slong_flyingchairs-384x448.py --work-dir work_dirs/test_visual


You can specify the save_dir in visualizer to modify the storage path. The default storage path is vis_data under your work_dir. For example, the vis_data path of a particular experiment is

work_dirs/test_visual/20220831_165919/vis_data


The scalar file in vis_data includes learning rate, losses and data_time etc, and also record metrics results during evaluation. You can refer to logging tutorial in mmengine to log custom data. The TensorBoard visualization results are executed with the following command:

tensorboard --logdir work_dirs/test_visual/20220831_165919/vis_data


## Prediction Visualization¶

MMFlow provides FlowVisualizationHook that can render optical flow of ground truth and prediction. Users can modify visualization in default_hooks to invoke the hook. MMFlow configures default_hooks in each file under configs/_base_/schedules. For example, in configs/_base_/schedules/schedules_s_long.py, let’s modify the FlowVisualizationHook related parameters. Set draw to True to enable the storage of network inference results. interval indicates the sampling interval of the predicted results, defaults to 50, and when set to 1, each inference result of the network will be saved.

default_hooks = dict(
timer=dict(type='IterTimerHook'),
logger=dict(type='LoggerHook', interval=50, log_metric_by_epoch=False),
param_scheduler=dict(type='ParamSchedulerHook'),
checkpoint=dict(type='CheckpointHook', interval=100000, by_epoch=False),
sampler_seed=dict(type='DistSamplerSeedHook'),
visualization=dict(type='FlowVisualizationHook', draw=True, interval=1))


There is a way not to change the files under configs/_base_. For example, in configs/pwcnet/pwcnet_8xb1_slong_flyingchairs-384x448.py inherited from configs/_base_/schedules/schedules_s_long.py, just add visualization field in this way:

default_hooks = dict(
visualization=dict(type='FlowVisualizationHook', draw=True, interval=1))


Additionally, if you want to keep the original file under configs unchanged, you can specify --cfg-options in commands by referring to this guide.

python tools/test.py \
configs/pwcnet/pwcnet_8xb1_slong_flyingchairs-384x448.py \
--work-dir work_dirs/test_visual \
--cfg-options default_hooks.visualization.draw=True default_hooks.visualization.interval=1


The default backend of visualization is LocalVisBackend, which means storing the visualization results locally. Backend related configuration is in configs/_base_/default_runtime.py. In order to enable TensorBoard visualization as well, modify the visulizer just as this configuration. Assume the vis_data path of a particular test is

work_dirs/test_visual/20220831_114424/vis_data


The stored results of the local visualization are kept in vis_image under vis_data, while the TensorBoard visualization results can be executed with the following command:

tensorboard --logdir work_dirs/test_visual/20220831_114424/vis_data


The visualization image consists of two parts, the ground truth on the left and the network prediction result on the right.

If you would like to know more visualization usage, you can refer to visualization tutorial in mmengie.