论文标题
使用剪切式转换和循环一致性的自我监督光场重建
Self-Supervised Light Field Reconstruction Using Shearlet Transform and Cycle Consistency
论文作者
论文摘要
使用剪切式变换（ST）的基于图像的渲染方法是最先进的光场（DSLF）重建方法之一。它通过迭代正则化算法在图像结构域中重建了面向平面图像（EPIP），该算法恢复了剪切域中的系数。因此，由于花在了数十个迭代的域转换上，ST方法往往很慢。为了克服这一局限性，这封信提出了一种新型的自我监督的DSLF重建方法，即Cyclest，该方法将ST和循环一致性应用于DSLF重建。具体而言，Cyclest由编码器 - 编码器网络和剩余学习策略组成，该策略使用EPI重建和循环一致性损失恢复了密集采样EPIP的剪切系数。此外，Cyclest是一种自我监督的方法，可以仅在稀疏采样的光场（SSLFS）上进行训练，带有小差异范围（$ \ leqslant $ 8像素）。从两个挑战性的现实世界光场数据集中，具有较大差异范围（16-32像素）的SSLF上的DSLF重建的实验结果证明了拟议的环les方法的有效性和效率。此外，至少在ST上可以实现〜9倍的速度。
The image-based rendering approach using Shearlet Transform (ST) is one of the state-of-the-art Densely-Sampled Light Field (DSLF) reconstruction methods. It reconstructs Epipolar-Plane Images (EPIs) in image domain via an iterative regularization algorithm restoring their coefficients in shearlet domain. Consequently, the ST method tends to be slow because of the time spent on domain transformations for dozens of iterations. To overcome this limitation, this letter proposes a novel self-supervised DSLF reconstruction method, CycleST, which applies ST and cycle consistency to DSLF reconstruction. Specifically, CycleST is composed of an encoder-decoder network and a residual learning strategy that restore the shearlet coefficients of densely-sampled EPIs using EPI reconstruction and cycle consistency losses. Besides, CycleST is a self-supervised approach that can be trained solely on Sparsely-Sampled Light Fields (SSLFs) with small disparity ranges ($\leqslant$ 8 pixels). Experimental results of DSLF reconstruction on SSLFs with large disparity ranges (16 - 32 pixels) from two challenging real-world light field datasets demonstrate the effectiveness and efficiency of the proposed CycleST method. Furthermore, CycleST achieves ~ 9x speedup over ST, at least.
