NOVEL VIEW SYNTHESIS WITH SKIP CONNECTIONS

Abstract

Novel view synthesis is the task of synthesizing an image of an object at an arbitrary viewpoint given one or a few views of the object [1]. The output image of novel view synthesis exhibits a significant structural change from the input. Because of the large change, the skip connections or U-Net architecture, which can sustain the multi-level characteristics of the input images, cannot be directly utilized for the novel view synthesis [2]. In this paper, we investigate several variations of skip connection on two widely used novel view synthesis modules, pixel generation [1] and flow prediction [3]. For pixel generation, we find that the combination of the skip connections with flow-based hard attention is helpful. On the other hand, flow prediction enjoys marginal benefit from skip connections in deeper layers. Our pipeline suggests how to make use of skip connections on tasks that involve large geometric changes.