Depth-aware Neural Style Transfer using Instance Normalization

University of Sheffield

CGVC 2022

Our Results

Abstract

Neural Style Transfer (NST) is concerned with the artistic stylization of visual media. It can be described as the process of transferring the style of an artistic image onto an ordinary photograph. Recently, a number of studies have considered the enhancement of the depth-preserving capabilities of the NST algorithms to address the undesired effects that occur when the input content images include numerous objects at various depths. Our approach uses a deep residual convolutional network with instance normalization layers that utilizes an advanced depth prediction network to integrate depth preservation as an additional loss function to content and style. We demonstrate results that are effective in retaining the depth and global structure of content images. Three different evaluation processes show that our system is capable of preserving the structure of the stylized results while exhibiting style-capture capabilities and aesthetic qualities comparable or superior to state-of-the-art methods.

User Study

These are some examples of the User Study conducted to evaluate the results of our approach.


The first set of 15 questions (1-15) were shown to the participants without revealing the content and style images used to derive the results.



The second set of 15 questions (16-30) were shown to the participants along the content and style images used to derive the results.

BibTeX

@article{ioannou2022depth,
    title={Depth-aware Neural Style Transfer using Instance Normalization},
    author={Ioannou, Eleftherios and Maddock, Steve},
    journal={arXiv preprint arXiv:2203.09242},
    year={2022}
}