Recasting Regional Lighting for Shadow Removal

AAAI 2024 (Oral)

Yuhao Liu1, Zhanghan Ke1, Ke Xu1, Fang Liu1, Zhenwei Wang1, Rynson W.H. Lau1
1City University of Hong Kong
arXiv

Method Overview. Given a shadow image I_s and a shadow mask I_m as input, the proposed method first decomposes the shadow image into a reflectance layer R_s and an illumination layer L_s via the shadow-aware decomposition network. R_s, L_s, and image features through skip-connections are then fed into the bilateral correction network for lighting correction via the Local Lighting Correction (LLC) module to generate the shadow-free lighting L̂_s, and texture restoration via the Illumination-Guided Texture Restoration (IGTR) module, and output the prediction Î.

Abstract

Shadow removal aims to restore the occluded area by shadows to have similar illumination as the non-shadow area. The key challenge is to harmonize the lighting in the shadow region with the rest of the image. Existing shadow removal methods often cast this as an image-to-image translation problem, without explicitly modeling the lighting. Recent methods tried to address this by estimating a single global lighting parameter or by decomposing an image into reflectance and illumination layers. However, they often fail to handle complex real-world shadows due to the locally varying lighting conditions. In this work, we propose a new deep learning approach for shadow removal. The key idea is to recast the shadow region by recovering locally appropriate lighting, referred to as regional lighting. Our method consists of two sub-networks: a shadow-aware decomposition network that decomposes the input image into reflectance and illumination layers, and a bilateral correction network that refines both layers. Extensive experiments show that our method achieves better results compared to state-of-the-art shadow removal methods.

Method

Detailed architecture of our RRL-Net, showing the shadow-aware decomposition network and bilateral correction network.

Results

Qualitative comparison with state-of-the-art shadow removal methods. Our method produces more natural shadow-free results.

BibTeX

@inproceedings{liu2024recasting,
  title={Recasting Regional Lighting for Shadow Removal},
  author={Liu, Yuhao and Ke, Zhanghan and Xu, Ke and Liu, Fang and Wang, Zhenwei and Lau, Rynson W.H.},
  booktitle={Proceedings of the AAAI Conference on Artificial Intelligence},
  year={2024}
}