Skip to main content

LUP Student Papers

LUND UNIVERSITY LIBRARIES

Real-world low-light image enhancement using Variational Autoencoders

Eriksson, Olle LU (2020) In Master's Theses in Mathematical Sciences FMAM05 20201
Mathematics (Faculty of Engineering)
Abstract
Low-light image enhancement is a hard task mainly due to the amount of noise and little information stored in the dark image. In this thesis project we develop a method for low-light image enhancement based on a Conditional Variational Autoencoder (CVAE). The CVAE is a deep learning model trained with a specific objective function (ELBO). A CVAE can be implemented as a variant of a U-Net utilizing skip connections. We train models on a dataset called `See in the dark' (SID), which contains aligned dark-bright image pairs. This thesis is mainly focusing on building models which accept an unprocessed RAW image input and predict a noise-free bright color image. Additionally we also consider alternative input data (sRGB instead of RAW) and the... (More)
Low-light image enhancement is a hard task mainly due to the amount of noise and little information stored in the dark image. In this thesis project we develop a method for low-light image enhancement based on a Conditional Variational Autoencoder (CVAE). The CVAE is a deep learning model trained with a specific objective function (ELBO). A CVAE can be implemented as a variant of a U-Net utilizing skip connections. We train models on a dataset called `See in the dark' (SID), which contains aligned dark-bright image pairs. This thesis is mainly focusing on building models which accept an unprocessed RAW image input and predict a noise-free bright color image. Additionally we also consider alternative input data (sRGB instead of RAW) and the task of predicting a grayscale image (instead of color). Different neural-network architectures are implemented and compared, so are different strategies for merging image-patches. Different variants of the CVAE are compared, in particular a model only using a reconstruction loss is considered.

Regarding the different tasks the results turn out as expected. Models with RAW input data perform better than the model with sRGB input. Models which predict color perform worse than the one predicting only luminance. The output images for the model predicting only grayscale have a bit more detail and have less artifacts. The specific type of interpolation (between predicted patches) is of significant importance in case the inputs are extremely noisy, otherwise the faster methods are good enough. Comparison of different types of CVAE models with the model trained only using reconstruction loss shows that the very type of loss-function is not of great importance for the results, but the CVAE models are holding up well. The overall results are rather a function of the network architecture and the type of input data. Overall we conclude that the CVAE model can successfully be applied to the task of real-world low-light image enhancement, for visual improvement, in case the input is not too noisy. In case of an extremely noisy input the results are not as good. (Less)
Please use this url to cite or link to this publication:
author
Eriksson, Olle LU
supervisor
organization
course
FMAM05 20201
year
type
H2 - Master's Degree (Two Years)
subject
keywords
machine learning, deep learning, U-Net, VAE, variational autoencoder, image processing, low-light vision
publication/series
Master's Theses in Mathematical Sciences
report number
2020:E75
ISSN
1404-6342
other publication id
LUTFMA-3432-2020
language
English
id
9030626
date added to LUP
2020-10-23 17:53:32
date last changed
2020-10-23 17:53:32
@misc{9030626,
  abstract     = {{Low-light image enhancement is a hard task mainly due to the amount of noise and little information stored in the dark image. In this thesis project we develop a method for low-light image enhancement based on a Conditional Variational Autoencoder (CVAE). The CVAE is a deep learning model trained with a specific objective function (ELBO). A CVAE can be implemented as a variant of a U-Net utilizing skip connections. We train models on a dataset called `See in the dark' (SID), which contains aligned dark-bright image pairs. This thesis is mainly focusing on building models which accept an unprocessed RAW image input and predict a noise-free bright color image. Additionally we also consider alternative input data (sRGB instead of RAW) and the task of predicting a grayscale image (instead of color). Different neural-network architectures are implemented and compared, so are different strategies for merging image-patches. Different variants of the CVAE are compared, in particular a model only using a reconstruction loss is considered.

Regarding the different tasks the results turn out as expected. Models with RAW input data perform better than the model with sRGB input. Models which predict color perform worse than the one predicting only luminance. The output images for the model predicting only grayscale have a bit more detail and have less artifacts. The specific type of interpolation (between predicted patches) is of significant importance in case the inputs are extremely noisy, otherwise the faster methods are good enough. Comparison of different types of CVAE models with the model trained only using reconstruction loss shows that the very type of loss-function is not of great importance for the results, but the CVAE models are holding up well. The overall results are rather a function of the network architecture and the type of input data. Overall we conclude that the CVAE model can successfully be applied to the task of real-world low-light image enhancement, for visual improvement, in case the input is not too noisy. In case of an extremely noisy input the results are not as good.}},
  author       = {{Eriksson, Olle}},
  issn         = {{1404-6342}},
  language     = {{eng}},
  note         = {{Student Paper}},
  series       = {{Master's Theses in Mathematical Sciences}},
  title        = {{Real-world low-light image enhancement using Variational Autoencoders}},
  year         = {{2020}},
}