Skip to main content

LUP Student Papers

LUND UNIVERSITY LIBRARIES

Temporal Artefact Mitigation in Rolling Shutter Videography

Wallinius, Frans LU and Remnélius Bou, Marcus LU (2024) In CODEN:LUTEDX/TEIE EIEL05 20241
Industrial Electrical Engineering and Automation
Abstract
Energy efficiency is key in our modern society, and in its pursuit, unexpected problems can appear. For instance, when Axis Communications in the pursuit of image quality and energy efficiency used strobing IR-illumination to illuminate for their global shutter cameras, they observed an artefact in the form of a white band in their rolling shutter cameras. The purpose of this thesis is to find a software solution to this problem with which to update existing Axis rolling shutter cameras with.
Rolling shutter cameras capture images by exposing its sensor to light and scanning its pixels row by row. This can therefore be affected by flashing lights where only some of the rows are exposed to the light which results in a white band in the... (More)
Energy efficiency is key in our modern society, and in its pursuit, unexpected problems can appear. For instance, when Axis Communications in the pursuit of image quality and energy efficiency used strobing IR-illumination to illuminate for their global shutter cameras, they observed an artefact in the form of a white band in their rolling shutter cameras. The purpose of this thesis is to find a software solution to this problem with which to update existing Axis rolling shutter cameras with.
Rolling shutter cameras capture images by exposing its sensor to light and scanning its pixels row by row. This can therefore be affected by flashing lights where only some of the rows are exposed to the light which results in a white band in the image. The PIE-team at Axis developed a solution that detected the band and found its position in the image from frame to frame. This was then used to calculate the strobe frequency which was used to synchronize the frame rate. When the band was stable in the image, a delay was applied to the exposure to capture images when the strobe is off. Unfortunately, the frame rate could never exactly match the frequency due to discretization, which resulted in the strobe eventually reappearing.
Initially, the preexisting work was analysed and alterations to the detection algorithm was implemented to store a background without the strobe of which to compare the current frame to. If there was a difference higher than a threshold, the band was identified, and its centre was calculated. The band was then synchronized, and the exposure delayed as before. To tackle the strobe’s reappearance, another method was developed that checked the top and bottom row of the image to see if the strobe reappeared, and if it did, the exposure was delayed again. Testing was performed in both a laboratory and in the field with a stable and a less stable strobe trigger.
The results showed that the algorithm functioned as intended in both the laboratory and the field, with some limitations. In the controlled laboratory environment, the algorithm performed satisfactorily every run. How accurately the camera calculated the frame rate was tested. With the less stable strobe, the algorithm calculated a frame rate with a standard deviation of 0.00241 every run, and with the stable strobe, a standard deviation of 0.00032. In the field, many factors counteracted the functionality, but mostly how much light reached the camera.
The laboratory tests with the stable and less stable strobe revealed that even when a stable strobe is used, some frequency drift still occurs, confirming the need to reapply a delay. In the field, surrounding light sources and obstructing objects affected the band’s appearance at different parts of the image, which made it difficult to get predictable results, but when the strobe was visible enough, the algorithm worked well. In conclusion, an algorithm was developed that detect a strobe artefact in the form of a white band, eliminates it and keeps it eliminated. (Less)
Popular Abstract (Swedish)
Energieffektivitet är viktigt i vårt moderna samhälle och i jakten på det kan oväntade problem uppstå. När Axis Communications i strävan efter ökad bildkvalitet och energieffektivitet tillämpade IR-stroboskop med hög toppeffekt som belysning åt kameror med global slutare, observerades en artefakt i form av ett vitt band i närliggande kameror med rullande slutare. Syftet med detta examensarbete är att utveckla en mjukvarulösning för detta problem där befintliga kameror med rullande slutare kan uppdateras.
Eftersom en rullande slutare läser av sensorn rad för rad sekventiellt är de känsliga för blinkande ljus eftersom de rader som exponeras för ljuset leder till ett vitt band i den slutliga bilden. PIE-gruppen på Axis utvecklade en lösning... (More)
Energieffektivitet är viktigt i vårt moderna samhälle och i jakten på det kan oväntade problem uppstå. När Axis Communications i strävan efter ökad bildkvalitet och energieffektivitet tillämpade IR-stroboskop med hög toppeffekt som belysning åt kameror med global slutare, observerades en artefakt i form av ett vitt band i närliggande kameror med rullande slutare. Syftet med detta examensarbete är att utveckla en mjukvarulösning för detta problem där befintliga kameror med rullande slutare kan uppdateras.
Eftersom en rullande slutare läser av sensorn rad för rad sekventiellt är de känsliga för blinkande ljus eftersom de rader som exponeras för ljuset leder till ett vitt band i den slutliga bilden. PIE-gruppen på Axis utvecklade en lösning som detekterade artefakten och följde dess position i bild mellan exponeringar. Stroboskopets frekvens kunde därefter beräknas och kamerans bildfrekvens synkroniserades till denna. När artefakten var stabiliserad beräknades en fördröjning för att flytta nästkommande exponeringar till när stroboskopet inte lyste. Tyvärr kunde inte bildfrekvensen exakt matcha stroboskopets frekvens på grund av diskretisering. Detta resulterade i att artefakten dök upp igen.
Inledningsvis analyserades det befintliga arbetet och förändringar av detektionsalgoritmen bedömdes nödvändiga. Genom att identifiera en bakgrund och lagra denna kunde den aktuella bilden jämföras och en skillnad beräknas. Om denna skillnad på någon rad översteg ett gränsvärde var artefakten identifierad och kunde elimineras. För att hantera att artefakten återkommer i bild implementerades ytterligare en metod som övervakade den översta och den understa raden av bilden. När artefakten sedan återkom applicerades en ny fördröjning. Detta testades både laboratoriet och i fält.
Resultaten visar att algoritmen fungerar enligt målen i både laboratoriet och i fält med vissa begränsningar. I en kontrollerad laboratoriemiljö fungerade algoritmen tillfredställande vid varje tillfälle. Hur noga bildfrekvensen synkroniserades testades. Med ett instabilt stroboskop beräknades bildfrekvensen med en standardavvikelse på 0,00241 och med ett stabilt stroboskop uppnåddes en standardavvikelse på 0,00032. I fält påverkade en mängd faktorer resultaten men främst mängden ljus som nådde sensorn.
Laboratorietesterna avslöjade även frekvensdrift hos kamerorna vilket bekräftade behovet av att återapplicera fördröjningen. I fälttesterna påverkade ljuskällor och andra faktorer artefaktens profil vilket försvårade möjligheten att få förutsägbara resultat. Men när artefakten var tydlig nog att påverka bildkvaliteten fungerade algoritmerna som tänkt. Sammanfattningsvis utvecklades en algoritm som detekterar en artefakt från ett stroboskop, eliminerar den och håller den eliminerad (Less)
Please use this url to cite or link to this publication:
author
Wallinius, Frans LU and Remnélius Bou, Marcus LU
supervisor
organization
alternative title
Artefaktreducering i videografi med rullande slutare
course
EIEL05 20241
year
type
M2 - Bachelor Degree
subject
keywords
Rolling shutter, Global shutter, Artefact, Artifact, Artefact mitigation, Strobe, IR, Image analysis, Rullande slutare, Global slutare, Bildartefakt, Artefaktreducering, Stroboskop, Bildanalys
publication/series
CODEN:LUTEDX/TEIE
report number
3136
language
English
id
9169862
date added to LUP
2024-10-02 15:46:37
date last changed
2024-10-02 15:46:37
@misc{9169862,
  abstract     = {{Energy efficiency is key in our modern society, and in its pursuit, unexpected problems can appear. For instance, when Axis Communications in the pursuit of image quality and energy efficiency used strobing IR-illumination to illuminate for their global shutter cameras, they observed an artefact in the form of a white band in their rolling shutter cameras. The purpose of this thesis is to find a software solution to this problem with which to update existing Axis rolling shutter cameras with.
Rolling shutter cameras capture images by exposing its sensor to light and scanning its pixels row by row. This can therefore be affected by flashing lights where only some of the rows are exposed to the light which results in a white band in the image. The PIE-team at Axis developed a solution that detected the band and found its position in the image from frame to frame. This was then used to calculate the strobe frequency which was used to synchronize the frame rate. When the band was stable in the image, a delay was applied to the exposure to capture images when the strobe is off. Unfortunately, the frame rate could never exactly match the frequency due to discretization, which resulted in the strobe eventually reappearing.
Initially, the preexisting work was analysed and alterations to the detection algorithm was implemented to store a background without the strobe of which to compare the current frame to. If there was a difference higher than a threshold, the band was identified, and its centre was calculated. The band was then synchronized, and the exposure delayed as before. To tackle the strobe’s reappearance, another method was developed that checked the top and bottom row of the image to see if the strobe reappeared, and if it did, the exposure was delayed again. Testing was performed in both a laboratory and in the field with a stable and a less stable strobe trigger.
The results showed that the algorithm functioned as intended in both the laboratory and the field, with some limitations. In the controlled laboratory environment, the algorithm performed satisfactorily every run. How accurately the camera calculated the frame rate was tested. With the less stable strobe, the algorithm calculated a frame rate with a standard deviation of 0.00241 every run, and with the stable strobe, a standard deviation of 0.00032. In the field, many factors counteracted the functionality, but mostly how much light reached the camera. 
The laboratory tests with the stable and less stable strobe revealed that even when a stable strobe is used, some frequency drift still occurs, confirming the need to reapply a delay. In the field, surrounding light sources and obstructing objects affected the band’s appearance at different parts of the image, which made it difficult to get predictable results, but when the strobe was visible enough, the algorithm worked well. In conclusion, an algorithm was developed that detect a strobe artefact in the form of a white band, eliminates it and keeps it eliminated.}},
  author       = {{Wallinius, Frans and Remnélius Bou, Marcus}},
  language     = {{eng}},
  note         = {{Student Paper}},
  series       = {{CODEN:LUTEDX/TEIE}},
  title        = {{Temporal Artefact Mitigation in Rolling Shutter Videography}},
  year         = {{2024}},
}