Automating Feature-Extraction for Camera Calibration Through Machine Learning and Computer Vision

Abstract

Machine learning as a field has expanded in an explosive manner, with
more companies interested in using the technology. One of these companies, Spiideo, uses Machine learning to automatically stream and record
sports, highlighting key events - all automatically without a cameraman.
However, these cameras have to undergo a lengthy calibration process
involving manual feature extraction. This work investigates the usage
of machine learning and computer vision to automate this work. In particular, both U-Net and DeepLab v3+ networks were trained on sets of
images and related data from previous feature extractions. From the
ML detected features, ridge detection and sub-pixel optimization was
used to remove outliers and for classification. The accuracy of the ML
and computer vision combination was compared to the manual feature
extraction, yielding similar results. The DeepLab v3+ network was
found to very accurately extract the intended features, leading to high
accuracy independent of camera position, camera angle or noise from
the stadium.

Keywords: Machine learning, computer vision, feature extraction, classification,
camera calibration, DeepLab, U-Net, ridge detection, sub-pixel optimization.

Popular Abstract

Teaching my computer to find the lines on a football field

The interest in sports is gigantic everywhere in the world, and the demand for
watching sports is increasing even more due to the Covid-19 pandemic. It does
not matter if it is watching professional teams, friends, your children or grandchildren playing: watching games live is not always easy - mostly professionals can be seen on TV!

Spiideo has one solution to this: streaming sports online without a camera man
by instead using “Artificial Intelligence”. This method teaches a computer to learn
human-like tasks. In this case, Spiideo trains a computer to act like a camera
man: e.g. following the ball during a football game and highlighting when goals
are scored. Their cameras do however, require a lengthy calibration to work as intended. Currently, this calibration is done manually and is both quite tedious and
time consuming. One of the main steps of this process is to click and save points
on the outermost lines on the pitch (from an image) for each camera. One solution to this was explored in this project. In collaboration with Spiideo, I created a
method using artificial intelligence, to find, mark and save lines on a football field
- removing the need for manual work.

One problem during the project was how football fields and arenas can look very
different: some have running tracks close by, some have advertisements in the
arena and some even has extra lines on the pitch! One way to work around this
problem was removing these "bad" points by a few so-called "computer vision
algorithms". These are mathematical models which helped by finding points that
seemed out of place!

The project eventually found a model that works: accurately finding and saving the lines on a football pitch after removing any outliers. There were some
problems when lines were not clearly visible - in one case the entire pitch was
covered in snow! However, in most cases the method was very accurate!

• author: Åkeborg, Elias
• supervisor: Mattias Borg
• organization: Department of Electrical and Information Technology
• course: EITM01 20211
• year: 2021
• type: H2 - Master's Degree (Two Years)
• subject: Technology and Engineering
• keywords: Machine learning, Computer vision, Feature-extraction, Classification, Camera Calibration, DeepLab, U-Net, Ridge Detection, Sub-pixel optimization, Artificial Intelligence
• report number: LU/LTH-EIT 2021-824
• language: English
• id: 9054961
• date added to LUP: 2021-06-23 10:51:14
• date last changed: 2021-06-23 10:51:14