Future Frame Prediction with Generative Adversarial Networks
(2019) In Master's Thesis in Mathematical Sciences FMAM05 20191Mathematics (Faculty of Engineering)
- Abstract
- This report is about using generative adversarial networks with predictive coding networks for future frame prediction. Model selection choices for the components of the network are explored by training different models and testing their performance on next frame prediction in digital video from driving scenarios. Benefits and issues of using adversarial loss for future frame prediction as well as different choices for the model are discussed.
- Popular Abstract
- Deep learning methods can teach computers to make short-term predictions of the future. In this work a specific method for training has been tested to see how it performs on prediction problems.
The possibilities that come with the ability to teach computers to predict the future are many. One example is autonomous vehicles. Making autonomous vehicles such as self driving cars involves many challenges. Perhaps the most challenging aspect is that the vehicles must be equipped with the ability to understand complicated traffic situations, anticipate what will happen, and then act accordingly. This means that the cars must be designed to be good at predicting the future to overcome this challenge.
Another interesting application is... (More) - Deep learning methods can teach computers to make short-term predictions of the future. In this work a specific method for training has been tested to see how it performs on prediction problems.
The possibilities that come with the ability to teach computers to predict the future are many. One example is autonomous vehicles. Making autonomous vehicles such as self driving cars involves many challenges. Perhaps the most challenging aspect is that the vehicles must be equipped with the ability to understand complicated traffic situations, anticipate what will happen, and then act accordingly. This means that the cars must be designed to be good at predicting the future to overcome this challenge.
Another interesting application is humanrobot interaction. In eye-to-eye conversations between people, speech is not the only way that people communicate. Body language, facial expressions and other visual clues also play a part in the conversation. In particular we predict the different visual queues and mimic or otherwise adapt to what other people are doing. If robots are going to have natural communication with humans, then they must be able to read the body language and facial expressions in the same way
as people do.
Deep learning is a method for teaching computers relationships in data using an artificial neural network. Artificial neural networks are inspired by the neural networks in the human brain and use a mathematical model of the neurons to learn relationships between data. When using deep learning particular care has to be taken to choose something called loss function.
The loss function is a way for the network to know what errors it is making and learn from them. The simple loss functions compare the output of the network to the actual data and construct an error from that. In prediction problems there can be possible future outcomes that are not represented in the data, which the simple loss functions do not take into account. The network might make one possible prediction, but one that is different from the one represented in the data. Generative Adversarial Networks (GAN) can be used to mitigate this problem.
Generative Adversarial Networks is a method where, instead of using traditional loss funtions, another network is used to represent the error of the first network. The two networks learn in parallel with adverse objectives. The goal for the adverse network, usually called the discriminator, is to distinguish between the predictions made by the generator network and the real data samples, while the generator network is learning to make predictions that can fool the discriminator network as real samples. It is a game between the networks and when the discriminator network improves, so must also the generator network improve to keep up.
We cannot make computers predict when we will be driven by autonomous vehicles and being served by humanoid robots in restaurants, but they can play a role in making it happen. (Less)
Please use this url to cite or link to this publication:
http://lup.lub.lu.se/student-papers/record/8974637
- author
- Wallgren, Magnus LU
- supervisor
- organization
- course
- FMAM05 20191
- year
- 2019
- type
- H2 - Master's Degree (Two Years)
- subject
- keywords
- Generative Adversarial Networks, GAN, Machine Learning, Deep Learning, Future Frame Prediction, WGAN
- publication/series
- Master's Thesis in Mathematical Sciences
- report number
- LUTFMA-3378-2019
- ISSN
- 1404-6342
- other publication id
- 2019:E16
- language
- English
- id
- 8974637
- date added to LUP
- 2019-07-15 10:43:38
- date last changed
- 2019-07-15 11:07:59
@misc{8974637,
abstract = {{This report is about using generative adversarial networks with predictive coding networks for future frame prediction. Model selection choices for the components of the network are explored by training different models and testing their performance on next frame prediction in digital video from driving scenarios. Benefits and issues of using adversarial loss for future frame prediction as well as different choices for the model are discussed.}},
author = {{Wallgren, Magnus}},
issn = {{1404-6342}},
language = {{eng}},
note = {{Student Paper}},
series = {{Master's Thesis in Mathematical Sciences}},
title = {{Future Frame Prediction with Generative Adversarial Networks}},
year = {{2019}},
}