Investigating Object Detection and Semantic Segmentation Using Preprocessed Radar Data

Abstract

While cameras are the most prevalent devices used in physical surveillance and monitoring, there are situations where they are ineffective. In adverse weather conditions, darkness or privacy-sensitive contexts, there are excellent opportunities to replace or complement cameras with radar.

There are advanced and successful computer vision solutions for cameras, in areas such as object detection or semantic segmentation. However, the equivalent solutions are potentially underutilized for radar. As with cameras, computer vision applied on radar data could be potentially very useful and have a variety of applications. Of interest to this thesis specifically is the possibility of using computer vision techniques for optimizing radar signal processing. To this end, this thesis aims to investigate the potential of instantaneous object detection and semantic segmentation on preprocessed radar data.

A novel annotation framework, which is automated and camera-assisted, is developed to generate a custom data set. Three models are implemented and tested: AdaBoost (classifier), YOLOv8 (state-of-the-art object detection) and an adapted U-Net (semantic segmentation).

The results indicate that object detection and semantic segmentation based on single frames of radar data generated early in the signal processing chain is not only feasible, but promising.

Popular Abstract

When thinking of physical security and surveillance, most people probably think of security cameras. However, there are situations where cameras do not work well, as anyone who has tried photographing in darkness or heavy rain knows. There are also situations where cameras can be inappropriate because of privacy reasons, but where there is still a need for security and monitoring.

In such cases, radar is a good alternative or complement to security cameras. Radars work by transmitting radio waves out into the environment, where they reflect off objects and then return to be received by the radar. How far away an object is can then be determined from how long it takes a signal to return, and where it is can be determined from the direction the signal returns from. This makes radars very good at finding where objects are located. In addition, if multiple signals are sent and received, the velocity of the object can be determined, which can be a big advantage over cameras.

There have been great advances made in machine learning and artificial intelligence over the past couple of years. One specific area of machine learning with a lot of applications is the field of computer vision. Put simply, this means teaching a machine learning model to look at images and recognize objects. This can for example be used to look at traffic camera photos and read license plates, or look at X-rays and help with diagnostics. While computer vision has been used for radar images as well, this field has not received as much attention. The aim of this thesis is to investigate some possibilities with using computer vision techniques for looking at radar images.

A radar typically receives a lot of data, but not all of it is useful. The data is processed and transformed in consecutive steps, called the ’signal processing chain’. For a very simplified example, imagine an empty field with only one radar and a single person standing ten meters in front of it. The radar transmits radio waves in many directions, some reflect off the person and return to the radar, which gets to work. The raw returning signal is itself hard to interpret, so the radar starts to transform it into something that is easier to work with. It performs a lot of calculations to determine how long it took the signal to return, from what angle it arrived, whether the object it bounced off is moving, and more. In a more realistic scenario, the radar is continuously transmitting signals and receiving signals which has to be interpreted, and the amount of processing required adds up quickly.

As mentioned, not all the information that is received and processed is useful. This is part of the problem this thesis wants to address. What if, in the example presented above, a machine learning model could tell the radar: ’Hey, the object ten meters in front of you seems to be a person’. If the radar knew what type of object was being detected early on, it could make it more precise and efficient by tailoring the processing of the signals. However, it creates some challenges for the machine learning model. First off, this means it has to be relatively quick itself. It also means the model has to act upon the data before it is fully processed, which means the available information is somewhat limited. Secondly, it has to be relatively light-weight so that it can be run on a radar unit, as opposed to a large and powerful computer. In this thesis, we implement, train and test three models, to investigate the possibilities and limitations for solving this problem.

• author: Erlander, Albin and Persson, Felix
• supervisor: Anders Heyden
• organization: Mathematics (Faculty of Engineering)
• course: FMAM05 20241
• year: 2024
• type: H2 - Master's Degree (Two Years)
• subject: Technology and Engineering
• keywords: object detection, semantic segmentation, computer vision, U-Net, YOLO, AdaBoost, FMCW radar, range-Doppler, radar data annotation
• publication/series: Master's Theses in Mathematical Sciences
• report number: LUTFMA-3538-2024
• ISSN: 1404-6342
• other publication id: 2024:E30
• language: English
• id: 9161872
• date added to LUP: 2024-06-17 11:19:22
• date last changed: 2024-06-17 11:19:22