FPGA Implementation of the ORB Algorithm

Abstract

Image feature extraction has become a key technology in the field of autonomous Artificial Intelligence. The algorithm Oriented FAST and Rotated BRIEF (ORB), uses established technologies in image processing to allow a computer to ”see” and navigate its surroundings. This process, however, is very performance intensive, but a field-programmable gate array (FPGA), a programmable semiconductor device based on an integrated circuit, can be used to accelerate this process with good power efficiency and make it feasible for small computationally challenged implementations, such as drones. In this thesis an FPGA implementation of the ORB algorithm is taken as the research object, aiming at finding out an efficient way to implement ORB on an FPGA board. The main work of this thesis is as follows:
• Construct ORB in C++ without the OpenCV library to understand the process of ORB algorithm better.
• Based on the above step, present two methods to implement ORB calculation on FPGA board.
• Discuss the results of the two methods and summarize their pros and cons.

Popular Abstract

It is a simple task for the human brain to process a vast amount of visual information. By just looking at a picture a person can immediately gain an understanding of the space and the objects depicted. For a computer this task can be very difficult, as all it ”sees” is just a grid of colored squares. So if you want a drone to be able to understand its surroundings based on images from a camera you’ll need to be creative.
This is where the ORB algorithm comes in. By contrasting individual pixels, this algorithm allows for the detection of points that stand out from their sur- roundings within an image. Then it uses the area surrounding these keypoints to create a sort of ”digital fingerprint” called a descriptor. These descriptors can then be used to recognize features between different images, and by tracking their movement in relation to the cameras movement a computer can begin to understand its surrounding to help navigate through it.
This project focuses on getting the ORB algorithm running on a programmable circuit board called an FPGA, specifically a Zynq UltraScale+ MPSoC board. It contains Arm®-based multiprocessors with ASIC-class programmable logic and provides real-time control with soft and hard engines for graphics, video, waveform, and packet processing
It is a well-established fact within electrical engineering that an implemen- tation created directly in hardware will be more efficient than one running in software on a general processor. However, it is also much more complicated to implement a direct hardware solution. To maximize performance with the avail- able resources the hope was to be able to implement the algorithm in hardware.
The plan for this project was to use a program called Vivado high-level synthesis to generate a hardware design of the ORB algorithm from a simple C++ implementation. This method usually reduces the time required to make a hardware design since C++ is more abstract than VHDL or other hardware description languages, and thus easier for most to code in.
While this project was successful in generating hardware implementations for large parts of the algorithm and speed up the whole process, it failed to tie the disparate hardware blocks together to implement the full algorithm.
Instead a software solution running on the board processor was devised, and while it could run the camera and perform the function of registering features and giving them descriptors, the performance was too slow and clunky for it to be viable for real-time navigation.
While achieving the goal of running ORB on an FPGA, the results that had been hoped for at the outset of the project failed to materialize. However, during this project a good understanding of the ORB algorithm was developed and the project ends with the beginning of a promising future path to a hybrid software-hardware implementation. This is to be achieved through ”overlays”, a way to implement the synthesized hardware mentioned earlier, but to get it to communicate via software.

• author: Zhang, Xinyuan and Sturk Sellstedt, Emil
• supervisor: Steffen Malkowsky
• organization: Department of Electrical and Information Technology
• course: EITM02 20211
• year: 2022
• type: H2 - Master's Degree (Two Years)
• subject: Technology and Engineering
• report number: LU/LTH-EIT 2022-859
• language: English
• id: 9075141
• date added to LUP: 2022-03-03 14:08:42
• date last changed: 2022-03-03 14:08:42