Exploring Processor and Memory Architectures for Multimedia

Abstract

Multimedia has become one of the cornerstones of our 21st century society and, when combined with mobility, has enabled a tremendous evolution of our society. However, joining these two concepts introduces many technical challenges. These range from having sufficient performance for handling multimedia content to having the battery stamina for acceptable mobile usage. When taking a projection of where we are heading, we see these issues becoming ever more challenging by increased mobility as well as advancements in multimedia content, such as introduction of stereoscopic 3D and augmented reality.

The increased performance needs for handling multimedia come not only from an ongoing step-up in resolution going from QVGA (320x240) to Full HD (1920x1080) a 27x increase in less than half a decade. On top of this, there is also codec evolution (MPEG-2 to H.264 AVC) that adds to the computational load increase. To meet these performance challenges there has been processing and memory architecture advances (SIMD, out-of-order superscalarity, multicore processing and heterogeneous multilevel memories) in the mobile domain, in conjunction with ever increasing operating frequencies (200MHz to 2GHz) and on-chip memory sizes (128KB to 2-3MB). At the same time there is an increase in requirements for mobility, placing higher demands on battery-powered systems despite the steady increase in battery capacity (500 to 2000mAh). This leaves negative net result in-terms of battery capacity versus performance advances.

In order to make optimal use of these architectural advances and to meet the power limitations in mobile systems, there is a need for taking an overall approach on how to best utilize these systems. The right trade-off between performance and power is crucial. On top of these constraints, the flexibility aspects of the system need to be addressed. All this makes it very important to reach the right architectural balance in the system.

The first goal for this thesis is to examine multimedia applications and propose a flexible solution that can meet the architectural requirements in a mobile system. Secondly, propose an automated methodology of optimally mapping multimedia data and instructions to a heterogeneous multilevel memory subsystem. The proposed methodology uses constraint programming for solving a multidimensional optimization problem.

Results from this work indicate that using today’s most advanced mobile processor technology together with a multi-level heterogeneous on-chip memory subsystem can meet the performance requirements for handling multimedia. By utilizing the automated optimal memory mapping method presented in this thesis lower total power consumption can be achieved, whilst performance for multimedia applications is improved, by employing enhanced memory management. This is achieved through reduced external accesses and better reuse of memory objects. This automatic method shows high accuracy, up to 90%, for predicting multimedia memory accesses for a given architecture.

Abstract (Swedish)

Popular Abstract in English

Consumption of multimedia content is one of the main tasks we perform in our day-to- day lives, such as listening to music on an iPod or watching the news from an online steaming service. As technology has evolved and enabled easier access to different content (basically access anything anywhere), our consumption has grown. In only the last decades we have evolved from being fed with preselected multimedia content, to self-fed consumers, to what can only be described as being constantly connected to a parallel world that feeds its subscribers as much as they feed it.

Some questions that directly arise from this include: How has this been technically possible? What are technical consequences of this evolution? What are the technical challenges that lay ahead?

If we start breaking down what this involves, we soon see that constant connection results in a need for a device that provides mobility, to stay connected wirelessly, and at same time has performance and durability to handle increased demand of multimedia content. As this is a huge area, we will focus more specific on how multimedia content is handled, with focus on processing and memory for handheld battery powered device. The goal is to be efficient in terms of power as well as performance with as flexible an architecture as is possible. This is a classical computer architecture trade-off problem where the task is to find the right balance between power and flexibility, when taking all the different factors into consideration.

Multimedia content can be defined in different ways, such as audio, video, images, animations, graphics and even texts. This also includes the combinations of all above- mentioned content forms. One important aspect when characterizing different multimedia content forms is if they are streaming content or not.

In this thesis, we focus on streaming multimedia applications where the major functionality is audio and video. They are often the driving force of any multimedia application. There are many similarities between audio and video in terms of how they run and work. Thus, providing a good and efficient solution is essential for a great end- user experience, such as a rich user interface (UI), crisp video recording and viewing and of course long battery life.

There are many challenges for delivering a great end-user experience, as this involves many different sets of optimizations and technology enhancements. Everything from application software optimization to hardware architecture optimizations needs to be taken in to account. The technology enhancements range from getting a faster CPU to getting a better battery that lasts longer on a single charge. When combining it all together, the result is a set of different trade-offs that are often contradictory. In order to get the best end-user experience out of this complex system one needs to take a methodological approach.

This is the main contribution of this thesis, where we propose a methodology on how to in an automated way combine a set of different design choices in order to find the best

possible solution for a given set of architectures and a given set of multimedia applications.

There are many different approaches for finding the best solution, including integer linear programming (ILP), heuristic based genetic algorithms (GA), or constraint programming (CP) which is the approach that we have chosen. In CP the object is to use constraint specific reasoning methods to formalize a problem and to create a model of it. . This is then used as input to a solver that can find one or many solutions (depending on the type of problem) for the specified problem. The problem we are solving here can be broken down to a combinatorial optimizations knapsack problem. The main benefit when using CP compared to other methods is that we can state the problem in a flexible way and obtain the best solution for a specific architecture.

Our results show that by using our method we can simplify the design choices of complex multimedia systems and make the right trade-offs early in order to get the best end-user experience.

• author: Iranpour, Ali
• supervisor: Krzysztof Kuchcinski; Flavius Gruian
• opponent: Professor Brorsson, Mats, School of Information and Communication Technology, Royal Institute of Technology (KTH)
• organization: Department of Computer Science; EDSLab-lup-obsolete (research group); Parallel Systems
• publishing date: 2012
• type: Thesis
• publication status: published
• subject: Computer Science
• keywords: Multimedia, Computer Architecture, Mobile, Embedded Systems, constraint programming, Memory, Processing, optimal memory mapping
• pages: 156 pages
• defense location: Room E:1406, E-building, Ole Römers väg 3, Lund University Faculty of Engineering
• defense date: 2012-02-03 13:15:00
• ISBN: 978-91-976939-6-7
• language: English
• LU publication?: yes
• additional info: Authorʼs Contributions In all the papers, Krzysztof Kuchcinski, my supervisor, has helped with the writing and through discussions. Otherwise, all work has been performed by me. This includes choice of topic, design and development of software components, and performing experimental evaluations.
• id: da93c833-90c9-40a3-9e1e-2f42a68d4a5f (old id 2270092)
• date added to LUP: 2016-04-04 09:38:53
• date last changed: 2021-05-06 19:43:30