Pursuing the Twin Challenges of Low Computational Complexity for Consumer Use and High Image Quality for Business Use
(Continued from Interview Part 1.)
Though H.264/AVC drew high expectations from society, it was deemed to be a "mammoth codec," and practical use was thought to be too challenging. However, after succeeding in the first step toward practical application, namely the development of an encoding algorithm, Nakagawa and his team set their sights on applying the algorithm in an H.264/AVC product to achieve practical use.
"Until that time, it was normal to have differences between LSI for consumer use and business use, such as in broadcasting equipment. For consumer use, image quality was certainly important, but the most important factor was cost. This includes not only the cost of the LSI unit itself but also power consumption. On the other hand, with business applications such as those in broadcasting, since transmission bandwidth such as satellite channel is limited, the crucial factor was to transmit the highest-quality images with the lowest possible information volume, so image quality became top priority. Because of this, the LSI scale tended to increase; as a result, it was common for costs to be substantially higher compared to consumer-use products.
Encoder and decoder equipped with H.264/AVC LSI
"However, even for consumer-use products, image quality has a direct impact on product competitiveness, and even for business-use products, reducing costs is important. If a video encoding LSI with the cost and power consumption of a consumer-use product could achieve an image quality that satisfied broadcasting stations, it would be competitive in both the consumer and business-use markets.
"At that time, Fujitsu had two sets of customers: those who purchased consumer devices in the electronic device category, and those in industrial fields such as broadcasting. H.264/AVC can be used by customers in both categories. Being perceived as a 'mammoth codec' without confidence in its practical viability, we resolved not to compromise and decided to develop a product that was suitable for both consumer and business uses."
Accelerated Development due to High Market Expectations
"Although we had determined the product development plan, the truly hard part still lay ahead. Although we succeeded in reducing the computational complexity of H.264/AVC, it was still about double that of MPEG-2, and the methods were very complex due to the large number of included functionalities. We also had to ensure we supported all functionalities stipulated in the more than 300 page standardization document. In addition, high market expectations forced us to accelerate the development schedule.
"To carry out this complicated development project, we gathered many skilled members who had expertise in various fields. It was truly a dream team.
"Usually, the hardware is ready first, but in this case, we had to simultaneously move forward with hardware design, software development, and development of the transmission equipment. With LSI, a single bug can mean building it all over again, which is tantamount to wasting millions of dollars. There is no room for mistakes. After a painstaking series of checks and thorough efforts, we completed a large-scale LSI and video transmission device, which would normally take two years to develop, in an astonishingly short period of one year.
"After developing the product, the next step was to demonstrate the technological potential of the video transmission device and LSI to our customers. With the video transmission device in particular, we earned the trust of a major overseas broadcasting station by discussing the technology we had to offer. We worked in collaboration with a major US broadcasting station that highly evaluated our technology to develop color-degradation control technology for multi-stage relaying, thereby establishing a broadcasting industry standard. This helped lead to more broadcasting stations around the world using our product."
Frequent, Unexpected Occurrences in Every Step of the Development Process
"At the early stages of our R&D, because the number of researchers we had (leading up to the encoding algorithm's development) were low, all members shared a similar background in terms of expertise and used a common technical terms. This allowed us to work rather freely at our own discretion, while carrying on with daily lively debates to push forward with substantive research and development. We built the experimental equipment ourselves, and we had a sense of accomplishment in advancing our own research. By having discussions oriented toward a common goal and by producing results, we cultivated an environment where everyone's motivation increased naturally, allowing us to develop the H.264/AVC algorithm so quickly.
"However, the product development stage required circuit design, software design, equipment design, technological development, scheduling and resource management, and various other tasks that necessitated different sets of expertise. This prompted us to gather some of the top specialists in each of the relevant fields. My role was to develop the LSI with the planned level of performance according to the schedule, so it was important to prepare solid plans for development and verification. However, a variety of unexpected circumstances frequently occurred in every step of the development process, such as in design, development, trials, and evaluation. There were even times when serious problems occurred, but in each step, we pooled the expertise of our team members to overcome every challenge that came our way. I believe this is the blessing of excellent teamwork."
The Truly Rewarding Experience of Being Involved from Upstream Research to Product Launch
Nakagawa first stumbled into this world while taking a course taught by Hiroshi Harashima (emeritus professor of the University of Tokyo), who was a famous expert in facial studies.
"I had the great opportunity to be involved in every step along the way from upstream research to product launch, including basic research, international standardization, practical application research, product development, and consultation support. As I faced challenges in various phases throughout the entire process of research and development, I feel I could strengthen my connections with colleagues and customers. In addition, I think that the knowledge and experience we gained in each research phase, as well as my colleagues' strong teamwork, saved us from many tight spots. As an engineer, being involved from upstream research to product launch was a truly rewarding experience."
The Medal Symbolizes Our Team's Strengthーand the Research Continues
Nakagawa, who among other achievements has earned a stellar reputation in the industry for receiving the best paper award from the SMPTE, an association of broadcasting equipment professionals, was awarded Japan's Medal of Honor with Purple Ribbon in 2016 for his contribution to globally spreading high-definition technology.
"I believe that the medal symbolizes Fujitsu Group's technological strengths and our team's strength. Research and development in video technology requires expertise in various technological fields. I believe that the reason why we could make a contribution to the world by developing practical-use technology for H.264/AVC was because we could leverage the fusion of superb technological expertise with the experience of many of our colleagues in Fujitsu Group.
It is truly an honor to represent my colleagues in receiving this medal of honor, and I share this great joy with my mentors who guided me along the way as well as with my colleagues."
After H.264/AVC, Nakagawa continued to participate in collaborate efforts between Fujitsu, Socionext, and Fujitsu Laboratories to develop the H.265/HEVC LSI, which they utilized to develop the IP-HE950, a 4K-compatible video transmission device. Their research still continues.
"This device is packed with the technology and know-how Fujitsu has cultivated to date; compared to the H.264/AVC, it can perform high-quality video transmission at half the data volume. Many of our customers are already using it, and with the start of the spread of 4K/8K this year, I think the device can prove its value in an even wider range of applications."
Be Curious About Many Things, and Keep Adding Pages to Your Book of Knowledge
A common theme in Nakagawa's stories of overcoming various challenges that were thought to be impossible is what rescues him in moments of desperation. Sometimes this is the presence of colleagues, while other times it is Nakagawa's own ideas.
"Sometimes, I like to go to Tsukiji Market and buy a whole large fish. I also like to travel, and I'm interested in cars."
"What I can say from my experience so far is that I am curious about many things, and I try to keep adding pages to my book of knowledge. I focus not only on fields related to my line of work but a wide range of unrelated subjects. I try to delve into such subjects to understand the theories and principles behind them and then tie that understanding to real knowledge that I can add to my book. I have found that if you have such pages in your book, they may come in handy when you need them. If your book of knowledge has many pages and you have conviction, I believe your chance of having serendipitous encounters increases.
"To develop the H.264/AVC algorithm for practical use, we repeated a trial-and-error process while considering what it means to improve subjective quality. Although in the beginning the image quality was poor, as we iterated this process, we gradually started to find clues about how to make improvements. Then, as if our doors of perception suddenly flew open, we grasped the theory and principle behind achieving high image quality.
"I believe that whatever higher power is out there offers hints to solve problems with equal likelihood to everyone, but whether you can recognize such opportunities and let them guide you to a solution greatly depends on how many pages you have in your book of knowledge, and how deeply you are committed to solving your problem."
Fujitsu Laboratories Ltd.
Akira Nakagawa graduated from the Department of Electrical and Electronic Engineering of the University of Tokyo in 1989, and he completed his master's course at the same university in 1991.
He joined Fujitsu Laboratories that same year. Since then, he has been involved in basic research related to video encoding technology, international standardization, and LSI/equipment development.
A Doctor of Engineering, he was awarded Japan's Medal of Honor with Purple Ribbon in spring 2016.