March 25, 2021Strategic partnership between GF and Compound Photonics will lead to more powerful, smaller, lighter and more energy-efficient AR/MR glasses. by Gary Dagastine Augmented- and mixed-reality (AR/MR) technology is at an historic inflection point, and the strategic partnership recently announced by GLOBALFOUNDRIES (GF) and Compound Photonics (CP, also known as CP Display) is driving it forward. The two companies will work together to transform the workings of near-eye microdisplays, which are at the heart of AR/MR systems. CP’s IntelliPix™ platform will be manufactured using GF’s best-in-class 22FDX™ semiconductor solution, creating the world’s first real-time AR/MR specific single-chip microdisplay enabling pixels as small as 2.5µm. The result will be the industry’s most advanced light modulation agnostic backplane/video pipeline with the ability to span a roadmap of current amplitude liquid-crystal-on-silicon (LCoS) thru microLED all the way to holographic while providing the required performance for real-time AR/MR systems. The scalable and flexible single-chip solution will support CP’s existing LCoS technology as well as CP’s forthcoming microLED display technology. With IntelliPix, the idea is to turn on only the pixels which need to be active in order to render the desired image, rather than to continually refresh all of the pixels in a display. This not only conserves power in the inactive pixel regions, it also results in higher image quality and brightness, faster refresh rates and, ultimately, AR glasses with more advanced features and performance, smaller and lighter form factor and which last much longer on a single charge. The IntelliPix architecture integrates CP’s proprietary video pipeline that reacts in real-time compensating for head motions and other environmental conditions; a software-programmable backplane to control the pixels dynamically; and driver circuitry to deliver the necessary power. Until now this has required multiple chips, but the next generation of AR glasses requires a single-chip design that is simpler, higher-performance, smaller and less power-hungry. A Natural Choice “GF’s industry-leading 22FDX solution is a natural choice for many reasons,” said Ed Kaste, Vice President of Industrial and Multi-Market at GF. “Its ultra-low power capabilities are a major advantage, but that’s only the beginning. 22FDX technology has a higher SRAM density than other planar technologies, and SRAM density correlates directly with pixel density, enabling IntelliPix to significantly shrink the pixel size. This results in a high performance, highly integrated solution which helps enable sleeker, lighter AR glasses. Also, the ability to turn pixels on and off, while leveraging body bias control, leads to far better on and off states, so that when a pixel is on, it’s brighter and when it’s off, it’s really off, further conserving power and reducing thermal effects.” The adaptive body bias (ABB) feature of 22FDX technology affords designers significantly more precision when fine-tuning the transistor threshold voltage of a circuit, enabling them to more effectively optimize the performance, energy efficiency, area, and reliability of a chip to meet the needs of a specific application. “22FDX technology also enables CP to easily integrate intellectual property (IP) specific to its customers into the design, as well as higher-voltage devices that some microdisplay architectures require. Our existing reference designs and ecosystem resources make this development process far less challenging than it would be with other technologies,” Kaste said. Tapeout is anticipated by the end of this year, with samples delivered to CP’s customer in the first quarter of 2022. Lessons Learned from Nature CP started to focus its microdisplay development effort in AR/MR and heads-up displays since 2016 by leveraging its wealth of IP portfolios in LCoS display and advanced electronic drive architecture. Based on its current generation platform, CP’s displays have been known in the industry for the smallest pixel pitch, highest optical efficiency, lowest display latency and highest frame rate performance compared with other display providers. “We had the industry’s best microdisplay sub-system – light modulator, backplane and driver – at that point, which served us well to gain traction,” said Edmund Passon, CP’s co-CEO. “But more recently, I began to think about how our optical system functions given we feed directly into it. The optical signal from the retina is split into multiple channels and is pre-processed there before it reaches the brain, The processing is similar to compression without the need to resend information the brain already has received,” he said. “I realized that to achieve the performance we were seeking for next-generation AR/MR glasses, we’d need to do something similar in reverse to reduce bandwidth and associated power consumption while maintaining performance. It allows us to send only data that is changing, while providing high performance to active objects/pixels, all at the lowest power consumption possible.” The IntelliPix architecture feature set partitions the processing between the SOC and the display sub-system, he said. CP’s customers who build compatible rendering pipelines can take full advantage of the feature set completely optimizing for real-time AR/MR systems. But the 28nm semiconductor technology CP initially turned to wasn’t up to the task for IntelliPix’s breakthrough design using smart pixels, which can be achieved with 22FDX technology. “Our current, multi-chip backplane architecture consisted of a one-bit pixel with an appetite for bandwidth. We started our work at 28nm, but to achieve the desired pixel size with the amount of logic IntelliPix required under the smart pixel, we needed GF’s 22FDX solution with its best-in-class high-performance, power efficiency and broad feature integration capability,” Passon said. “Also, looking toward the future, while our pixel size is 3.015µm right now, we’ve found that the IntelliPix architecture will enable us to get it down to as small as 2.5µm depending on the feature set. That opens up the possibility of designing microLED-based displays with a bayer like pixel grid that supports fewer required pixels. and again partitioning properly between the microdisplay and SOC video pipeline/rendering will result in the optimal power/performance balance. So the scalability of any single-chip solution is critical, and 22FDX technology fits the bill.” A Unique Partnership “We chose to partner with CP not only because they are technically innovative with respect to both hardware and software, but because they’ve been around a while as a company and have demonstrated the value of what they offer through an extensive network of corporate relationships,’ said Ran (Ruby) Yan, GF product manager for wearables, smart home and machine vision products. “What we especially like about CP is that they always target the most difficult challenges, and it is exciting and rewarding to be involved in such endeavors, which can bring about not only industrial progress but significant positive change in our daily lives,” she said. Yan said GF brings to the partnership a great deal of experience with display drivers, both integrated with backplanes and as standalone ICs, for applications including smartphones, automotive and medical devices. “We are using the baseline 22FDX platform for this work and it is also an extension of what we are already doing,” she said. “For example, the required customization will impact the backend of the line process, to add some unique features to the optical interconnect.” The Future of Display Kaste said that displays overall are a key focus for GF because they span a growing number of applications across all of the company’s business units. The demonstrated performance and lessons learned from this engagement could revolutionize display technology and the display value chain going forward, he said, and he asked CP’s Passon what he thinks 22FDX technology can bring to applications beyond AR/MR glasses. “It’s a great question,” Passon said. “Ever since we started to work with 22FDX technology, new ideas just seem to pop out of the woodwork. For example, we can see how it would be possible to use 22FDX technology to make improved performance video walls. The size of the LEDs is quite different, but their care and feeding is the same with the ability to reach the highest duty cycles that IntelliPix can provide – you could put together any number of display tiles to make a wall-based television of any size.” “Also, over the years we’ve done a lot of work with holography, and the IntelliPix platform along with the high performance capabilities of 22FDX technology serves this very well” Passon continued. “We think there’s a fantastic opportunity in automotive AR heads-up displays to place holographic objects in 3D space for better driver awareness and responsiveness. In fact, the automotive sector is particularly appealing for holographic applications in the near term, given the current computer-generated hologram (CGH) compute/power required. Work in the CGH algorithm area is showing promise to reduce this power, and IntelliPix will be ready to integrate easily with those systems to produce the highest fidelity holograms.” Passon said that while the future holds many new and exciting opportunities, there are still technical challenges that must be overcome on an industry-wide basis to get there. “For microLED-based displays to become truly practical both technically and economically, the industry must find ways to commercialize the pixelated epitaxial fabrication processes currently used in manufacturing. For holography, meanwhile, we must find ways to reduce the required compute power,” he said. Nonetheless, right now AR/MR technology is on the brink of major change, and the partnership between CP and GF is playing a key role in making it happen.
