January 16, 2024 By Bob O’Donnell At this point, most people have heard about generative AI and some of the amazing things the technology can do. What most people don’t know, however, is that in some of the more advanced applications of GenAI, organizations are starting to run into challenges that are limiting the performance of large-scale models. The problem is creating models with hundreds of billions of parameters in them—as is becoming increasingly common—demands moving huge amounts of data into large arrays of interconnected GPUs and data center systems. In order to do that, organizations need the fastest possible data connections between these devices to ensure that latencies and other data delays don’t limit the performance of these extremely expensive systems (both to purchase and to run!) To start to address this issue, companies like Nvidia have moved away from traditional copper-based Ethernet connections and towards proprietary InfiniBand-based interconnect technologies. These certainly address some of the initial issues that large AI model developers started to notice, but even they have their power and bandwidth limitations. Looking ahead, several organizations have started to look for optical-based interconnect technologies, such as those powered by silicon photonics. With these types of connections, data literally travels at the speed of light via photons that are sent over thin glass fibers. In truth, silicon photonics has been around in some form or other for over 30 years, but it’s never reached widespread adoption because of the complexity of manufacturing, the lack of standards, and the challenges of working with the technology at foundry scale. Thanks to new advancements in the fabrication of silicon photonics, such as what semiconductor manufacturer GlobalFoundries has been able to achieve with their GF Fotonix portfolio of products, however, real-world silicon photonics applications have started to come to life. What’s unique to GF’s offerings is the monolithic integration of both the optical and electrical components onto a single die. Up until now, this has had to be done with separate components made with different manufacturing processes. Integrating the two allows companies to create chips that incorporate optical elements such as modulators, couplers, multiplexers and more as well as CMOS-based electrical elements including RF components and traditional logic into a single design. In addition to reducing the cost and complexity of creating silicon photonics-based products, the monolithic integration brings system level performance advantages that are unrealizable with other approaches. Recently, GF recently partnered with a startup called Ayar Labs that’s created a custom optical I/O chiplet named TeraPHY™ using the GF Fotonix fabrication process that currently allows up to 16 simultaneous optical connections (with 128 simultaneous logical data channels) on a single fiber. The net result is 4 Tbps bi-directional throughput and 10 times lower latency, with up to 8 times the power efficiency versus traditional electrical I/O interconnects. By integrating this solution into future data center-based server designs, companies could completely avoid the potential performance bottlenecks that they’ve started to run into when training extremely large generative AI models. Moving into the real world, this means that organizations will start to be able to build data center hardware with the connection and data transfer needs that even the largest and most demanding GenAI-based foundation models require. Plus, because the technology is based on silicon photonics, they can do so over longer distances and use less power in the process. This, in turn, will enable more flexibility in data center designs as well as reduced operational costs, two factors that many organizations who are working with large foundation models are going to really appreciate. Looking further ahead, the GF Fotonix manufacturing technology—which leverages IP that is unique to GlobalFoundries—is going to enable many other types of silicon photonics-based applications as well, including satellite communications and wireless infrastructure, co-packaged optics, and enterprise networking. The optical interconnect technology underlying silicon photonics has held a lot of promise for a very long time. By making its integration with traditional CMOS components possible, as GF has done with Fotonix, the possibility of living up to that promise is looking more real every day. Bob O’Donnell is the president and chief analyst of TECHnalysis Research, LLC a market research firm that provides strategic consulting and market research services to the technology industry and professional financial community. You can follow him on Twitter @bobodtech.
