March 3, 2023

AI/ML to Drive Significant Networking Bandwidth Growth in 2H23 and into 2024

112 Gbps SERDES and 800G to the Server to Help Propel AI/ML Growth

Alan Weckel, Founder and Technology Analyst at 650 Group

 

During DesignCon 2023, many demos highlighted the 800G ecosystem. Specifically, 112G SERDES-based products will help drive 800G and higher links to the server. Our forecasts in servers indicate that most unit growth over the next five years will come from AI/ML-based servers.

 

Why is there such a sudden and robust demand for higher-speed connectivity? Several technologies have come together at the same time AI/ML is becoming critical to digitization efforts. First, the industry was stuck on PCIe Gen 3 for nearly a decade. Most current announcements use PCIe Gen 5, significantly increasing the bus speed. Second, new AI, x86, ARM, and customer accelerator processors continue to increase core count. Third, servers continue to get multiple accelerators per server, and most designs focus on purpose-built hardware for specific workloads. Finally, 112G SERDES allows for a clean upgrade to by eight (x8) architectures and 800G. 8x56G and 400G flow nicely into 8x112G and 800G designs.

 

The confluence of the above technologies and AI moving from niche to the mainstream will cause bandwidth growth in AI to exceed a 100% CAGR over the next 3 years. Coming out of DesignCon and looking towards OFC in March, we will have 800G momentum to continue with different demos, interoperability, and product announcements. We see these server designs directly and hinted at in hyperscaler announcements. Microsoft’s investment in ChatGPT and Google’s rushed announcement of Bard show a new multi-billion wave of investment in new computing platforms.

 

Active copper cables (ACCs) will play an essential role in making these AI clusters achieve the performance and power efficiencies needed. Active copper helps solve the issue of DAC no longer having the required reach on higher-speed links, not to mention the bulky nature of the cable. In addition, they allow customers to continue with copper and not have to adopt fiber directly to the server. Fiber to the server also brings up a more considerable change in architectures and conversations of replacing the Top-of-Rack switch. In terms of timing, we expect these next-generation servers and networking platforms to be right around the corner and enter the high-growth part of the ramp in 2H23. This will provide a vital building block for growth in 2024.

 

Original Article Here

 

---

---

November 8, 2022

Active Copper Market Helps Reduce Power Consumption in the Data Center

Recent Increases in Power Cost Creating Concern and Changing Architectures in The Cloud

Alan Weckel, Founder and Technology Analyst at 650 Group

 

Most Cloud data center designs assume small increases in power costs over the life of a facility. In 2022, Cloud operators struggled to increase power to some facilities, especially in large metro areas, and already played out their cheap power options by locating massive facilities in rural areas and near cheap power sources. Things have changed quickly in 2022, with power costs increasing rapidly and sometimes faster than inflation

 

Historical data center consumption was less than10 kW per rack. However, when we add the drive toward virtualization and accelerators to support AI/ML and HPC, power can easily exceed 20 kW per rack. In power-constrained locations, this has caused operators to remove equipment racks, leaving a significant amount of empty floor space because they hit their power cap.

 

The current inflationary pressures on power and real concerns over power availability are causing cloud customers to re-evaluate basic power assumptions. Power costs will continue to rise faster over the next several years than at any time during the build out of the Cloud. As a result, our Cloud customer interviews indicate power will be a significant design factor and a more important criterion for 2023-2024 build outs.

 

Volume of data/information created, captured, copied, and consumed worldwide from 2010 to 2020, with forecasts from 2021 to 2025

 

Data center build outs will be different going forward. Cloud providers will adjust their locations and designs to a new world where power is not guaranteed, and costs may vary significantly amongst regions and countries. We should look at how Japan built its data centers as a guide. Because of limited, and expensive power, many Japanese data centers exist in the pacific northwest of the United States and Canada, not Japan.

 

While Cloud providers adjust regional preferences, the equipment inside becomes increasingly essential. This is especially true with AI/ML workloads which typically need much more power per rack. Cabling is an area that can help contribute to reducing power consumption.

 

We expect ACC cables to become an increasing portion of connectivity as servers move towards 100 Gbps and higher speeds. As a result, ACC can play an important role in saving power budgets. When talking about massive hyperscaler facilities, it can make the difference in maintaining the current rack footprint instead of reducing the number of racks. On a bandwidth basis, ACC cables help minimize power consumption and cost as server speeds increase in the subsequent two upgrade cycles. Our forecast projections for server connectivity show that approximately two-thirds of Cloud connections will be active copper by 2025/2026.

 

Original Article Here

 

---

---

 

June 3rd, 2022

Active Copper Market Poised to take Significant Share of Short Distance Market in the Data Center

AI/ML Workloads and Next-Generation Server Designs to Drive Growth

 Alan Weckel, Founder and Technology Analyst at 650 Group

 

New network architectures and increased server speeds are causing the connectivity market inside the data center to move away from Direct Attach Copper (DAC) to active solutions. The current DAC technology can’t scale with the speed coming from next-generation server designs in the Cloud. Similar to other technologies, the Cloud will lead the way, but the broader markets of enterprise and service providers will enjoy the benefits of the technology. 

 

Server speed is quickly moving to 112 Gbps SERDES and 100 Gbps ports for server access. There are several reasons behind this, such as the rapid move to PCIe Gen 4 and 5, faster processors, and the introduction of AI/ML workloads. But the more significant reason is the increase in accelerators via Smart NICs and DPUs. 2022-2024 server architectures take advantage of these factors and can drive multiple 100 Gbps lanes per server. DAC can’t scale as the distance becomes too small and a cable gauge becomes too large, and there is no appetite or need to move towards fiber or middle-of-row architectures. Cloud customers will be the first to proceed with the overall enterprise market following when they transition to higher speeds. These short distant server interconnects (up to 7m) account for over 50% of all data center network interconnects.

 

 Beyond server access, customer preference continues to shift away from large Modular chassis and toward Fixed 1RU systems. At the same time, the aggregation and core layers are being moved closer together in a move away from the physical blast radius to the virtual blast radius. These trends allow aggregation switches to connect at short distances, not necessarily across a whole data center. As a result, these Distributed Disaggregated Chassis (DDC) architectures grow in volume each year and are prevalent in the three major verticals of Enterprise, Cloud, and Service Provider. DDC doesn’t need fiber for short distances, and one can think about active copper cables replacing the modular switch fabric in a chassis.

 

 Last year, 90% of servers used DAC cables for connectivity, with the remaining 10% being a mix of 10G-Base-T, fiber, blade server enclosures, and other technologies. By 2026, we expect over two-thirds of the Cloud server market to be active copper (Figure 1). Furthermore, the percentage of 100 Gbps and above will be even higher, with almost the entire market at that speed using active copper. From a revenue perspective, the market will quickly exceed $1B and can exceed $3B by 2026, far higher than the DAC market it will replace.

 

 Active copper cables fall into 2 main categories, ACC and AEC. ACC cables use an analog redriver approach to recover the signal that is attenuated by the losses in the copper conductors. AEC cables use a DSP retimer approach to accomplish the same. While AEC technology is able to achieve longer lengths, it also uses significantly more power. Spectra7 is a leader in the ACC segment.

 

 The industry has moved technologies in the past, with the 1 Gbps to 10 Gbps as an excellent example of a drastic and rapid shift for server connectivity. The market for 1 Gbps was dominated by 1G-Base-T, but the need for 10G shifted rapidly to DAC, with 10G-Base-T never gaining traction beyond some enterprises. 100 Gbps should have the same trend, where there is a rapid inflection over the next 2-3 years, first starting with the Cloud providers and then shifting to the broader enterprise market.

 

 Cloud providers currently are examining when to transition away from DAC, and we expect to see many new designs between now and the end of 2023 transition away from DAC. This times well with servers moving to 100 Gbps links and new 25.6T and 51.2T switch ASICs. While Cloud providers tend to be secretive, we expect to see demonstrations throughout the year as both cable and switch vendors demonstrate products and interoperability.

 

 

Original Article Here

 

---

---