Ferric brings power efficiency to the heart of AI chips

As power-hungry AI processors strain energy systems, Ferric’s integrated voltage regulators promise more efficient power delivery.

At a time when AI accelerators are pushing power consumption to kilowatt levels and beyond, power semiconductor players are joining forces to hone power delivery to these high-performance processors. Earlier this year, US-based custom silicon and SoC firm, Marvell Technology, partnered with half a dozen power tech manufacturers from around the world to integrate voltage regulators directly into its silicon platforms.

In these integrated voltage regulator (IVRs), all components of the switched inductor DC-DC power convertor are integrated on-chip, bringing voltage regulation closer to the processor. This vertical power delivery set-up offers clear benefits: it shortens electrical paths to reduce losses, improves power delivery response, and frees-up all-important board space – ultimately enabling more computer density per rack and easing the energy bottlenecks for hyperscalers.

Placing voltage regulators close to the point of load can save power, board-space and cost. [Ferric]

US-based IVR manufacturer, Ferric, is amongst the power players working with Marvell Technology to prove the devices are well-and-truly ready for future data centres. Ferric chief executive, Noah Sturcken, has been working on these devices for nearly twenty years – and back then, he and colleagues at Columbia University, New York City, had already realised a processor power delivery bottleneck was looming. As he points out, power demand was on the rise yet processor voltages were decreasing, and traditional power delivery systems with external voltage regulators couldn't efficiently deliver the required low-voltage to processors.

“With funds from Intel and US government, we were developing some of the underlying technologies critical to enabling IVRs at Columbia,” he says. “We were working on [thin film] ferromagnetic materials that could be integrated with semiconductors to miniaturise the entire power converter system and enable dense IVRs that can address this bottleneck – and that's where we are now.”

Early days
Ferric was founded by Sturcken and Professor Ken Shepard, also from Columbia, in 2012, to commercialise integrated DC-DC converters using thin film magnetic power inductors, rather than bulky, wire-wound inductors, for vertical power delivery. Early on, the firm partnered with Tier-1 fabless semiconductor companies to implement CMOS compatible thin film fabrication processes, and by 2017, had joined forces with TSMC to integrate its thin film inductors onto CMOS chips. TSMC licensed the technology, and as Mark Liu, TSMC's President and Co-Chief Executive has said, integrating Ferric's inductors with his company's CMOS back-end-of-line enables efficient, high density on-chip/on-package power conversion at any process node.

Along the way, Ferric has won more than $80 million in US government funds and venture capital - a mighty $32 million came late last year - to support manufacturing and production ramp-up. Following the recent Marvell Technology partnership, the company launched its latest IVR in August this year, which specifically targets AI and high-performance processors, delivering 160 A from what is claimed to be the industry's smallest footprint of 4.2 x 8 x 1 mm (35.5 mm²), with 4.5A/mm² current density. Ferric also says this IVR can provide three times more power per area and more than twenty times the power per volume, compared to competitors' devices, and could scale to more than 10 kW with 64 IVRs.

Ferric's latest IVR: the Fe1766 is a 16-phase e buck converter with integrated power switches and inductors designed to address power demands of AI processors, datacentre infrastructure and advanced electronics [Ferric]

“Some incumbent devices have a profile of a couple of millimetres, but we're achieving what we believe to be the highest current with a component that's barely a millimetre thick,” says Sturcken. “A high current density is very important for AI processors - right now the current density on some processors can exceed four amps per square millimetre in hot spots, and this is likely to increase to more than six amps per square millimetre in the future. It's really important that the current density of converters keeps up with the processor.”

“Our low profile also means you can put our converters on the bottom-side of a motherboard in form-factors that might not provide enough room to accommodate vertical power delivery - that could be a mezzanine card or a PCI Express card,” he adds.

Critically for high performance processors, IVRs also provide granular power management, so the amount of power reaching a portion of a processor can be specifically controlled and rapidly adjusted according to workload. “This is a consequence of the power converters being very small and modular, being closer to the processor and having a faster regulation bandwidth,” highlights Sturcken. “The power converters can operate ten to 100 times faster than conventional power converters and move the supply voltage around that much faster to provide best energy efficiency.”

Power-hungry processor demands aside, feedback from Ferric's customers confirms that the timing of its latest IVR, and work with Marvell Technology, is apt. According to Sturcken, within the last 18 months, AI processor developers have been telling Ferric that the conventional power delivery path simply doesn't work any more. “They could lose hundreds of Watts through inefficiencies while also facing reliability issues from delivering a lot of current through the package ball grid arrays [connecting the processor to the PCB],” he says. “Right now power delivery is the biggest threat to the ongoing development of these AI processors and future AI services.”

So what now for Ferric? Collaboration is clearly the way forward. The company has multiple projects underway with Marvell Technology, in which its power converters are being integrated into different locations within the silicon, such as the motherboard or the processor package substrate. “[With Marvell], we can assess the different technical and business advantages of each different integration scheme, although we expect all to be used effectively in high volume production,” says Sturcken.

Ferric is also working with other organisations to develop power conversion technologies upstream of its own IVRs, so power delivery systems will be ready for, say, the 800 VDC power architecture of future data centres, as announced by NVIDIA earlier this year. And the firm also continues to work on ever-higher levels of integration, developing modular and flexible set-ups. For example, a power convertor might have terminations on both the top and bottom side to enable the most direct power path from the PCB to a processor. “We expect this and embedding the IVR in the package substrate to become common,” comments Sturcken.

Still, as the Ferric chief executive highlights, such developments will take time to be implemented in commercial applications, as any change raises questions. “Customers might ask, 'if you take that substrate and embed a bunch of power converters into it, how will you then test it?' Or, 'what happens if one of those power converters isn't assembled correctly, do I then have to throw out the entire system and deal with some pretty expensive economics?'” he says. “These questions create uncertainty that needs to be addressed before the technology is adopted at large-scale.”

However, Sturcken is certain that every high-performance processor will need integrated voltage regulation. “We've had this sudden recognition, of yes, this is what is needed for future processors,” he says. “Right now, many organisations still need to go out and find technology partners such as us, so that they can stay competitive – so [IVR adoption] is only a matter of time.”