Photo by Taylor Vick on Unsplash
THE BUILDING BLOCKS OF SEMICONDUCTOR CHIPS FOR DATA CENTERS
The connected world is leading to real-time information exchange, and this is why consumers to enterprises expect the request to be processed in the fastest time possible. The devices used to send such requests can only process and store a certain amount of data. Anything beyond a threshold requires the use of data centers, and that also means transferring/receiving data over the air.
The computing industry has relied on data centers since mainframe days. However, the importance of data centers has mainly grown due to the connected systems. These data centers have to run 24×7 and also have to cater to numerous requests simultaneously.
To ensure a quick and real-time response from data centers, three major systems have to work in synchronization:
Software: If a smartphone user is sending the request, then the data needs to be encrypted in packets before sending it over the air to the remote location where the massive data centers are located. This means the software solutions, both on the client and the server-side, have to work in harmony. This is why software is the first major system required for accurate data center operation.
Connection: The second major system is the network of wired and wireless systems that aid the transmission of data from the client to the server (data centers). If a robust connection is not available then data centers will be of no use.
Hardware: The third and most critical piece is the silicon chip or hardware that makes up the data center. These tiny semiconductor chips end up catering to all the request that comes from different part of the world. To ensure the request is fulfilled in real-time, a smart silicon chip is also required that can handle the data-efficient without adding bottlenecks.
The growing internet user base along with data-driven computing solutions has to lead to the high demand for data centers. To cater to all such growing services, different types of data centers are required. Some data centers are small in size (less number of servers) and some are giant. Data centers with more than 5000 servers are also called hyperscale data centers. In 2020, there were more than 500 hyperscale data centers running 24×7 and were catering to request coming from any part of the world.
Data Centers Require Different Types Of Semiconductor Chips.
To run these hyperscale data centers requires large facilities, but the key piece is still the tiny semiconductor chips that have to run all the time to handle different types of requests. Due to the growing focus on data centers, there is a need to change the way new semiconductor chips are being designed for data center usage.
This is why all the semiconductor chip solutions that end up getting used in the data centers should be built around the following blocks:
Processing: Semiconductor chips for data centers should be designed to process not only a large amount of data but also a new type of data. This requires designing semiconductor chips that can cater to the request in the shortest time possible, while also ensuring there are no errors during the processing.
Security: Data centers receive different types of data processing requests and this data can have any information from credit card processing requests to personal information to login credentials. Semiconductor products by default have to focus on the security aspect when designing silicon solutions for data center usage.
Workloads: Rapid software development has lead to different types of data. Data eventually lead to the formation of workloads that the computing system has to process. Given the rise of AI/ML/DL, there is a need to process the data elegantly. This requires doing away with traditional processing blocks and instead of adapting to more workload-centric architecture that can enable a high level of parallelism to train and infer information out of it.
Adaptive: Smart world not only requires data capturing but also demands adaptive decisions. This often requires on-the-go training and modeling to ensure the user request is fulfilled intuitively. This is why there is a demand to drive AI drive architectures that can train data efficiently (eFPGA or NPU) and ensure any new (and never seen) request is handled without errors.
Storage: Memory is one of the major building blocks of the computing world. The surge in the use of cloud storage is leading to new innovative storage systems that can provide more storage per dollar. This requires driving new semiconductor solutions so that data centers become powerful but at the same time are compact enough to not consume a large amount of energy.
Efficiency: Data centers are considered to be one of the most power-hungry systems in the world. The year-on-year growth in hyperscale data centers is only going to increase the power consumption. To balance the processing need with the power consumption, semiconductor solutions have to consider the energy consumption per user request. By building an efficient semiconductor chip, the data centers can expand in number without impact the total power consumption.
The above building blocks are not specific to XPUs for data centers only. These blocks are valid for another type of semiconductor chip that data centers require. This can be a networking solution (PCIe) or a new data transfer (HBM) interface. Eventually, the above points discussed are the major reasons why data centers require different types of semiconductor chips.
THE BOTTLENECKS TO DRIVE NEXT-GEN SEMICONDUCTOR CHIPS FOR DATA CENTERS
Designing and manufacturing semiconductor chips for any type of solution requires a thorough understanding of different issues and opportunities. The same strategy is applicable when coming up with semiconductor solutions for data centers. For data centers, the complexity is much more than consumer systems mainly due to the need to provide bottleneck-free semiconductor products that run all the time.
Over the past few years, new data center-focused semiconductor companies have emerged and they have been providing different solutions. Fungible’s Data Processing Unit and Ampere’s ARM-powered XPU are a couple of such examples. However, in the end, the goal of all the semiconductor solutions is to focus on a set of features that ensures all the request is catered by the data centers in the real-time without adding any bottlenecks.
When it comes to bottlenecks in the world of computing, the list is endless. These bottlenecks can originate either from software or hardware. Eventually, the software features have to be mapped onto the hardware so that both software and hardware can work in synchronization to drive next-gen solutions.
The next-gen semiconductor chips need to focus on a few criteria’s to drive bottleneck free semiconductor powered data centers:
Features: Traditional semiconductor chips for data centers were (and still are) purely focused on performance. As the world is increasingly adopting connected systems, there is growing demand to balance performance with efficiency. This requires a new set of features that can ensure that tomorrow’s data centers are more efficient than today’s. These features can range from using advanced transistor-level techniques to new packaging solutions.
Data: The amount of data that hyperscale data centers have to crunch will keep increasing every year. The storage aspect of it will also grow along with it. This growth is leading to huge cooling systems and thus adds to total energy requirements. This challenge of managing data while lowering the impact on power consumption is pushing new solutions. More modular approaches are needed to drive next-gen semiconductor solutions.
Parallelism: Any given chip of any type in a data center can receive any amount of request. To ensure there are no bottlenecks, intelligent parallelism techniques are required. Some of the parallelism techniques require software support but many often do require hardware features (cache, data pipeline, etc.) that can support parallelism. Networking and XPUs solutions often have to consider this problem while designing chips for data centers.
Speed: While there is a growing concern of power consumption (by data centers) due to performance requirements, there is also demand to drive faster response out of data centers. This requires designing semiconductor chips for faster processing. Balancing the power, performance, and area aspect for data centers is becoming more difficult than ever. This is leading to more modular data centers but it is still going to demand semiconductor chips that can provide a high-speed solution without adding to the power requirement.
Network: Data centers have to communicate with different systems that are located in remote areas, and such communication requires heavy usage of networking solutions. To drive communication efficiently, robust networking chips are required that can handle the data without any errors. This demands designing and manufacturing semiconductor solutions with reliability and error correction. In the long run, network chips are going to play a vital role and require the bottleneck-free design to drive new data centers.
Architecture: Intel is considered the leader in XPU solutions for data centers. To design XPUs, Intel has been relying on its homegrown x86 architecture. In the last decade, the emerging workloads have changed a lot and that requires new XPU solutions. To provide newer solutions, emerging companies are focusing more on ARM and RISC-V to power their solutions. The major driving factor to use ARM or RISC-V is the ability to adapt and change the architecture to suit future requirements. Picking up the architecture is vital to avoid any kind of bottlenecks in the XPUs for next-gen data centers.
In the last two years, the world has moved towards data center solutions mainly due to the remote feature required by different services. The growth in the number of smartphone and smart device users is also driving the need for new and efficient hyperscale data centers. To cater to the future demand of green hyperscale data centers, the existing and emerging semiconductor companies will keep coming up with a newer solution.
In the long run, newer data-centric semiconductor solutions are only going to benefit the computing industry and the race to wind data centers has just begun.
