从作用上来看,数据中心就是一个超大号的机房,里面有很多很多的服务器,专门对数据进行集中管理(存储、计算、交换)。 In terms of function, a data center is an oversized computer room with many servers dedicated to centralized data management (storage, computing, and exchange).
根据业界机构统计,2020年全球经数据中心处理的数据流量高达15.3ZB(1ZB≈10亿TB),占全球总流量的99.35%。也就是说,几乎所有的互联网数据,都离不开数据中心的处理,由此可见其重要性。 According to statistics from industry organizations, the data traffic processed by data centers in the world in 2020 reached 15.3ZB (1ZB≈1 billion TB), accounting for 99.35% of the global total traffic. In other words, almost all Internet data cannot be separated from the processing of data centers, which shows their importance.
按现在比较流行的说法,数据中心是像水厂、电厂一样的重要基础设施,是数字经济的动力引擎,也是国家和社会发展的支撑底座。 According to a popular saying now, data centers are important infrastructure like water plants and power plants. They are the power engine of the digital economy and the supporting foundation for national and social development.
█ 数据中心的发展阶段 █ Data center development stages
我们先来看看数据中心的发展历史。 Let’s first take a look at the development history of data centers.
上世纪60年代的时候,人类还处于大型机时代。那时,为了存放计算机系统、存储系统和电力设备,人们修建了机房,并将其称为“服务器农场”(Server farm)。 In the 1960s, when humans were still in the mainframe era, people built computer rooms to store computer systems, storage systems, and power equipment, and called them ‘server farms.’
这个“服务器农场”,被认为是数据中心的最早原型。 This ‘server farm’ is considered the earliest prototype of a data center.
到了90年代,随着互联网的诞生和蓬勃发展,很多公司开始推行信息化。他们建设自己的网站,还搭建了大量的邮件、FTP、OA办公自动化等服务器。 In the 1990s, with the birth and rapid development of the Internet, many companies began to promote informatization. They built their own websites and set up a large number of mail, FTP, OA office automation and other servers.
有些公司将服务器放在企业内部的机房。也有些公司,因为服务器不多,但又不愿意放在办公室(噪音大、容易断电、安全性低),于是,就“托管”在运营商机房,租用运营商的场地、电力、网络带宽,让对方代为管理和维护。 Some companies put their servers in their internal computer rooms. Some companies, because they don’t have many servers but don’t want to put them in the office (noise is high, power outages are easy, and security is low), so they “host” them in the operator’s computer room, renting the operator’s space, electricity, and network bandwidth, and letting the operator manage and maintain them on their behalf.
数据中心的早期阶段(第一阶段)
The early stages of data centers (Phase 1)
于是,数据中心的概念开始逐渐形成。1996年。一家名叫Exodus的美国公司(专门从事机房设施建设和带宽服务),最早提出了“IDC”这个叫法。 Thus, the concept of data center began to take shape. In 1996, an American company called Exodus (specializing in computer room facility construction and bandwidth services) first proposed the term ‘IDC’.
这就是IDC数据中心发展的早期阶段。 This was the early stage of the development of IDC data centers.
1997年,苹果公司推出了一款名叫“Virtual PC(虚拟PC)”的虚拟机软件。后来,VMWare也推出了现在大名鼎鼎的VMWare Workstation,标志着虚拟机时代的到来,为数据中心的演进打下了基础。 In 1997, Apple launched a virtual machine software called ‘Virtual PC’. Later, VMWare also launched the now famous VMWare Workstation, marking the arrival of the virtual machine era and laying the foundation for the evolution of data centers.
随着时间推移,第一代数据中心的托管服务开始精细化,从完整的服务器主机托管,延伸出了网站托管,出现了虚拟主机服务。 Over time, the hosting services of the first generation of data centers began to become more sophisticated, extending from full server hosting to website hosting and the emergence of virtual hosting services.
也就是说,在某一台服务器上,通过虚拟主机软件,虚拟出N个网站主机,出租给N个客户使用。 That is to say, on a certain server, through the virtual host software, N website hosts are virtualized and rented to N customers.
除了网站之外,还出现了数据存储空间租用等多样化的服务。这就是IDC数据中心的第二个阶段。 In addition to websites, diversified services such as data storage space rental have also emerged. This is the second stage of IDC data centers.
数据中心的第二阶段
The second phase of the data center
再往后,到了21世纪初,亚马逊、谷歌等公司提出了云计算,从而将数据中心带入了第三个阶段(云计算阶段),持续至今。 Later, in the early 21st century, companies such as Amazon and Google proposed cloud computing, which brought data centers into the third stage (cloud computing stage), which continues to this day.
云计算阶段,是第二阶段的升级演进。它通过虚拟化技术、容器技术,彻底实现了数据中心服务器算力资源的池化。所有的CPU、内存、硬盘等资源,都由更为强大的虚拟化软件管理,然后分配给用户使用。 The cloud computing stage is an upgrade and evolution of the second stage. It completely realizes the pooling of data center server computing resources through virtualization technology and container technology. All CPU, memory, hard disk and other resources are managed by more powerful virtualization software and then allocated to users.
从物理硬件出租,进化为虚拟硬件出租,甚至软件平台出租、服务出租。IaaS、PaaS、SaaS,就这样出现在我们面前。 From physical hardware leasing to virtual hardware leasing, and even software platform leasing and service leasing, IaaS, PaaS, and SaaS have appeared before us.
数据中心的第三阶段(云计算阶段)
The third stage of data center (cloud computing stage)
█ 数据中心的组成结构 █ Data center structure
接下来,我们再看看数据中心到底由哪些部分组成。 Next, let’s take a look at what parts a data center consists of.
前面小枣君说了,数据中心就是一个大机房。所以,从硬件种类上来说,数据中心和我们以前经常看到的企业内部机房差不多,只不过规格、档次和管理级别更high level一点。 As Xiaozaojun said before, a data center is a large computer room. Therefore, in terms of hardware types, a data center is similar to the internal computer rooms of enterprises that we often see before, except that the specifications, grades and management levels are a little higher.
数据中心内部 Inside the data center
整体来看,数据中心的硬件分为两类,分别是主设备和配套设备。 Overall, the hardware of a data center is divided into two categories: main equipment and supporting equipment.
主设备,是真正实现计算和通信功能的设备,也就是以服务器、存储为代表的IT算力设备,以及以交换机、路由器、防火墙为代表的通信设备。 The main device is the device that truly realizes computing and communication functions, that is, IT computing equipment represented by servers and storage, and communication equipment represented by switches, routers, and firewalls.
配套设备,则是为了保证主设备正常运转而存在的底层基础支撑设备(也包括一些设施)。 Supporting equipment refers to the underlying basic supporting equipment (including some facilities) that exists to ensure the normal operation of the main equipment.
底层基础支撑设备设施,又分为多种,主要是供配电系统和散热制冷系统,另外还有消防系统、监控系统、楼宇管理系统等。 The underlying basic supporting equipment and facilities are divided into many types, mainly power supply and distribution systems and heat dissipation and refrigeration systems, as well as fire protection systems, monitoring systems, building management systems, etc.
主设备 Master Device
我们先看看主设备。 Let’s look at the master device first.
数据中心最基础的主设备,当然是服务器。服务器其实也就是个高性能计算机,大家应该都见过,里面和台式机一样,是CPU、内存、主板、硬盘、显卡(GPU)、电源等。 The most basic main equipment in a data center is, of course, the server. A server is actually a high-performance computer. You should have seen it before. Like a desktop computer, it contains a CPU, memory, motherboard, hard disk, graphics card (GPU), power supply, etc.
某型号服务器正面视角
Front view of a certain server model
以前,服务器基本上都是Intel架构(更早的时候,还有PowerPC、SPARC等)。如今,随着国家政策变化,国产CPU崛起,占据越来越多的份额。这些国产CPU采用ARM架构,性价比更高,成本更低。 In the past, servers were basically based on Intel architecture (and earlier, PowerPC, SPARC, etc.). Today, with the change of national policies, domestic CPUs have risen and occupied an increasing share. These domestic CPUs use ARM architecture, which is more cost-effective and has lower costs.
华为芯片Hi1620(数据中心用ARM架构处理器)
Huawei chip Hi1620 (ARM architecture processor for data centers)
服务器,一般都摆放在机架(也叫机柜)上。 Servers are generally placed on racks (also called cabinets).
服务器机架
Server rack
一个常见标准机架,高度尺寸通常是42U。U是一种表示服务器外部尺寸的单位,是unit的缩略语,1U等于4.445cm。机架宽度的话,有600mm或800mm。 A common standard rack usually has a height of 42U. U is a unit that represents the external dimensions of a server, and is an abbreviation of unit. 1U is equal to 4.445cm. The width of the rack is 600mm or 800mm.
机架的深度有很多种,包括600mm、800mm、900mm、1000mm、1200mm等。通常来说,IT设备(服务器)机架的深度更深(1100mm或1200mm),而通信设备的深度会浅一些(600mm)。 There are many different depths for racks, including 600mm, 800mm, 900mm, 1000mm, 1200mm, etc. Generally speaking, IT equipment (server) racks are deeper (1100mm or 1200mm), while communication equipment racks are shallower (600mm).
IT设备和通信设备的机架深度对比
Comparison of rack depths for IT equipment and communications equipment
机架里面的IT设备,除了服务器之外,还有磁盘阵列这样的专业存储设备。 In addition to servers, the IT equipment in the rack also includes professional storage devices such as disk arrays.
现在到处都讲大数据,我们人类产生的数据量每年都在激增,也就增加了对存储设备的数量和性能要求。 Big data is being talked about everywhere now. The amount of data generated by humans is surging every year, which also increases the number and performance requirements of storage devices.
大家应该都知道,现在主流的计算机存储硬盘分为HDD和SSD两种。HDD就是我们传统的机械硬盘,而SSD是逐渐开始普及的固态硬盘。 Everyone should know that the mainstream computer storage hard disks are divided into two types: HDD and SSD. HDD is our traditional mechanical hard disk, while SSD is a solid-state hard disk that is gradually becoming popular.
SSD属于半导体存储器,存储速率快,体积小,非常受欢迎。但是,它的价格昂贵。对于数据中心来说,出于性价比考虑,HDD仍然是主流选择。而SSD,目前主要用于高端客户、高性能需求业务。 SSD is a semiconductor memory with fast storage speed and small size, which is very popular. However, it is expensive. For data centers, HDD is still the mainstream choice for cost-effectiveness. SSD is currently mainly used by high-end customers and high-performance demand businesses.
除了IT算力设备之外,就是交换机、路由器、防火墙等数据通信设备了。 In addition to IT computing equipment, there are data communication equipment such as switches, routers, and firewalls.
说到交换机,就要提到一个名词——TOR,Top of Rack。 When talking about switches, we have to mention a term – TOR, Top of Rack.
TOR交换机,是数据中心领域的常见名词。顾名思义,就是机架顶部交换机的意思。这类交换机,是数据中心最底层的网络交换设备,负责连接本机架内部的服务器,以及与上层交换机相连。 TOR switch is a common term in the data center field. As the name suggests, it means the top of rack switch. This type of switch is the lowest level network switching device in the data center, responsible for connecting the servers inside the rack and connecting to the upper switches.
TOR交换机的位置 Location of TOR switches
事实上,机架交换机并没有说一定要放在机架顶部。它既可以在机架顶部,也可以在机架的中部或底部。之所以通常放在顶部,只是因为这样最有利于内部布线。 In fact, the rack switch does not have to be placed on the top of the rack. It can be on the top, in the middle or at the bottom of the rack. The reason why it is usually placed on the top is that it is most conducive to internal wiring.
机架再往上,就是一排机架、N排机架。将这些机架和服务器连接起来,就需要数据中心组网技术。 Above the rack, there are rows of racks and N rows of racks. Connecting these racks and servers requires data center networking technology.
现在最流行的数据中心组网架构,就是叶脊网络(Spine-Leaf)。我有文章(链接)专门介绍,这里就不废话了。 The most popular data center networking architecture is now the Spine-Leaf network. I have an article (link) specifically introducing it, so I won’t waste time here.
叶脊网络架构
Leaf-spine network architecture
值得一提的是,现在数据中心为了高带宽传输数据,普遍使用光纤替代网线。所以,光纤、光模块和光通信设备(OTN等),成为数据中心重要的组成部分。 It is worth mentioning that in order to transmit data with high bandwidth, data centers now generally use optical fibers instead of network cables. Therefore, optical fibers, optical modules and optical communication equipment (OTN, etc.) have become important components of data centers.
尤其是光模块,高速率光模块(例如400G)价格很昂贵,占了数据中心很大一块成本,制约了发展。 Especially for optical modules, high-speed optical modules (such as 400G) are very expensive, accounting for a large part of the cost of data centers and restricting development.
光模块
Optical Module
现在还有一个比较流行的光通信名词,叫做DCI,也就是Data Center Inter-connect(数据中心互联)。现在流行分布式部署,数据中心之间的数据流量很大,对带宽要求很高。 There is another popular term in optical communication now, called DCI, which stands for Data Center Interconnect. Distributed deployment is popular now, and the data flow between data centers is very large, which requires high bandwidth.
所以,运营商和云服务商就搞DCI,建设数据中心之间专门的光通信骨干网,是很大一块市场。 Therefore, operators and cloud service providers are developing DCI and building a dedicated optical communication backbone network between data centers, which is a huge market.
我们国家搞的那个“东数西算”,就涉及到数据中心的互联互通,对DCI相关市场有不可忽视的刺激作用。 The ‘East Data West Computing’ project that our country is carrying out involves the interconnection of data centers, which has an important stimulating effect on the DCI-related market.
光通信骨干网设备
Optical communication backbone network equipment
配套设备 Supporting Equipment
接下来,我们看看数据中心的配套支撑设备和设施。 Next, let’s look at the supporting equipment and facilities of the data center.
先看供配电。 Let’s look at the power supply and distribution first.
供电是数据中心正常运作的基础。没有电,数据中心就是废铁。 Power supply is the basis for the normal operation of the data center. Without electricity, the data center is useless.
数据中心的配电设备,主要作用就是电能的通断、控制和保护。最主要的配电设备,就是配电柜。 The main function of the power distribution equipment in the data center is to switch on and off, control and protect the electric energy. The most important power distribution equipment is the power distribution cabinet.
数据中心配电柜分为中压配电柜和低压配电柜。中压配电柜主要是10kV电压等级,向上接入市电,向下接低压配电柜。低压配电柜主要是400V电压等级,对电能进行进一步的转换、分配、控制、保护和监测。 The power distribution cabinets in the data center are divided into medium-voltage power distribution cabinets and low-voltage power distribution cabinets. The medium-voltage power distribution cabinet is mainly 10kV voltage level, connected to the mains, and connected to the low-voltage power distribution cabinet. The low-voltage power distribution cabinet is mainly 400V voltage level, which further converts, distributes, controls, protects and monitors the electric energy.
数据中心供配电示意图(案例)
Data center power supply and distribution diagram (case)
数据中心低压配电柜(图片来自网络)
Data center low voltage distribution cabinet (picture from the Internet)
除了配电柜之后,为了保证紧急情况下的正常供电,数据中心还会配备大量的UPS(不间断电源)甚至柴油发电机组。 In addition to the power distribution cabinet, in order to ensure normal power supply in emergency situations, the data center will also be equipped with a large number of UPS (uninterruptible power supplies) and even diesel generators.
“UPS+市电”是传统的供电方案。现在,更流行的是“HVDC+市电”的方案。 ‘UPS + mains’ is the traditional power supply solution. Now, the more popular solution is ‘HVDC + mains’.
HVDC是High Voltage Direct Current,高压直流输电。它和UPS之间的区别涉及到较为复杂的强电知识,后续有机会再专门介绍。 HVDC is High Voltage Direct Current, and the difference between it and UPS involves relatively complex knowledge of high voltage electricity, which will be introduced in detail later.
总之,“HVDC+市电”的可靠性和安全性更高,供电效率强于“UPS+市电”,是不间断电源的主流发展趋势。 In short, ‘HVDC+mains power’ has higher reliability and safety, and better power supply efficiency than ‘UPS+mains power’, and is the mainstream development trend of uninterruptible power supply.
我们简单说说-48V和220V。 Let’s briefly talk about -48V and 220V.
有过ICT行业实际从业经验的同学们都知道,服务器这样的IT设备通常是使用220V交流电,而核心网、无线等通信设备则更多使用的是-48V直流电。 Students who have practical experience in the ICT industry know that IT equipment such as servers usually use 220V AC, while core network, wireless and other communication equipment mostly use -48V DC.
市电供电,一般都是交流。数据中心,一般既会提供-48V直流,也会提供220V交流(通过AC-DC转换和DC-AC逆变转换)。 The mains power supply is usually AC. Data centers usually provide both -48V DC and 220V AC (through AC-DC conversion and DC-AC inverter conversion).
事实上,直流现在正成为更多数据中心的选择(例如谷歌),因为直流的损耗更小,对电能的利用率更高,符合现在数据中心高算力下的高能耗发展趋势。 In fact, DC is now becoming the choice of more data centers (such as Google), because DC has lower losses and higher energy utilization, which is in line with the current trend of high energy consumption under high computing power in data centers.
再来看看散热制冷。 Let’s take a look at heat dissipation and cooling.
制冷系统是数据中心除了主设备之外的第二大耗能主体。关于数据中心的耗电数据,我在“东数西算”那篇文章中有详细介绍,这里就不多说了。 The cooling system is the second largest energy consumer in the data center after the main equipment. I have introduced the power consumption data of the data center in detail in the article ‘Eastern Numbers and Western Calculations’, so I will not go into details here.
目前,数据中心制冷主要包括两种方式,一种是风冷,另一种是液冷。 At present, there are two main methods of data center cooling: air cooling and liquid cooling.
风冷一般采用风冷空调系统。和我们家用空调一样,数据中心风冷空调也分为室内机和室外机。相对来说,技术成熟,结构简单,容易维护。 Air cooling generally uses an air-cooled air conditioning system. Like our home air conditioners, data center air-cooled air conditioners are also divided into indoor units and outdoor units. Relatively speaking, the technology is mature, the structure is simple, and it is easy to maintain.
液冷,是采用液体作为冷媒,进行降温散热。 Liquid cooling uses liquid as a refrigerant to cool and dissipate heat.
液体的导热能力是空气的25倍,相同体积下,液体带走的热量是空气的近3000倍。从噪音角度来看,同等散热水平下,液冷的噪音比风冷降低20-35分贝。从能耗的角度来看,液冷比风冷节约电量30%-50%。 The heat conductivity of liquid is 25 times that of air. Under the same volume, the heat carried away by liquid is nearly 3,000 times that of air. From the perspective of noise, at the same heat dissipation level, the noise of liquid cooling is 20-35 decibels lower than that of air cooling. From the perspective of energy consumption, liquid cooling saves 30%-50% of electricity compared to air cooling.
目前,液冷技术被行业普遍看好,但仍处于探索阶段。从总体来看,液冷的市场前景非常广阔,据称市场规模超过千亿。 At present, liquid cooling technology is generally favored by the industry, but it is still in the exploratory stage. Overall, the market prospect of liquid cooling is very broad, and the market size is said to exceed 100 billion.
中兴展示的服务器浸没式液冷技术
ZTE’s server immersion liquid cooling technology
关于制冷和散热,值得一提的是,机柜池级、排级和机柜级等近端制冷方式,正在崛起,成为新建数据中心的主流选择。 Regarding cooling and heat dissipation, it is worth mentioning that proximal cooling methods such as cabinet pool level, row level and cabinet level are emerging and becoming the mainstream choice for newly built data centers.
传统的制冷都是房间级,对整个机房进行空调制冷,这种方式制冷路径太长,效率太低,无法满足高功耗设备的散热需求,能耗也很大。 Traditional refrigeration is room-level, and the entire computer room is cooled by air conditioning. This method has a too long cooling path and too low efficiency to meet the heat dissipation requirements of high-power equipment, and also consumes a lot of energy.
机柜池级、排级和机柜级,是以一个机柜池、一排机柜或者单个机柜为中心,进行散热设计。 The cabinet pool level, row level and cabinet level are centered on a cabinet pool, a row of cabinets or a single cabinet for heat dissipation design.
机柜排级散热,以一排机柜为对象,进行风道设计。 Cabinet row-level heat dissipation, with a row of cabinets as the object, carries out air duct design.
这种方式,气流路径明显缩短,散热效率很高。 In this way, the airflow path is significantly shortened and the heat dissipation efficiency is very high.
除了配电和散热制冷之外,数据中心还有一些和管理运维有关的设备设施,例如动环监控系统、楼宇自动控制系统、消防系统等。 In addition to power distribution and heat dissipation and refrigeration, data centers also have some equipment and facilities related to management and operation, such as environmental monitoring systems, building automation control systems, fire protection systems, etc.
动环监控,就是动力和环境监控,实时监测和管理数据中心运行状态的。 Dynamic environment monitoring refers to power and environmental monitoring, which monitors and manages the operating status of the data center in real time.
这些年,在传统动环监控系统的基础上,开始演进出了DCIM。 In recent years, DCIM has evolved based on traditional dynamic environment monitoring systems.
DCIM全名叫做Data Center Infrastructure Management,是知名咨询公司Garter提出来的。它的管理范围更为全面,采用工具监控、管理和控制数据中心的所有IT主设备以及配套基础设施。 DCIM stands for Data Center Infrastructure Management, which was proposed by the well-known consulting company Garter. Its management scope is more comprehensive, using tools to monitor, manage and control all IT main equipment and supporting infrastructure in the data center.
数据中心的消防系统比较有意思。因为机房里都是电子设备,所以发生火灾的话,肯定不能直接喷水、泡沫或粉尘。 The fire protection system in the data center is quite interesting. Because the computer room is full of electronic equipment, if a fire occurs, you cannot directly spray water, foam or dust.
那怎么办呢?气体灭火。 So what to do? Extinguish the fire with gas.
出现火灾后,火灾和烟雾传感器的警铃响起,然后,机房区域可以释放氩气、氮气这样的惰性气体,剥夺火焰中的氧气,实现灭火(大约几十秒就能搞定)。 When a fire occurs, the alarms of the fire and smoke sensors sound, and then the computer room area can release inert gases such as argon and nitrogen to deprive the flames of oxygen and extinguish the fire (it can be done in about tens of seconds).
气体灭火(图片来自网络)
Gas fire extinguishing (picture from the Internet)
模块化数据中心 Modular Data Center
数据中心是一个庞大的系统,建设过程非常复杂繁琐,工期也很长。这些年,为了更快速、更灵活地部署数据中心,厂商推出了模块化数据中心的概念。 The data center is a huge system, and the construction process is very complicated and time-consuming. In recent years, in order to deploy data centers more quickly and flexibly, manufacturers have introduced the concept of modular data centers.
说白了,就是将数据中心的结构系统、供配电系统、暖通系统、消防系统、照明系统、综合布线等进行集成,变成一个一个的“积木”。然后,将“积木”运送到现场后,进行简单吊装、搭建,就可以完成建设和部署。 To put it simply, the data center’s structural system, power supply and distribution system, HVAC system, fire protection system, lighting system, integrated wiring, etc. are integrated into ‘building blocks’. Then, after the ‘building blocks’ are transported to the site, they are simply hoisted and erected to complete the construction and deployment.
采用这种方式,大型数据中心的建设周期从18-24个月压缩成了6个月左右,经济效益明显。 By adopting this approach, the construction period of a large data center can be shortened from 18-24 months to about 6 months, with obvious economic benefits.
模块化数据中心(模型) Modular Data Center (Model)
█ 结语 Conclusion
好了,以上就是关于数据中心的介绍。 Well, that’s the introduction to data centers.
正如我前文所说,数据中心是数字时代重要的信息化基础实施,是算力的重要载体,直接决定了国家的数字竞争力。 As I said before, data centers are important information infrastructure in the digital age and an important carrier of computing power, which directly determines a country’s digital competitiveness.
在“东数西算”战略的带动下,国内数据中心将迎来又一波发展热潮。根据数据统计,2022年我国数据中心业务市场规模将达到3200.5亿元,年均复合增长率高达27.0%。2025年,国内数据中心IT投资规模有望达到7070.9亿元。 Driven by the ‘East Data West Computing’ strategy, domestic data centers will usher in another wave of development. According to statistics, the scale of my country’s data center business market will reach 320.05 billion yuan in 2022, with an average annual compound growth rate of 27.0%. In 2025, the scale of domestic data center IT investment is expected to reach 707.09 billion yuan.
除了数量激增之外,数据中心正在向绿色化、智能化的方向发展,积极引入AI人工智能,提升能效,降低运营复杂度。 In addition to the surge in numbers, data centers are moving towards greener and more intelligent directions, actively introducing AI artificial intelligence to improve energy efficiency and reduce operational complexity.
未来,数据中心是否会出现新的形态变化,让我们拭目以待吧! Let us wait and see whether data centers will undergo new changes in the future!