For example, in the past decade, Facebook has built a global network of 15 large data centers and hundreds of edge network sites, operating hundreds of millions of users, thousands of software programs, and every two There was a software update in an hour. Jay Parikh, director of engineering and infrastructure at the company, said: 'When you are building a decentralized system that affects billions of people every day, it's really cool...and scary.'
At the company's first large-scale distributed software system seminar held at the company's headquarters in California, Parikhy said to the audience about 200 software engineers and guests: 'Everything we deal with is a decentralized system problem, this in the past It has never been done in a scaled environment... including the construction of submarine cables by itself—all that the industry has never dealt with before.
He pointed out that the challenges of computer and network hardware, as well as the database and other software implemented in hardware, will interact with each other, resulting in issues such as efficiency, culture, budgeting, etc., and everything will be linked to each other. '
The event also invited programmers from Amazon Web Services, Google, rider Lyft, and the e-commerce platform Shopify to share the latest technologies and explore the use of management including global decentralization. Database, system debugging, and how to accelerate recovery when the system is interrupted.
Facebook discussed the progress of two open source decentralized software systems, one that quickly pushes configuration changes to the architecture of millions of servers, and one that handles memory out-of-order issues in the user space of the operating system. Program.
They are part of Facebook's vast cloud computing software platform to support its four widely used applications -- News Feed, Instagram, Messenger and WhatsApp -- and Oculus software, a small user community.
Open source software and open hardware...chips?
Facebook has been slowly replacing existing off-the-shelf commercial software and hardware in the network architecture with its own software or hardware solutions and turning it into open source; for example, in the past few years, the company will be ready-made. The database (MySQL), the Memcached program and the Web Services Language (PHP) are replaced with their own developed code and released as open source.
The Systems@Scale event at Headquarters is also designed to allow Facebook software engineers to interact with peers in other companies with the goal of accelerating the search for common solutions for the decentralized system challenges they face.
On the hardware side, Facebook now generally uses servers and switches built on off-the-shelf chips with its own specifications, rather than commercial systems from vendors. According to industry sources, Facebook has been recruiting semiconductor engineers since this spring. The company's spokesperson declined to comment, only to disclose plans for chip development in an event that may be held in September.
Facebook's system hardware specifications are usually released through the Open Compute Project, which was established in 2011. Whether the company will open its own chip specifications in the future remains to be seen.
20180723_datacenter_NT01P1
Facebook has 15 large data centers around the world, but it still dwarfs Amazon's 18 locations around the world as of 2017; Amazon's strongholds each have a different number of data centers (numbers in the yellow circle)
Another bigger question is, where does Facebook and its competitors take the future of the computing world at a crazy rate, and what bottlenecks may they encounter?
Some US legislators believe that Facebook has touched a political obstacle because it used the automation system to influence millions of voters in the last presidential election. Some people say they are playing with consumers. Every data that innocent users are happy to share is sold, but so far it is almost unsupervised by the government.
Technically, the Internet giant has been pushing faster copper and optical networks for years, and they need both technologies to connect a larger number of servers and data centers themselves in the data center.
Over the years, network engineers have been delaying the generation of Terabit/sec-rated networks. Terabit/sec networks are faster than other commercial requirements, and in some cases even beyond the limits of Moore's Law or the laws of physics. Whatever the results of companies such as Facebook that continue to expand the size of their data centers, it's clear that we will see them soon.
The future scene, as Parikh said, will be 'cool and scary'; Internet giants are building at speeds never seen before, expanding large automation systems, tens of billions of consumers and thousands of Enterprises, they are consuming a lot of services they operate. He said: 'The first big motivation for me is to act quickly on a stable infrastructure.'
Compile: Judith Cheng