Big data in China

Chapter 14 China's first batch of 2 billion companies that value big data ([-])
Since big data itself is a problem set, among many solutions, the most important and effective technology is cloud computing. The combination of the two will produce the effect of 1+1﹥2.And this is also recognized as the most effective distributed processing method for processing large data sets.Cloud computing provides an infrastructure platform for the processing of big data. Big data applications can run on this platform. The two parties are inseparable and mutually guaranteed.

Regarding the impact of big data on cloud computing, Best, the technical director of an IT company in the United States, said that the impact of big data on cloud computing is only manifested in the private cloud architecture, and has no impact on the data warehouse for the public cloud architecture. .Because the CIO of the enterprise will not put financial data or customer data on the cloud for no reason, because that is an extremely dangerous thing.The private cloud architecture is different. It has two impacts on the data warehouse:

First, through the private cloud, data sets can be consolidated and the problem of insufficient utilization can be reduced; second, data can be integrated in a sensitive way to realize business value.This ensures that the two sides will not have any conflicts, but instead play a complementary and strengthening role.

☆The difference between cloud computing and big data--the division of labor of application

difference in concept

From a macro concept, cloud computing has changed IT, while big data has changed business.At the same time, big data must have the cloud as its infrastructure in order to be promoted smoothly and demonstrate strong practical value.

Differences in target audience

The target audiences of the two parties are also different. Cloud computing represents an IT-level solution for CIOs; while big data is a strategic framework for managers and business layers. It allows us to In terms of business, it has demonstrated stronger competitiveness and fully improved its overall strength.

Development in China
☆Development status of cloud computing in the world
Speaking of the development prospects of cloud computing, the major Internet giants are full of confidence in cloud computing, just like seeing a rapidly rising A-share market.For this reason, many companies have begun to adjust their future development strategies.For example, Amazon uses Elastic Compute Cloud (EC2) and Simple Storage Service (S3) to provide computing and storage services to enterprises.Among them, the charged service items include storage server, bandwidth, CPU resource and monthly rent.The meaning of the monthly fee is similar to that of the telephone monthly fee. The storage server and bandwidth are charged according to the capacity, and the CPU is charged according to the duration (hour) of operation.Amazon turned cloud computing into a business similar to mobile communications, and it took less than two years.

According to data provided by a third-party organization, there are 44 registered developers on Amazon, many of whom are enterprise-level users.And Amazon's business revenue related to cloud computing has reached 1 million US dollars.Of all of Amazon's fastest-growing businesses, cloud computing is one.

To say that the website with the most cloud computing users is Google.This is unquestionable, because Google's search engine is powered by an infrastructure of more than 200 million servers in more than 100 locations, and the number of these facilities is growing rapidly.Whether it is Google Earth, Maps, Gmail or Docs, etc., these infrastructures are also used.From this point of view, Google is indeed very powerful, because using applications such as GoogleDocs, user data will be stored somewhere on the Internet.Moreover, users can easily access these data through any system connected to the Internet.

Another point worthy of praise for Google is its "sharing" spirit.Currently, it already allows third parties to run massively parallel applications through GoogleAppEngine in Google's cloud computing.Moreover, it has published its three magic weapons of cloud computing in the form of academic papers: GFS, MapReduce and BigTable.In some colleges and universities in the United States and China, Google has also offered courses on how to program cloud computing.

Let's look at IBM again. In November 2007, IBM launched the game-changing "Blue Cloud"

Computing platform, this cloud computing platform brings customers an out-of-the-box experience.It includes a series of automated, self-managing and self-healing virtualized cloud computing software, enabling applications from all over the world to access distributed large server pools, enabling data centers to run computing in an environment similar to the Internet.

As the world's PC software leader, Microsoft also followed the pace of cloud computing and officially launched the Windows Azure operating system in October 2008. Azure (translated as "blue sky" in Chinese) is another subversive transformation of Microsoft after Windows replaced DOS.By creating a new cloud computing platform based on the Internet architecture, let Windows truly extend from the PC to the "blue sky".Microsoft owns the desktops and browsers of millions of Windows users around the world, and now it connects them to the "blue sky".The bottom layer of Azure is supported by Microsoft's global basic service system, which is composed of fourth-generation data centers all over the world.

☆The development of cloud computing in China
Regarding the future development direction of the Internet, Wang Jianzhou, the former chairman and CEO of China Mobile, believes that there is no doubt that it must be cloud computing and the mobilization of the Internet.In other words, the future of China's Internet industry depends to a large extent on the future of cloud computing in China.

The development history of cloud computing in China:
On May 2008, 5, the first cloud computing center in China established by IBM in the Science and Education Industrial Park of Taihu New City, Wuxi, China, was put into operation;

On June 2008, 6, IBM established the second China Cloud Computing Center - IBM Greater China Cloud Computing Center at the IBM China Innovation Center in Beijing;
On November 2008, 11, Guangdong Electronics Industry Research Institute signed a contract with the Management Committee of Dongguan Songshan Lake Science and Technology Industrial Park. Guangdong Electronics Industry Research Institute will invest 28 million yuan in Dongguan Songshan Lake to establish a cloud computing platform;
On December 2008, 12, Alisoft, a subsidiary of Alibaba Group, formally signed a 30 strategic cooperation framework agreement with the Nanjing Municipal Government of Jiangsu Province to establish the first domestic "e-commerce cloud computing center" in Nanjing, with an initial investment amounting to Hundreds of millions of RMB;

In 2009, [-]Vianet launched the CloudEx product line, including a complete Internet host service CloudExComputingService, CloudExStorageService based on online storage virtualization, data security services for Internet cloud backup for individuals and enterprises, and a series of Internet cloud computing services.

Since 2010, China's cloud computing has been included in the national key projects, and has received policy, financial and technical support.For example, China Mobile Research Institute, which is an early researcher on cloud computing in China, has successfully completed the experiment of the cloud computing center.

For "cloud security", the concept created by Chinese companies is unique in the field of international cloud computing.Our concept of "cloud security" is: cloud security monitors the abnormal behavior of software in the network through a large number of mesh clients, obtains the latest information on Trojan horses and malicious programs in the Internet, pushes them to the server for automatic analysis and processing, and then Distribute virus and Trojan solutions to every client.

China's strategic vision for "cloud security" is: the more users, the more secure each user is, because such a huge user base is enough to cover every corner of the Internet, as long as a certain website is linked to a horse or a new Trojan When the virus appears, it will be intercepted immediately.

“云安全”的发展就像一阵龙卷风，很快席卷了各大安全应用企业。瑞星、趋势、卡巴斯基、MCAFEE、SYMANTEC、江民科技、PANDA、金山、360安全卫士、卡卡上网安全助手等都相继推出了云安全解决方案。例如瑞星，基于云安全策略开发的2009新品每天拦截木马攻击的数量就达到了几百万次，而其中仅在2009年1月8日这一天就拦截了765万余次。

It is reported that Cloud Security can support an average of 55 billion hit queries per day, and collect 2.5 million samples from these hits every day for analysis. According to this huge database, the first hit rate can reach 99%.With cloud security, Trend Micro now blocks up to 1000 million virus infections per day.

In fact, it is not the first time that the core idea of ​​cloud security has been proposed. As early as 2003, Liu Peng, a Chinese cloud computing expert, proposed an anti-spam grid, which is very similar to the idea of ​​cloud security.Liu Peng’s thinking at the time was that, in view of the proliferation of Internet spam, technical means alone cannot be used to automatically filter it well, because the artificial intelligence method relied on for mail filtering is not mature.Then, according to the characteristics of spam "sending the same content to millions of recipients", a distributed statistical and learning platform can be established to filter spam with large-scale user collaborative computing.

How is this method implemented?First, the user needs to install a client on the computer, and then a unique identification code can be calculated for each email received, just like a "fingerprint". By comparing the "fingerprints", the number of similar emails can be counted. The number of copies, when the number of copies reaches a certain number, you can determine which emails are spam.

Secondly, since multiple computers on the Internet have more information than one computer, distributed Bayesian learning algorithms can be used to realize collaborative learning processes on hundreds or thousands of client machines to collect, analyze and share the latest information. Information.

From this point of view, the practice of filtering spam with large-scale statistical methods is indeed much more mature, and the false positive rate is low, which has strong practicability.From the perspective of the core idea, the anti-spam grid also more truly embodies the grid idea, because each user who joins the system serves as a service object and is also an information node that completes the distributed statistical function.As the scale of the system continues to expand, the accuracy of the system to filter spam also increases.

This is not only a service process, but also a process of feeding back and improving technological progress.The anti-spam grid is like a "Skynet", making full use of the coordinated work of tens of millions of hosts distributed in the Internet, thus building a natural barrier to intercept spam.

IEEE Cluster2003 International Conference once selected the anti-spam grid as an outstanding grid project, and made a live demonstration in Hong Kong, which aroused widespread attention from all over the world.In the 2004 International Symposium on Grid Computing, a special report and on-site demonstration on the anti-spam grid were also made, and many mail service providers showed great interest.Ding Lei, the founder of NetEase, China's largest mail service provider, attaches great importance to this.

So let's go back to the previous view, spam can still be handled in this way, and the same is true for viruses, Trojan horses, etc. From this point of view, it is very close to the idea of ​​​​cloud security.

Regarding the prosperity of big data in China, a senior big data researcher in China expressed his opinion: "Now the so-called big data companies in China are still understanding big data with Internet thinking, not real big data thinking. , there is still a lot of room for innovation in the future.”

That is to say, although the development of big data in China has made some progress, it is still far from what we hope to see.Just like people's predictions about the first information technology revolution, no one thought that it would be the world of information technology now.Obviously, Chinese investors don’t want to wait another ten or eight years. By then, they will have missed the earliest opportunity to deploy in this great change. The market has long been monopolized by others, and the day lily will be cold by then.

For the big data companies currently on the market, we can roughly divide them into three categories: the first category has a large amount of user information, and solves problems such as precise marketing and personalized advertising promotion of the company through big data analysis of user information. .Such as Amazon, Google and Facebook.The second category is to provide other companies with an overall solution of "hardware + software + data" by integrating information and applications of big data.Such as IBM and HP.The third category provides individual solutions in a more targeted manner by selling data and services.This category is basically emerging startups.What needs to be pointed out is that, as the third type of emerging start-up companies, they commercialize big data, which will trigger a new wave of entrepreneurship and industrial revolution after portal websites, search engines, and social networks, and will definitely affect Traditional consulting firms have had a strong impact.

However, if we only frame the influence of big data on the impact on traditional consulting companies, we seem to underestimate its power.Big data analysis has a certain continuity with traditional data analysis and data mining. The key difference is that the amount of data it analyzes is larger, and most of them are unstructured data.For example, many small videos, or various evaluations, listings, etc. in e-commerce.This is very different from traditional data analysis that uses cookies to obtain structured data such as how many times a user logs in to a website every month, and is very different in terms of technical processing.

However, in terms of what big data analysis fundamentally needs to do, it still needs to analyze these large amounts of data and derive some patterns (modes, methods) that are helpful for business decision-making.Its application space will be very broad.

Regarding the phenomenon that some domestic advertising platform companies and market companies are launching big data business one after another, we have to point out that many of them do not really understand their own needs, nor do they understand what big data means.In practical applications, big data mainly includes three categories: big transaction data, big interaction data and big machine data.The first type of large transaction data has existed for many years, including transaction data from traditional banks and telecommunications to various online banking payment data; the second type of large interaction data mainly refers to data from Facebook, Twitter, Weibo, etc. Unstructured data of social networks; the third type of big machine data refers to the data generated by various sensors in the Internet of Things.

If you don't really do these three types of work and deal with the relationship between them, even if you start big data, you can only look alike and spend a lot of money, but you won't get much actual value.

Now, Chinese bosses have seen the value of "data wealth" in their eyes, and most of them are taking some actions.This is a huge sunrise industry. Only in the industrial chain of big data itself, it can be divided into four links: data collection, data cleaning, data analysis, and vertical industry algorithms.However, since China's market size has just started and the division of labor has not yet been refined, China's big data pioneers have to start from scratch, and even on the basis of combining foreign experience, they have to cross the river by feeling the stones before they can step by step sum up a solution that suits China's national conditions. Big data application strategy.

Alibaba: Cloud Empire Vision
I am still deeply impressed by a sentence Ma Yun said: "If you don't move, you will die!" Since its establishment, Alibaba has always followed a strategic principle of "If you don't move, you will die." In 1999, Jack Ma founded Alibaba as a business-to-business online trading platform. In 2003, another 1 million yuan was invested to establish Taobao.com. In 2004, Alibaba began to launch the Alipay service, providing an intermediary-based secure transaction platform for China's e-commerce market.

Taobao and Alipay have become the two complementary pillars of Alibaba in the field of e-commerce, and have become the most powerful e-commerce company in China.But Ma Yun did not stop the pace of expansion, and Alibaba continued to "move".He first purchased AutoNavi Maps, invested in Sina Weibo, and increased capital in UC.After the tenth anniversary event, Ma Yun announced his resignation, but Alibaba's "cloud empire" has just begun.

☆Communication channel--Tianxiawangshang
With communication channels, the spread of the brand is guaranteed.Communication is also an important platform for information radiation, so the more information-rich society is, the more important the media will be.In 2010, Alibaba and Zhejiang Publishing United Group worked hard to create a new media "Tianxia Wangshang" to provide information dissemination services for its e-commerce field. Brand status has been enhanced more broadly.

☆Core data source - its flagship product

Good products are the foundation of success, and almost all products under Alibaba are a powerful source of data.Since 1999, products including Alibaba Yellow Pages, Taobao.com, Tmall, Yitao, Juhuasuan, Aliwangwang and other products have emerged one after another. They are unique and occupy a leading position in the industry, providing countless substantial content for Ali’s brand strategy. It also provides solid core data for Ali's big data strategy, and fulfills Jack Ma's dream of an all-network strategy.

☆Core Technology--Alibaba Cloud

(End of this chapter)