Big data in China

Chapter 18 Big Data and Technological Change (2)
At present, the real big data players in the data service industry chain should be companies such as Google, which obtain benefits by reusing data.Google has successfully established a "web search + advertising" business model, and all its businesses are built on big data.Therefore, Google is the biggest player in big data. In 2012, its total revenue reached 501.75 billion US dollars, with a profit of 107.4 billion US dollars, 2017% of which came from advertising.A consulting firm predicts that the market space for global big data technology in 500 will be approximately US$2012 billion, approximately equal to Google's total revenue in [-].This includes not only technology, but also big data tools and corresponding services.

From this point of view, in the future of the big data era, the biggest profit will be the "data is king" or "data-driven" business connotation and model. It is imperative to develop big data and tap new value of big data. The driving force.China must also support related industries and companies as soon as possible to counter multinational giants like Google and eventually catch up.

Technical support and development
Big data is not a slogan, but technology, and it is also the integration of technology.The arrival of big data has become an inescapable challenge in real life.In any case, big data has become the strongest voice of the new round of technological change.Thinking about the model, questioning about security, and exploring about applications, we must calm down and look at big data, and really understand the problems that big data still needs to solve.

The national economy, the people's livelihood, and business innovation are all related to big data. Big data has gradually shown people the huge opportunities it brings to academia, industry, and government.Whenever we need to make a decision, big data is everywhere.In any case, we must face the arrival of the era of big data.

The huge challenges that big data brings to China are three important technical issues.

☆How to use information technology and other means to process unstructured and semi-structured data

An important feature of big data is data dispersion.In big data, 85% is unstructured data, and structured data only accounts for about 15%.Another characteristic of big data is uncertainty, which is manifested in high-dimensionality, changeability, and strong randomness.90% of the data comes from open source data, and the rest is stored in databases.Big data exists in large quantities in areas such as social networks, the Internet, and e-commerce.

It is worth noting that big data stimulates a large number of research questions.However, each representation of big data only presents the side performance of the data itself, not the whole picture.For example, an image, how to convert it into a multidimensional data table, an object-oriented data model or a data model directly based on an image?

If the process of extracting "rough knowledge" through data mining is called "primary mining", then the process of combining rough knowledge with quantified subjective knowledge to generate "intelligent knowledge" is called "secondary mining".These structured rough knowledge can be processed and transformed by subjective knowledge to generate semi-structured and unstructured intelligent knowledge, which are some new features of structured rough knowledge produced by data mining based on big data.

Due to the semi-structured and unstructured characteristics of big data, seeking "intelligent knowledge" also reflects the core value of big data research.The individual performance, general characteristics and basic principles of unstructured and semi-structured data are not yet clear. In order to achieve a quantitative and qualitative leap from "primary mining" to "secondary mining", it is necessary to include mathematical , economics, sociology, computer science and management science, including multi-disciplinary research and discussion.These need to be given a semi-structured or unstructured data, including specific experience, common sense, instinct, situational knowledge and user preferences.

☆How to explore the complexity of big data, the description method of uncertain characteristics and the system modeling of big data

The complex form of big data makes many measurement and evaluation of "rough knowledge" particularly important.The breakthrough of this problem is the premise and key to realize the knowledge discovery of big data.Here, human-computer interaction will play a crucial role.Management science, especially optimization-based theories, will play an important role in developing general methods and regularities for knowledge discovery in big data.

In the short term, the academic community encourages the development of transformation principles between semi-structured and unstructured data to support cross-industrial applications of big data.From a long-term perspective, known optimization, data envelopment analysis, expectation theory, and utility theory in management science can be applied to the "secondary mining" process to study how to integrate subjective knowledge into the rough data produced by data mining. knowledge.The challenges brought about by the individual complexity and randomness of big data will promote the formation of the mathematical structure of big data, which will lead to the completion of the unified theory of big data.

☆The influence of the relationship between data heterogeneity and decision-making heterogeneity on big data knowledge discovery and management decision-making
In the big data environment, management decision-making is faced with two "heterogeneity" problems: "decision heterogeneity" and "data heterogeneity".Big data has changed the traditional model of management decision-making structure.Changes in the decision-making structure require people to explore how to do "secondary mining" to support higher-level decision-making.Exploring the impact of changes in the decision-making structure in the big data environment on the management decision-making structure will become an open scientific research problem.Searching for scientific models of big data will lead to the exploration of general methods for studying the beauty of big data, and known data mining methods will become tools for big data mining.

Regardless of the data heterogeneity brought about by big data, the "rough knowledge" in big data can still be regarded as the category of "one-time mining".Due to the complexity of big data itself, this issue is undoubtedly an important scientific research topic. The traditional management decision-making mode depends on the learning of business knowledge and accumulated practical experience, while management decision-making is based on data analysis.Big data is a man-made nature with hidden laws. If we find a way to convert unstructured and semi-structured data into structured data, we can use the "intelligent knowledge" generated by "secondary mining" as data anomalies. If there is a bridge between heterogeneity and heterogeneity of decision-making, then we will be able to deal with the new challenges raised by traditional data mining theory and technology well.Although such exploration is very difficult, it is very necessary to study big data.

In addition, there are some data science problems, and the above is just a starting point for studying big data challenges.In the future, related problems can be well resolved.

Since humans entered the information age, we have continuously generated a large amount of data, coupled with the large-scale explosion of Internet of Things and mobile Internet applications, a large amount of new data is growing at a rate of 50% per year, or doubled every two years many.Data has penetrated into every industry and service functional area. With the continuous development of Internet technology, data itself is an asset, which has formed a consensus in the industry.

People's use of massive amounts of data will herald a new wave of productivity growth and consumer surplus.In the era of cloud computing, human beings obtain business and social value through the efficient analysis of massive big data.With the advent of the cloud era and the popularity of mobile terminals, the main body of data creation has gradually shifted from enterprises to individuals, and most of the data generated by individuals is unstructured data such as pictures, documents, and videos.With the rapid popularization of cloud computing technology, human society is entering an era of big data detonated by the Internet and communication technology.The development prospect of big data technology in China is bright, the premise is that we can improve and expand our technology kingdom and build a beautiful blueprint.

Gartner, a global technology research and consulting company, listed big data technology as one of the top ten technologies and trends of strategic significance to many companies and organizations in 2012. Gartner regards big data technology as a transformational technology in its emerging technology maturity curve, which means that big data technology will enter the mainstream in the next 3 to 5 years.China will not be left behind either. As the earliest domestic leader rooted in cloud computing technology and business model, "Cloud Base" has been actively paying attention to the development opportunities brought by big data.

From the strategic to the tactical level, from the concept to the technical level, China has begun its own evolution to better adapt to this new era.After decades of accumulation in China, the continuously generated massive data is becoming an inexhaustible source of energy for the virtual world, but they are still far from being developed.

The popularization of information technology has enabled more office processes of Chinese enterprises to be realized through the Internet, and the resulting data is also mainly unstructured data.Research in other fields, such as cloud computing, next-generation analysis, and in-memory computing, also complements the research on big data.We are still not sure whether there is data in everything, but at least we have opened such a door: think about big data with a rational attitude, jointly maintain the driving force for continuous change, and actively embrace this change.As early as 2012, unstructured data had reached more than 75% of the entire Internet data volume, and the big data used to extract wisdom is often these unstructured data.And now, this proportion has become larger, and we also have enough technical support.In other words, China's accumulation of big data technology has reached a breakthrough stage.

"Footprint Tracking"--Personalized Data Recommendation System

Don’t be surprised if you see things like “guess what you like” on the websites you often shop online that meet your requirements, because each of us has entered the era of big data.You can imagine that in the future, when you turn on the computer every day, it will automatically make a list of all your needs, and you only need to sit on the comfortable sofa and click a few confirmation options, and you can easily get everything done.

Don't think this happens only in sci-fi movies.Merchants can only sell products when they meet the needs of the public, and all of this is realized on the basis of meeting the individual needs of the public.

In September 2011, Taobao launched a user-customized TV campaign, and 9 customized TVs were sold out within two days.In this activity, users can choose various attributes of the TV, including size, frame, color, etc., and manufacturers will produce TV products according to the user's customized content, and then deliver them to customers' homes.

From this representative case, we can find that the future business model is undergoing qualitative changes. It improves the efficiency of business operations by meeting individual needs, and obtains better services while providing consumers with better services. much profit.

☆The Origin of "Guess You Like"
How did the "guess you like it" that can be seen everywhere in online shopping come from?In fact, this recommendation method comes from Amazon's technological innovation.

Amazon's content was originally done manually, and they hired a book review team of 20 people to recommend interesting new books on the web.But as more and more books are listed on Amazon, such manual operations will naturally become more and more inefficient.

Later, Bezos, the president of Amazon, decided to try a more creative approach, recommending products to users based on their habits.But if you want to realize personalized recommendation, you must compare different users and then find the association between users.However, in the face of huge data, this recommendation system algorithm is cumbersome, and the results are not satisfactory.

(End of this chapter)