Big data platform computing power

1. The strength of the company is very big, not bad

2. With the development and integration of artificial intelligence, big data and computing power, the three have been organically integrated into an intelligent whole. Its connotation and denotation tend to be diversified, and the applications in various subdivision fields are also rich and superimposed. You have me and I have you. The difference and boundary between artificial intelligence, big data and computing power are more and more blurred
at this stage, the application of artificial intelligence and big data has penetrated into various fields such as instry, agriculture, medicine, national defense, economy, ecation and so on, and the commercial and social value generated is almost unlimited. With the development of artificial intelligence and Internet of things, cloud computing is no longer limited to storage and computing, and has become an important driving force for the development and transformation of various instries. You can learn more about AI, big data and computing power on the 10th power computing power platform.

3. We search every time in the search engine, browse and purchase every time in the e-mall, and pay every time. These seemingly unrelated data can be summarized and analyzed to describe your behavior and habits, and make a high probability prediction of your future behavior, We can call these data customer big data
at the time of the rise of the mobile Internet, everyone was scrambling for online traffic and data. But China Internet, you know, basically, big data of huge consumers is in bat's hands, and it is difficult for small Internet companies to obtain core data. But with the development of offline consumption upgrading, more and more people begin to see the importance of offline customers' big data. After all, offline stores are the main battlefield of customers' consumption, and the traffic has not been divided up by giant enterprises like bat, which can be regarded as a blue ocean full of business opportunities
Blue Ocean belongs to blue ocean, but there is also a problem, that is, offline customers' big data is too large and scattered. In addition to Starbucks and McDonald's, which are big enterprises with the ability to collect, it is difficult for general stores to establish their own big data platform, let alone the intelligent processing of big data
in this regard, as far as I know, there is a smart store enterprise named Zhangbei, which specializes in serving the offline store market. This is a store marketing tech intelligent marketing company. Relying on the store big data precipitated by the integration of business entrance, it helps merchants build their own customer big data platform and realize automatic precision marketing, so as to drive the return of old customers and the drainage of new customers. It can be said that it's just the key to the customer big data market. People who are interested can go to understand it.

4. Big data is actually a lot of knowledge accumulation, and then select some useful and valuable information from it

5.

The construction and application of financial big data platform are two parts. For the financial big data platform, these two parts are very important. So in the following part, we will elaborate from the two perspectives of big data platform and which indicators banks can analyze

The overall architecture of big data platform can be composed of the following parts:

1. A customer

Customer theme: customer attributes (customer number, customer category), indicators (total assets, procts held, number of transactions, transaction amount, RFM), contract signing (channel contract signing 2. Made a transaction

subject: transaction financial attribute, business category, payment channel

3. Which account to use

account theme: account attributes (customer, account opening date, branch, proct, interest rate, cost) constitute a wide table

4. Through which channel

channel theme:

channel attributes, dimensions, limits constitute a wide table

5; Proct

Proct theme: proct attributes, dimensions, index composition wide table

< H2 > III. case

in view of the space problem, here you can refer to this article:

Huaxia Bank: big data technology service business demand, realize high-speed sales growth

6.

1. Big data refers to the data set that cannot be captured, managed and processed by conventional software tools within a certain time range. It is a massive, high growth rate and diversified information asset that needs new processing mode to have stronger decision-making power, insight and process optimization ability, The relationship between big data and cloud computing is as inseparable as the positive and negative sides of a coin. Big data can not be processed by a single computer, so it must adopt distributed computing architecture. It is characterized by massive data mining, but it must rely on cloud computing distributed processing, distributed database, cloud storage and virtualization technology

you can understand the relationship between them in this way, cloud computing technology is a container, big data is the water stored in this container, big data is to rely on cloud computing technology for storage and computing

< H2 > extended data:

4V characteristics of big data: Volume (large), velocity (high speed), variety (diversity), value (value)

the key word of cloud computing is "integration". Whether you use the mature traditional virtual machine segmentation technology or the massive node aggregation technology later used by Google, it integrates massive server resources through the network and allocates them to users, So as to solve the problem caused by the lack of storage and computing resources

big data is a new topic brought about by the explosive growth of data, such as how to store the massive data proced in the Internet era, how to effectively use and analyze these data, and so on

trend of big data:

trend 1: resource utilization of data

what is resource utilization? It means that big data has become an important strategic resource for enterprises and society, and has become a new focus for everyone. Therefore, enterprises must make big data marketing strategic plan in advance to seize the market opportunity

trend 2: deep combination with cloud computing

big data cannot do without cloud processing, which provides elastic and scalable infrastructure for big data and is one of the platforms to generate big data. Since 2013, big data technology has been closely combined with cloud computing technology, and it is expected that the relationship between them will be closer in the future. In addition, the Internet of things, mobile Internet and other emerging computing forms will also help the big data revolution and make big data marketing play a greater role

With the rapid development of big data, just like computers and the Internet, big data is likely to be a new round of technological revolution. The rise of data mining, machine learning, artificial intelligence and other related technologies may change many algorithms and basic theories in the data world, and achieve breakthroughs in science and technology

< H2 > reference: network big data network cloud data

7.

There are about four major work directions in the field of big data, in addition to big data platform application and development, big data analysis and application, and big data platform integration and operation and maintenance , there are big data platform architecture and R & d , in addition to the above four major work directions, Another work direction is big data technology promotion and training , which many people are engaged in at present

big data platform application development is a hot direction of employment at present. On the one hand, there are many scenes of big data development, on the other hand, the difficulty is not high, and the number of employees who can accept is also very large. Big data development is mainly to meet the application development of enterprises on the big data platform, which is closely related to scenarios

8.

Big data has three main parts: mathematics, statistics and computer science. The basic knowledge of big data often determines the growth height of developers in the future, so we should pay attention to the learning of basic knowledge

big data platform is a series of technical platforms for the collection, storage, calculation, statistics, analysis and processing of massive structured, unstructured and semi institutional data. The amount of data processed by big data platform is usually TB level, even Pb or EB level data, which cannot be processed by traditional data warehouse tools. The technologies involved include distributed computing, high concurrency processing, high availability processing, clustering, real-time computing, etc., which gather all kinds of popular technologies in the current it field

extended data:

precautions:

the first stop of big data is to collect and store massive data (public / private). Now everyone is a huge data source, releasing a lot of personal behavior information through smart phones and personal laptops. The first mock exam of data collection is the high speed requirement of massive data and the comprehensive consideration of data. p>

the traditional business intelligence method in data cleaning (ETL) is to put the accurate data into the defined format, and generate high-dimensional data through the basic extraction statistics, which is convenient for direct use. However, big data has one of the most prominent features - unstructured or semi-structured data. Because the data may be pictures, binary and so on. The biggest challenge of data cleaning is how to transform and process a large number of unstructured data for distributed computing and analysis

9.

Java: as long as you know some basics, you don't need deep Java technology to do big data. Learning java se is equivalent to learning big data. Basic

linux: because big data related software runs on Linux, Linux needs to learn more solidly. Learning Linux well will help you to master big data related technology quickly, which can make you better understand the running environment and network environment configuration of Hadoop, hive, HBase, spark and other big data software, and can avoid many pitfalls, If you learn shell, you can understand the script, which makes it easier to understand and configure the big data cluster. It can also make you learn the new big data technology faster

it's easy to finish talking about the basics. What big data technologies need to be learned can be learned in the order I write

oozie: now that you have learned hive, I believe you need this thing. It can help you manage your hive or maprec, spark scripts, check whether your program is executed correctly, send you an alarm when there is a mistake, and help you try the program again. The most important thing is to help you configure the dependency of tasks. I'm sure you'll like it, or you'll feel like shit when you look at all the scripts and the dense crond

HBase: This is the NoSQL database in the Hadoop ecosystem. Its data is stored in the form of key and value, and the key is unique, so it can be used for data plication. Compared with MySQL, it can store a lot of data. So it is often used for the storage destination after the completion of big data processing

Kafka: This is an easy-to-use queue tool. What is queue for? Line up to buy tickets, don't you know? When there is too much data, you also need to queue up to process it. In this way, other students who work with you will not shout. Why do you give me so much data (such as hundreds of gigabytes of files) and how can I handle it? Don't blame him because he doesn't work in big data. You can tell him that I put the data in the queue and take them one by one when you use it, In this way, he won't complain any more. He will go to optimize his program immediately, because it's his business if he can't handle it. It's not the question you give. Of course, we can also use this tool to store online real-time data or HDFS. At this time, you can use it with a tool called flume, which is specially used to provide simple data processing and write to various data receivers (such as Kafka)

spark: it is used to make up for the shortcomings in the speed of data processing based on maprec. It is characterized by loading data into memory for calculation, rather than reading the slow hard disk which is dying and evolving very slowly. It's especially suitable for iterative operations, so the algorithm streams are particularly interested in it. It's written in scala. Java language or Scala can operate it, because they all use the JVM

10. Piwik, aggregation analysis, quantum statistics, Google Analytics, cnzz.