current position：Home>What does RDD in spark mean?
What does RDD in spark mean?
2022-02-04 16:32:02 【Alibaba cloud Q & A】
spark In the middle of RDD What do you mean ？
Take the answer 1：
RDD yes Spark Soul , Also known as elastic distributed data sets . One RDD Represents a read-only dataset that can be partitioned .RDD There can be many partitions inside (partitions), Each partition has a lot of records (records).Rdd Five characteristics of ： 1. dependencies: establish RDD Dependency of , The main RDD There is a wide and narrow dependence between them , Having a narrow dependency RDD Can be in the same stage Calculation in . 2. partition: One RDD There will be several zones , The size of the partition determines the size of this RDD The granularity of computation , Every RDD All of the partitions are calculated in a single task . 3. preferedlocations: according to “ Mobile data is not as good as mobile computing ” principle , stay Spark When scheduling tasks , Priority is given to assigning tasks to locations where data blocks are stored . 4. compute: Spark The calculation in is based on partition ,compute Function just composes iterators , The results of a single calculation are not saved . 5. partitioner: Is only found in （K,V） Type of RDD in , Not （K,V） Type of partitioner The value is None.
author[Alibaba cloud Q & A],Please bring the original link to reprint, thank you.
The sidebar is recommended
- Is there any difference between on and where in SQL?
- What is the migration scheme for DB2 for LUW, the source of DTS?
- What are the differences between MySQL InnoDB and MyISAM?
- Serverless deconstructs the game industry's pain points. What are the data, operation and maintenance and cost control that customers pay attention to?
- What is the difference in principle between MPP and Mr?
- What is the migration scheme for DTS whose source is DB2 for I?
- What is the migration scheme for Teradata, the source of DTS?
- The principle of database index why use B + tree?
- The source of DTS is self built HBase. What is the migration scheme?
- What is Mars?
guess what you like
What is the migration scheme for the third-party cloud as the source of DTS?
Appbarlayout of Android realizes the effect of hovering adsorption and scaling, and Android develops games
Introduction to the filter interface in Android, androidhook mechanism, even simple actual combat will not get high salary
Common design patterns in Android are known only when you become a project manager
Android - customize the title bar, which is a necessary knowledge point for Android advanced development post
Android - teach you how to develop a smile capture artifact on Android in 30 minutes
Android SMS encryption, contact letter sidebar positioning, Android development will be able to use technology
Running time problems when scanf is different
It's hard to start with small problems in C language
Speech emotion analyzer
- On the linear programming problem, please solve it
- When running Vue, it will report an error and download the resource package. That's it
- How to remove 50Hz power frequency interference
- Help, please call me in C language
- C language programming, can someone help me
- How to call CSH command under Linux with Python
- What data can observation cloud collect
- What application scenarios are cloud observation applicable to?
- How to convert this code difference
- What deployment methods does observation cloud support?
- How do you use DEM to extract the geomorphic parameters of river valley width height ratio? Are there any steps? Red envelope paid
- What's wrong with the proportion of MAC display
- Spring project WebSphere annotation web XML filter error
- I want to ask you a question
- How does "observation cloud" perceive abnormal data?
- How should beginners do this problem
- Power supply mode of DP interface 20pin VDD
- For beginners' exercises, help!
- Quartus simulation stops at this interface
- Pseudo code method for solving missionary and savage problems
- Why is the second output different?
- How to do exercise three and exercise four?
- The cells in latex coincide with the contents
- How to install Gnome on Linux
- Questions about errors caused by multithreaded access to bitmap objects that are currently being used elsewhere
- Questions about C: 2 Randomly generate an integer array intarr (6 elements), and then define a string array strarr to convert the elements in the integer array into strings and store them in the string array
- ABAP database table could not be activated.
- Why does Fiddler show that the network is unavailable after the mobile agent is configured for remote connection
- Creating any project with dev C + + will report the following errors. What's going on.
- Which of the following arrays is not 5 in length?
- What's wrong with this code
- How to compile this solution
- Information matrix in visual slam
- How to do the direct router? Please let me know
- For the interface design of mobile application foundation, it is not clear how to add response mode
- C language code problem, what went wrong
- Who knows vs why
- How can things in idea be exported normally
- On Jacobian matrix in visual slam
- What type of wechat applet does the freshman change develop
- What are the main considerations of MapReduce optimization method?