Yarn consists of two parts: resource manager and node manager. The former is mainly responsible for the resource scheduling management of the whole cluster, while the latter is mainly responsible for the resource and task management on the server.

Resource Manager: It consists of two parts-scheduler and application manager. The application administrator is responsible for submitting the application and monitoring its running status.

Yarn's entire workflow:

1. The customer submits the application to yarn.

2. The resource manager process communicates with the node manager process, allocates the first container for the user program according to the cluster resources, distributes the mp applicationmaster to the container, and starts the mp applicationmaster in the container.

3. After the 3.mp applicationmaster is started, it immediately registers with the resource management node process and applies for container resources for its own application.

4. After the 4.mp applicationmaster applies for the required container, it immediately communicates with the corresponding node manager process, distributes the user mp program to the server where the node manager process is located, and runs it in the container, which is the mp or reduce task.

5. The MP or reduce task communicates with the mp applicationmaster during its running and reports its running status. If the run is over, mp? The applicationmaster logs off from the resource manager process and releases all container resources.

The 5-20-2 rule of learning new knowledge;

5 minutes to understand the characteristics, application scenarios and problems to be solved of this new knowledge.

20 minutes to understand its main design principles, core ideas and concepts.

2 hours to see the key details, try it out or make a demonstration.