This paper is divided into two parts. The first part introduces how to deploy Ambari in Docker virtualization environment. The second part introduces how to deploy and manage Hadoop cluster based on Ambari.
# # # Environmental information
* Docker release: Docker for Mac
* Docker version: 17.06.2-ce
* Docker container operating system: Ubuntu 14.04.
* Ambari version: 2.5.2.0
Docker environment preparation
Pull Docker Image: Execute the command docker pull ubuntu: 14.04 on the host to get the image of ubuntu from the remote warehouse, and also get the image of other OS. This paper takes Ubuntu as an example.
Start the Docker container: Execute the following command to start the container based on ubuntu: 14.04 image:
Docker run-itd-name ambari _ new-p8080: 8080-p3306: 3306-v/users/Yu Xiaolei/workspace/dockershared:/dockershared Ubuntu:14.04/bin/bash.
Because Ambari occupies port 8080 when starting the Web program, to access Ambari page from Docker host, port mapping is needed through parameter -p;
As a novice, after deploying Ambari in the container, I found that the 8080 port of the webpage and the 3306 port of MySQL (optional) were not exposed on the Docker host, so I couldn't log in to Ambari from the host through the browser, so I had to find a way to open the port on the existing container.
There are two ways:
1) If the host is a Linux system, modify the mapping rules of the designated ports of the iptables firewall;
2) If it is a non-Linux system, you can submit the container with Ambari as a new image, and then create a new container based on this image. At this point, you can specify the port mapping by adding the parameter -p to the docker run command.
Another problem is that Ambari stores its data in a database and supports MySQL, PostgreSQL and other databases. After MySQL is installed in the container, in the new container created based on the previous step, it will be found that MySQL cannot be started, and the command /etc/init.d/mysql restart failed to start. The log file of /var/log/mysql/error.log is printed as17080214: 02: 59 [error]. Fatal error: Unable to open and lock permission table: Error 140 obtained from storage engine. After searching for data on the Internet, you need to add the parameter -v /var/lib/mysql when creating the container, and declare the mysql data storage path as a data volume, which can solve the problem.
After starting the container, execute the docker exec -it ambari /bin/bash command to enter the container.
Anbari device
Configure Ubuntu's depot source:
In China, it is recommended to use the software source code of Alibaba Cloud, open the /etc/apt/sources.list file with vim under the root account, delete all the contents of the file, and paste the following contents:
Deb /ubuntu/ trusty main restricted universe multiverse
Deb/Ubuntu/trusty- secure main restricted universe multiverse
Deb/Ubuntu/trusty- update the main restricted universe multiverse
Deb/Ubuntu/trusty- the multiverse proposed by the main restricted universe
Deb /ubuntu/ trusty-backports main restricted universe multiverse
Deb-src /ubuntu/ trusty main restricted universe multiverse
Deb-src /ubuntu/ trusty-security main restricted universe multiverse
Deb-src/Ubuntu/trusty- update the main restricted universe multiverse
Deb-src/Ubuntu/trusty- the multiverse proposed by the main restricted universe
Deb-src/Ubuntu/trusty-backports main restricted universe multiverse
Execute apt-get update command to update the software list.
Install the software that Ambari relies on.
Apt-get installation software-properties-general
sudo add-apt-repository PPA:webupd 8 team/Java
Sudo apt-get update
Sudo apt-get installs oracle-java8-installer.
Sudo apt-get installs oracle-java8-set-default.
After the installation, add the command "export Java _ home =/usr/lib/JVM/Java-8-Oracle" at the end of ~/. Bashrc file to configure JAVA\_HOME environment variables.
Create database ambari Grant all permissions on * with ambari. * to "root" @ "%"identified by "root_password" with GRANT option; Refresh permissions; Quit;
Because Ambari's data is stored in MySQL database, a separate database is created for Ambari and given full control. Description: Assume that the password of MySQL database root user is: root\_password.
Modify the configuration of ` /etc /etc/mysql/my.conf, comment out ` skip-external-locking', and ensure that ` bind-address = 0.0.0.0, so that remote hosts can access MySQL.
Execute the command` /etc /etc/init.d/mysql restart to restart the mysql service.
Time synchronization server ntp: execute apt-get install ntp command to install ntp time synchronization server, so that the clocks of all nodes in the cluster environment are consistent; Execute the command sudo service ntp restart to restart the ntp service.
Mysql: Execute the command apt-get install mysql-server to install MySQL server. After the installation is completed, execute the command mysql -uroot -proot to enter the mysql client and execute the following sql code:
Oracle JDK: execute the following commands one by one to add the webupd8team (/Ambari/Ubuntu14/2.x/updates/2.5.2.0/ambari.list) to download the ambari source file from HortonWorks warehouse, and do not modify the list file name after downloading;
Execute the command apt-keyadv-recv-keys-keyserver keyserver.ubuntu.com b9733a 7a07513cad to trust the GPG signature of the remote warehouse.
Execute apt-get update command to update Ambari software source.
Execute the command apt-get install ambari to install the ambari suite. Because the software package is relatively large (more than 700 MB), we have to wait patiently here. However, apt-get supports breakpoint downloading, and the network terminal will not download from the beginning when it executes the command again.
Configure Ambari:
Execute the command mysql -uroot -proot to enter the mysql client, execute the command source ambari to enter the database of ambari, and execute the command Source/var/lib/Ambari-server/resources/Ambari-DDL-MySQL-create.sql to complete the database table initialization operation of Ambari;
Execute the command Ambari-server setup to start the guided configuration operation of Ambari and configure it according to the instructions. It should be noted that Ambari should not download JDK from the network. Instead, you should choose a custom path and enter /usr/lib/jvm/java-8-oracle.
Start Ambari: execute the command ambari-server start, and the storage path of the startup log is/var/log/ambari-server/ambari-server.log.
After startup, because we have mapped the port of the Docker container before, we can open a browser on the host and enter http://localhost:8080 to access the Ambari login page.
The login user name and password are admin. After logging in, you can see Ambari's homepage, as shown in the following figure:
{% asset_img