大数据平台HDP搭建
一、概述
Apache Ambari是一个基于Web的支持Apache Hadoop集群的供应、管理和监控的开源工具，Ambari已支持大多数Hadoop组件，包括HDFS、MapReduce、Hive、Pig、 Hbase、Zookeeper、Sqoop和Hcatalog等。提供Web UI进行可视化的集群管理，简化了大数据平台的安装、使用难度。
创新互联-专业网站定制、快速模板网站建设、高性价比昂昂溪网站开发、企业建站全套包干低至880元,成熟完善的模板库,直接使用。一站式昂昂溪网站制作公司更省心,省钱,快速模板网站建设找我们，业务覆盖昂昂溪地区。费用合理售后完善，十载实体公司更值得信赖。
二、安装部署
2.1 主机规划
| 序号 | IP地址 | 主机名 | 系统版本 |
|---|---|---|---|
| 1 | 172.20.2.222 | ambari-server | centos7.3 |
| 2 | 172.20.2.203 | hadoop-1 | centos7.3 |
| 3 | 172.20.2.204 | hadoop-2 | centos7.3 |
| 4 | 172.20.2.205 | hadoop-3 | centos7.3 |
2.2 部署
2.2.1 基础环境部署
a.修改主机名配置hosts
systemctl stop firewalld
hostnamectl set-hostname ambari-server #更改个主机名
sed -i 's/SELINUX=enforcing/SELINUX=disable/g' /etc/selinux/config
setenforce 0
cat >>/etc/hosts<b.配置ambari-server免密钥登录其他节点
ssh-keygen -t rsa #在ambari-server生成密钥对
for num in `seq 1 3`;do ssh-copy-id -i /root/.ssh/id_rsa.pub root@hadoop-$num;donec.在ambari-server安装数据库
在ambari的setup中我们可以选择使用默认的postgresql，也可以自定义使用其他数据库，此处选用mariadb，便于后期管理维护
yum -y install mariadb-server MySQL-connector-java
systemctl start mariadb
mysql -uroot password "mysqladmin"
登录数据库后我们采用root登录d.配置本地yum源
由于ambari的各个组件包很大,HDP大约快6G，我们采取离线安装方式，将ambari-server配置为yum源，事先在网上下载HDP、HDP-UTILS和Ambari
软件版本：
参考url：ambari各版本参考链接
HDP和HDP UTILS参考链接
| ambari | HDP | HDP-UTILS |
|---|---|---|
| 2.4.1.0 | 2.5.0.0 | 1.1.0.21 |
yum install httpd -y && systemctl start httpd
mkdir /software && cd /software
#下载软件
wget -c http://public-repo-1.hortonworks.com/ambari/centos7/2.x/updates/2.4.1.0/ambari-2.4.1.0-centos7.tar.gz
wget -c http://public-repo-1.hortonworks.com/HDP/centos7/2.x/updates/2.5.0.0/HDP-2.5.0.0-centos7-rpm.tar.gz
wget -c http://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.21/repos/centos7/HDP-UTILS-1.1.0.21-centos7.tar.gz
#创建目录
mkdir -p /var/www/html/ambari
mkdir /var/www/html/hdp/{HDP,HDP-UTILS-1.1.0.21}
#解压软件
tar -zxvf ambari-2.4.1.0-centos7.tar.gz -C /var/www/html/ambari/
tar -zxvf HDP-UTILS-1.1.0.21-centos7.tar.gz -C /var/www/html/hdp/HDP-UTILS-1.1.0.21/
tar -zxvf HDP-2.5.0.0-centos7-rpm.tar.gz -C /var/www/html/hdp/
配置ambari YUM源
cat >/etc/yum.repos.d/ambari.repo</etc/yum.repos.d/hdp.repo< 2.2.2 安装配置ambari
a.安装ambari server
yum clean all
yum makecache
yum -y install ambari-server
#进入数据库创建ambari库和导入数据
mysql>create database ambari DEFAULT CHARSET utf8;
mysql>use ambari
mysql>source /var/lib/ambari-server/resources/Ambari-DDL-MySQL-CREATE.sqlb.配置ambari
[root@ambari-server hdp]# ambari-server setup
Using python /usr/bin/python
Setup ambari-server
Checking SELinux...
SELinux status is 'enabled'
SELinux mode is 'permissive'
WARNING: SELinux is set to 'permissive' mode and temporarily disabled.
OK to continue [y/n] (y)? y
Customize user account for ambari-server daemon [y/n] (n)? y
Enter user account for ambari-server daemon (root):root
Adjusting ambari-server permissions and ownership...
Checking firewall status...
Checking JDK...
Do you want to change Oracle JDK [y/n] (n)? y
[1] Oracle JDK 1.8 + Java Cryptography Extension (JCE) Policy Files 8
[2] Oracle JDK 1.7 + Java Cryptography Extension (JCE) Policy Files 7
[3] Custom JDK
==============================================================================
Enter choice (1): 1
To download the Oracle JDK and the Java Cryptography Extension (JCE) Policy Files you must accept the license terms found at http://www.or
acle.com/technetwork/java/javase/terms/license/index.html and not accepting will cancel the Ambari Server setup and you must install the JDK and JCE files manually.Do you accept the Oracle Binary Code License Agreement [y/n] (y)? y
Downloading JDK from http://public-repo-1.hortonworks.com/ARTIFACTS/jdk-8u77-linux-x64.tar.gz to /var/lib/ambari-server/resources/jdk-8u77
-linux-x64.tar.gzjdk-8u77-linux-x64.tar.gz... 100% (173.0 MB of 173.0 MB)
Successfully downloaded JDK distribution to /var/lib/ambari-server/resources/jdk-8u77-linux-x64.tar.gz
Installing JDK to /usr/jdk64/
Successfully installed JDK to /usr/jdk64/
Downloading JCE Policy archive from http://public-repo-1.hortonworks.com/ARTIFACTS/jce_policy-8.zip to /var/lib/ambari-server/resources/jc
e_policy-8.zip
Successfully downloaded JCE Policy archive to /var/lib/ambari-server/resources/jce_policy-8.zip
Installing JCE policy...
Completing setup...
Configuring database...
Enter advanced database configuration [y/n] (n)? y
Configuring database...
==============================================================================
Choose one of the following options:
[1] - PostgreSQL (Embedded)
[2] - Oracle
[3] - MySQL / MariaDB
[4] - PostgreSQL
[5] - Microsoft SQL Server (Tech Preview)
[6] - SQL Anywhere
[7] - BDB
==============================================================================
Enter choice (3): 3
Hostname (ambari-server): ambari-server
Port (3306):
Database name (ambari):
Username (root): root
Enter Database Password (mysqladmin):
Configuring ambari database...
Copying JDBC drivers to server resources...
Configuring remote database connection properties...
WARNING: Before starting Ambari Server, you must run the following DDL against the database to create the schema: /var/lib/ambari-server/r
esources/Ambari-DDL-MySQL-CREATE.sqlProceed with configuring remote database connection properties [y/n] (y)? y
Extracting system views...
............
Adjusting ambari-server permissions and ownership...
Ambari Server 'setup' completed successfully.
#启动服务
ambari-server startc.浏览器文本界面安装
浏览器访问：http://172.20.2.222:8080 默认用户名密码admin/admin
启动安装向导
添加将要创建的集群名称
下载hdp版本
选择本地hdp源
添加写需要安装的各个目标节点的主机名，填写ambari-server免密钥登录其他节点的私钥，确认注册
选择需要安装的服务
选择安装软件所在的那个节点上
自定义服务路径或密码
Review
进行安装部署
添加服务
查看各组件的概况和配置
网页名称：大数据平台HDP搭建
转载来于：http://6mz.cn/article/jspssj.html