Hive安装
Hive安装
1. 集群环境
192.168.43.79 slave1
192.168.43.32 slave2
192.168.43.205 master
2. 安装mysql
默认情况下，hive的元数据是保存在内嵌的derby数据库里，但大多情况都是使用mysql来存储hive元数据，需要先安装mysql
版本：mysql5.7
CentOS 7版本下载
[root@master ~]# rpm -Uvh https://repo.mysql.com//yum/mysql-5.7-community/el/7/x86_64/mysql57-community-release-el7-10.noarch.rpm
[root@master ~]# yum list
[root@master ~]#yum -y install mysql-community-server
[root@master ~]#systemctl start mysqld
[root@master ~]# lsof -i :3306
第一次通过# grep "password" /var/log/mysqld.log 命令获取MySQL的临时密码
[root@localhost ~]# grep "password" /var/log/mysqld.log
2018-05-24T09:51:57.585660Z 1 [Note] A temporary password is generated for root@localhost: sJ?dm)NB2%o+
用该密码登录到服务端后，必须马上修改密码，不然操作查询时报错误
刚开始设置的密码必须符合长度，且必须含有数字，小写或大写字母，特殊字符。
如果想设置简单密码，如下操作：
方法一：首先，修改validate_password_policy参数的值
mysql> set global validate_password_policy=0; #定义复杂度
mysql> set global validate_password_length=1; #定义长度 默认是8
mysql>set password for 'root'@'localhost'=password('311311');
mysql> flush privileges;
方法二：在/etc/my.cnf 可关闭密码强度审计插件，重启mysql服务
validate-password=ON/OFF/FORCE/FORCE_PLUS_PERMANENT: 决定是否使用该插件(及强制/永久强制使用)。
查看安装后的文件
[root@localhost ~]# rpm -qa | grep mysql
mysql-community-server-5.7.22-1.el7.x86_64
mysql57-community-release-el7-10.noarch
mysql-community-common-5.7.22-1.el7.x86_64
mysql-community-client-5.7.22-1.el7.x86_64
mysql-community-libs-compat-5.7.22-1.el7.x86_64
mysql-community-libs-5.7.22-1.el7.x86_64
[root@localhost ~]# rpm -ql mysql-community-server-5.7.22-1.el7.x86_64
/etc/logrotate.d/mysql
/etc/my.cnf
/usr/sbin/mysqld
/var/lib/mysql
/var/lib/mysql-files
/var/run/mysqld
授权root可以远程连接访问
mysql> grant all privileges on *.* to root@'%' identified by '123456';
Query OK, 0 rows affected, 1 warning (0.08 sec)
mysql> flush privileges;
Query OK, 0 rows affected (0.11 sec)
或者这样
mysql授权用户远程登录
mysql > use mysql;
mysql > show tables;
mysql > select host from user;
mysql > update user set host='%' where user ='root'; #需要授权的用户
重启mysql
创建hive用户，并授予所有权
mysql> create user 'hive' identified by 'hive';
mysql> grant all privileges on *.* to 'hive' with grant option;
mysql> flush provoleges;
用hive用户登录数据库，并创建hive数据库
# mysql -uhive -phive
mysql> create database hive;
mysql> use hive;
mysql> show tables;
3. 安装hive
# wget https://mirrors.tuna.tsinghua.edu.cn/apache/hive/hive-3.1.1/apache-hive-3.1.1-bin.tar.gz
[root@master src]# tar -xvf apache-hive-3.1.1-bin.tar.gz -C /opt/
[root@master src]# cd /opt/
[root@master opt]# mv apache-hive-3.1.1-bin/ hive-3.1.1
添加环境变量
[root@master ~]# vim /etc/profile
####################hive
export HIVE_HOME=/opt/hive-3.1.1
export PATH=$PATH:$HIVE_HOME/bin
[root@master ~]# source /etc/profile
[root@master ~]# hive --version
4. 修改Hive配置文件
[root@master conf]# cp hive-default.xml.template hive-site.xml
[root@master conf]# vim hive-site.xml
首先3210行的“”是无线字符，删掉，删掉后如下：
3210 Ensures commands with OVERWRITE (such as INSERT OVERWRITE) acquire Exclusivv
e locks for transactional tables. This ensures that inserts (w/o overwrite) runnn
ing concurrently
在原有中添加进去，注意最后一个<configuration>是原本的，添加的内容要在这个里面
<!-- 指定HDFS中的hive元数据地址 -->
<property>
<name>hive.metastore.warehouse.dir</name>
<value>/opt/hive-3.1.1/hive/hdfs</value>
</property>
<property>
<name>hive.exec.scratchdir</name>
<value>/opt/hive-3.1.1</value>
</property>
<!-- 该属性为空表示嵌入模式或本地模式，否则为远程模式 -->
<property>
<name>hive.metastore.uris</name>
<value></value>
</property>
<!-- 指定数据库连接地址名称 -->
<property>
<name>javax.jdo.option.ConnectionURL</name>
<value>jdbc:mysql://master:3306/hive?createDatabaseIfNotExist=true</value>
</property>
<!-- 指定驱动类 -->
<property>
<name>javax.jdo.option.ConnectionDriverName</name>
<value>com.mysql.jdbc.Driver</value>
</property>
<!-- 指定用户名 -->
<property>
<name>javax.jdo.option.ConnectionUserName</name>
<value>hive</value>
</property>
<!-- 指定密码 -->
<property>
<name>javax.jdo.option.ConnectionPassword</name>
<value>hive</value>
</property>
<property>
<name>hive.metastore.schema.verification</name>
<value>false</value>
<description>
</description>
</property>
<!—清除登录警告 -->
<property>
<name>hive.metastore.local</name>
<value>false</value>
<description>controls whether to connect to remove metastore server or open a new metastore server in Hive Client JVM</description>
</property>
<property>
<name>hive.metastore.schema.verification</name>
<value>false</value>
<description>
</description>
</property>
添加以上内容之后再修改配置文件
将配置文件中所有的${system:java.io.tmpdir}改为/opt/hive-3.1.1/tmp
并将此文件夹赋予读写权限，将${system:user.name}更改为hive
修改：
:% s/${system:java.io.tmpdir}/\/opt\/hive-3.1.1\/tmp/g
:% s/${system:user.name}/hive/g
修改后
图片.png
修改配置文件hive-env.sh
[root@master conf]# cp hive-env.sh.template hive-env.sh
[root@master conf]# vim hive-env.sh
export HADOOP_HOME=/opt/hadoop-2.8.4/
export HIVE_CONF_DIR=/opt/hive-3.1.1/conf
export HIVE_AUX_JARS_PATH=/opt/hive-3.1.1/lib
5. 创建要使用的目录并授权
[root@master hive-3.1.1]# mkdir tmp
[root@master hive-3.1.1]# mkdir hive
[root@master hive-3.1.1]# mkdir hive/hdfs
Hdfs创建文件
[root@master hive-3.1.1]# hadoop fs -mkdir -p /opt/hive-3.1.1/hive/hdfs
授权
[root@master hive-3.1.1]# hadoop fs -chmod -R 777 /opt/hive-3.1.1/
查看有没有创建成功
[root@master hive-3.1.1]# hadoop fs -ls /opt/hive-3.1.1/hive/
Found 1 items
drwxrwxrwx - root supergroup 0 2019-06-26 12:10 /opt/hive-3.1.1/hive/hdfs
6. 安装MySQL连接工具
#Master
1）下载安装包
wget https://cdn.mysql.com//Downloads/Connector-J/mysql-connector-java-5.1.44.tar.gz
##wget https://cdn.mysql.com//archives/mysql-connector-java-5.1/mysql-connector-java-5.1.46.tar.gz
tar zxvf mysql-connector-java-5.1.44.tar.gz
2）复制连接库文件
cp mysql-connector-java-5.1.44/mysql-connector-java-5.1.44-bin.jar /opt/hive-3.1.1/lib/
7. 初始化数据库
生成元数据数据库，这里元数据数据库前面配置是hive，但是我前面已经创建有hive了，。。。。。。
[root@master ~]# schematool -initSchema -dbType mysql
SLF4J: Class path contains multiple SLF4J bindings.
。。。。。。
Starting metastore schema initialization to 2.1.0
。。。。。。
schemaTool completed
8. 拷贝安装包
#Master
[root@master ~]# scp -r /opt/hive-3.1.1/ slave1:/opt/
[root@master ~]# scp -r /opt/hive-3.1.1/ slave2:/opt/
还要传/etc/profile给其它两台服务器：source /etc/profile
9. 启动Hive服务
#Master
hive
报错：
Mon Jul 02 08:17:07 CST 2018 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.
解决：
jdbc:mysql://localhost:3306/Peoples?autoReconnect=true&useSSL=false
jdbc:mysql://master:3306/hive?createDatabaseIfNotExist=true
jdbc:mysql://master:3306/framework?characterEncoding=utf8&;useSSL=true
jdbc:mysql://master:3306/hive?characterEncoding=utf8&useSSL=false