hadoop + zookeeper +hive + hbase安装学习12页文档资料.docx
- 文档编号:906999
- 上传时间:2022-10-13
- 格式:DOCX
- 页数:15
- 大小:31.96KB
hadoop + zookeeper +hive + hbase安装学习12页文档资料.docx
《hadoop + zookeeper +hive + hbase安装学习12页文档资料.docx》由会员分享,可在线阅读,更多相关《hadoop + zookeeper +hive + hbase安装学习12页文档资料.docx(15页珍藏版)》请在冰豆网上搜索。
hadoop+zookeeper+hive+hbase安装学习12页文档资料
伪分布式安装Hadoop+zookeeper+hive+hbase安装配置
1.安装JDK,配置环境JAVA环境变量
exportJAVA_HOME=/usr/lib/java-1.6.0/jdk1.6.0_37
exportPATH=$PATH:
$JAVA_HOME/bin
exportCLASSPATH=$CLASSPATH:
$JAVA_HOME/lib:
$JAVA_HOME/jre/lib
exportHADOOP_INSTALL=/usr/hadoop/hadoop-1.0.3
exportPATH=$PATH:
$HADOOP_INSTALL/bin
exportJAVA_HOME=/user/local/jdk1.6.0_27
exportJRE_HOME=/user/local/jdk1.6.0_27/jre
exportANT_HOME=/user/local/apache-ant-1.8.2
exportCLASSPATH=.:
$JAVA_HOME/lib:
$JRE_HOME/lib:
$CLASSPATH
2.安装Hadoop-1.0.3
2.1.下载hadoop文件,地址为:
http:
//hadoop.apache.org/coases.html,下载完成后解压hadoop-1.0.3.tar.gz
sudotar-xzfhadoop-1.0.3.tar.gz
2.2.配置Hadoop环境变量
exportHADOOP_INSTALL=/user/local/hadoop-1.0.3
exportPATH=$PATH:
$HADOOP_INSTALL/bin
激活profile文件:
[root@localhostetc]#chmod+xprofile
[root@localhostetc]#sourceprofile
[root@localhostetc]#hadoopversion
2.3.查看hadoop版本
[root@localhost~]#hadoopversion
Hadoop1.0.3
Subversionhttps:
//svn.apache.org/repos/asf/hadoop/common/branches/branch-1.0-r1335192
CompiledbyhortonfoonTueMay820:
31:
25UTC2012
Fromsourcewithchecksume6b0c1e23dcf76907c5fecb4b832f3be
输入hadoopversion命令后输入下图,则安装hadoop成功
2.4.修改配置文件
a)解压hadoop-1.0.3/hadoop-core-1.0.3.jar
b)去解压后的hadoop-core-1.0.3文件夹下,复制文件core-default.xml,hdfs-default.xml,mapred-default.xml三个文件到hadoop-1.0.3/conf/下,删除hadoop-1.0.3/conf/文件夹下的core-site.xml,hdfs-site.xml,mapred-site.xml,将复制过来的三个文件的文件名中的default修改为site
c)在hadoop-1.0.3文件夹同级创建文件夹hadoop,打开core-site.xml文件,修改属性节点下的name节点为hadoop.tmp.dir对应的value节点,修改为/user/local/${user.name}/hadoop/hadoop-${user.name},这样hadoop生成的文件会放入这个文件夹下.修改name节点为fs.default.name对应的value节点,修改为hdfs:
//localhost:
9000/
打开mapred-site.xml文件,修改property节点下name为mapred.job.tracker对应的的value,改为:
localhost:
9001
3.安装ssh
1.执行命令安装ssh:
sudoapt-getinstallssh
2.基于空口令创建一个新SSH密钥,以启用无密码登陆
a)ssh-keygen-trsa-P''-f~/.ssh/id_rsa
执行结果:
b)cat~/.ssh/id_rsa.pub>>~/.ssh/authorized_keys
3.测试
sshlocalhost
测试结果:
输入yes
再次输入sshlocalhost:
成功之后,就不需要密钥
4.格式化HDFS文件系统
输入指令:
hadoopnamenode–format
[root@localhost~]#hadoopnamenode–format
13/07/1714:
26:
41INFOnamenode.NameNode:
STARTUP_MSG:
STARTUP_MSG:
StartingNameNode
STARTUP_MSG:
host=localhost.localdomain/127.0.0.1
STARTUP_MSG:
args=[–format]
STARTUP_MSG:
version=1.0.3
STARTUP_MSG:
build=https:
//svn.apache.org/repos/asf/hadoop/common/branches/branch-1.0-r1335192;compiledby'hortonfo'onTueMay820:
31:
25UTC2012
Usage:
javaNameNode[-format]|[-upgrade]|[-rollback]|[-finalize]|[-importCheckpoint]
13/07/1714:
26:
41INFOnamenode.NameNode:
SHUTDOWN_MSG:
SHUTDOWN_MSG:
ShuttingdownNameNodeatlocalhost.localdomain/127.0.0.1
5.启动和终止守护进程
启动和终止HDFS和MapReduce守护进程,键入如下指令
启动start-all.sh(start-dfs.sh,start-mapred.sh)
出错了,JAVA_HOMEisnotset
需要修改文件,打开hadoop-1.0.3/conf/hadoop-env.sh
将红线以内部分注释解开,修改为本机JAVA_HOME
再次执行启动命令start-all.sh
停止stop-all.sh(stop-dfs.sh,stop-mapred.sh)
到此,hadoop就已经安装完成了
6.Hadoop文件系统
6.1.查看hadoop所有块文件
执行命令:
hadoopfsck/-files–blocks
执行结果:
此结果显示,hadoop文件系统中,还没有文件可以显示
本机出错,出错原因:
datanode没有启动,具体见evernote笔记。
[root@localhost~]#hadoopfsck/-files–blocks
13/07/1714:
44:
15ERRORsecurity.UserGroupInformation:
PriviledgedActionExceptionas:
rootcause:
java.ConnectException:
Connectionrefused
Exceptioninthread"main"java.ConnectException:
Connectionrefused
atjava.PlainSocketImpl.socketConnect(NativeMethod)
atjava.PlainSocketImpl.doConnect(PlainSocketImpl.java:
351)
atjava.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:
211)
atjava.PlainSocketImpl.connect(PlainSocketImpl.java:
200)
atjava.SocksSocketImpl.connect(SocksSocketImpl.java:
366)
atjava.Socket.connect(Socket.java:
529)
atjava.Socket.connect(Socket.java:
478)
atsunworkClient.doConnect(NetworkClient.java:
163)
atsun.http.HttpClient.openServer(HttpClient.java:
388)
atsun.http.HttpClient.openServer(HttpClient.java:
523)
atsun.http.HttpClient.
227)
atsun.http.HttpClient.New(HttpClient.java:
300)
atsun.http.HttpClient.New(HttpClient.java:
317)
atsun.protocol.http.HttpURLConnection.getNewHttpClient(HttpURLConnection.java:
970)
atsun.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:
911)
atsun.protocol.http.HttpURLConnection.connect(HttpURLConnection.java:
836)
atsun.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:
1172)
atorg.apache.hadoop.hdfs.tools.DFSck$1.run(DFSck.java:
141)
atorg.apache.hadoop.hdfs.tools.DFSck$1.run(DFSck.java:
110)
atjava.security.AccessController.doPrivileged(NativeMethod)
atjavax.security.auth.Subject.doAs(Subject.java:
396)
atorg.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:
1121)
atorg.apache.hadoop.hdfs.tools.DFSck.run(DFSck.java:
110)
atorg.apache.hadoop.util.ToolRunner.run(ToolRunner.java:
65)
atorg.apache.hadoop.util.ToolRunner.run(ToolRunner.java:
79)
atorg.apache.hadoop.hdfs.tools
- 配套讲稿:
如PPT文件的首页显示word图标,表示该PPT已包含配套word讲稿。双击word图标可打开word文档。
- 特殊限制:
部分文档作品中含有的国旗、国徽等图片,仅作为作品整体效果示例展示,禁止商用。设计者仅对作品中独创性部分享有著作权。
- 关 键 词:
- hadoop zookeeper +hive hbase安装学习12页文档资料 hive hbase 安装 学习 12 文档 资料
![提示](https://static.bdocx.com/images/bang_tan.gif)