1,现象:flume再往hadoop HDFS写文件时flume.log报错 be replicated to 0 nodes, instead of 1
2012-12-18 13:47:24,673 WARN hdfs.BucketWriter: Caught IOException writing to HDFSWriter (java.io.IOException: File /logdata/20121218/bj4aweb04/8001_4A_ACA/8001_4A_ACA.1355799411582.tmp could only be replicated to 0 nodes, instead of 1
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:1558)
at org.apache.hadoop.hdfs.server.namenode.NameNode.addBlock(NameNode.java:696)
2,查看相关进程状态 datanode没有正常启动
[hadoop@dtydb6 hadoop]$ jps
7427 Jps
7253 TaskTracker
6576 NameNode
6925 SecondaryNameNode
7079 JobTracker
3,查看datanode的日志
Incompatible namespaceIDs
java.io.IOException: Incompatible namespaceIDs in /hadoop/logdata: namenode namespaceID = 13513664; datanode namespaceID = 525507667
4,根据报错信息定位到namespaceIDs版本不一致
根据参考文档的解决方案,原因是多次格式化hadoop导致版本信息不一致,修改为一致状态即可解决问题
解决的办法很简单,两个方案
1). 所有的datanode删掉,重新建(很麻烦,但看你了)
2) 登上datanode,把位于{dfs.data.dir}/current/VERSION中的namespaceID改为最新的版本即可
[hadoop@dtydb6 current]$ cat VERSION
#Fri Dec 14 09:37:22 CST 2012
namespaceID=525507667
storageID=DS-120876865-10.4.124.236-50010-1354772633249
cTime=0
storageType=DATA_NODE
layoutVersion=-32
5,重新启动hadoop,datanode已经成功启动
[hadoop@dtydb6 current]$ jps
8770 JobTracker
8436 DataNode
8266 NameNode
8614 SecondaryNameNode
9107 Jps
参考资料:
http://hi.baidu.com/windhl/item/222ce41b8933f8623e87ce5a
分享到:
相关推荐
CDH集群大数据hadoop报错解决办法及思路整理,主要解决大数据在运行过程中所遇到的问题,相关解决办法都是实践验证过。
mac 版hadoop3.2.4或其他版本 Unable to load native-hadoop library 缺失文件
ERROR org.apache.hadoop.mapred.TaskTracker: Can not start task tracker because java.io.IOException: Failed to set permissions of path: \tmp\hadoop-admin \mapred\local\ttprivate to 0700 at org.apache...
Logstash6整合Hadoop-报错与解决方案.docx
windows下Hadoop报错null\bin\winutils.exe-附件资源
包含winutils 注意版本 已经成功运行 之后配置环境变量 解决 Hadoop报错:Failed to locate the winutils binary in the hadoop binary path
eclipse远程调试hadoop时 报出eclipse Hadoop Failed to set permissions of path错误 修改hadoop core包中FileUtil java文件 里面有checkReturnValue方法 将代码throw new IOException "Failed to set ...
Understanding the internals of Hadoop and how Hadoop can be extended to suit your needs How to use HBase, Hive, Pig, Mahout, and Nutch to get things done easily and efficiently How to use MapReduce to...
Hands-on recipes to configure a Hadoop cluster from bare metal hardware nodes Practical and in depth explanation of cluster management commands Easy-to-understand recipes for securing and monitoring a...
org.apache.hadoop.io.nativeio.NativeIO.java解决办法 将org放入项目的工程目录下 本地运行MR必备的源码包,本地运行MR必备的源码包,本地运行MR必备的源码包,
Failed to set permissions of path: \tmp\hadoop-Administrator,的解决方法,更换hadoop-core-1.0.2-modified.jar包
本地开发Spark/Hadoop报错“ERROR Shell: Failed to locate the winutils binary in the hadoop binary path java.io.IOException: Could not locate executable null\bin\winutils.exe in the Hadoop binaries.” ...
It morphed the Hadoop compute layer to be a common resource-management platform that can host a wide variety of applications. Many organizations leverage YARN in building their applications on top of...
By the end of the book, you will be confident to begin working with Hadoop straightaway and implement the knowledge gained in all your real-world scenarios. Table of Contents Chapter 1: Introduction ...
Field.Guide.to.Hadoop.2015.3.pdf
Moving Hadoop to The Cloud 英文epub 本资源转载自网络,如有侵权,请联系上传者或csdn删除 本资源转载自网络,如有侵权,请联系上传者或csdn删除
Along with Hadoop 2.x and illustrates how it can be used to extend the capabilities of Hadoop. When you nish this course, you will be able to tackle the real-world scenarios and become a big data ...
Field Guide to Hadoop An Introduction to Hadoop, Its Ecosystem, and Aligned Technologies 英文无水印原版pdf pdf所有页面使用FoxitReader、PDF-XChangeViewer、SumatraPDF和Firefox测试都可以打开 本资源...