我的hadoop版本是2.7.3;mahout的版本是0.13
2017-11-13 14:50:23,290 INFO [org.apache.hadoop.mapreduce.Job] - Job job_1510544823024_0005 failed with state FAILED due to: Application application_1510544823024_0005 failed 2 times due to ApplicationMaster for attempt appattempt_1510544823024_0005_000002 timed out. Failing the application.
2017-11-13 14:50:23,312 INFO [org.apache.hadoop.mapreduce.Job] - Counters: 0
Exception in thread "main" java.io.FileNotFoundException: File does not exist: /class/output/temp/rec001/preparePreferenceMatrix/numUsers.bin
at org.apache.hadoop.hdfs.server.namenode.INodeFile.valueOf(INodeFile.java:71)
java.io.FileNotFoundException: File does not exist: 文件找不到,你修改下路径应该就可以了。
你好
我想知道这个文件/class/output/temp/rec001/preparePreferenceMatrix/numUsers.bin存在哪?
运行程序如下
package mymahout.ItemCF;
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.util.ToolRunner;
import org.apache.mahout.cf.taste.hadoop.item.RecommenderJob;
public class MyCFRecommder {
public static void main(String[] args) throws Exception{
// TODO Auto-generated method stub
System.out.println("程序开始运行");
Configuration conf =null;
conf= HadoopConUtil.getConf();
String[] args1 ={
"-i","hdfs://172.17.73.138:9000/class/input/user.csv",
"-o","hdfs://172.17.73.138:9000/class/output/rec001",
"-n","3","-b","false","-s","SIMILARITY_EUCLIDEAN_DISTANCE",
"--maxPrefsPerUser","7","--minPrefsPerUser","2",
"--maxPrefsInItemSimilarity","7",
"--outputPathForSimilarityMatrix","hdfs://172.17.73.138:9000/class/output/matrix/rec001",
"--tempDir","hdfs://172.17.73.138:9000/class/output/temp/rec001"};
ToolRunner.run(conf, new RecommenderJob(), args1);
System.out.println("运行成功");
}
}
package mymahout.ItemCF;
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.yarn.conf.YarnConfiguration;
public class HadoopConUtil {
private static Configuration conf=null;
private static final String YARN_RESOURCE="172.17.73.138:8032";
private static final String DEFAULT_FS="hdfs://172.17.73.138:9000";
public static Configuration getConf(){
if(conf==null){
conf = new YarnConfiguration();
conf.set("fs.defaultFS", DEFAULT_FS);
conf.set("mapreduce.framework.name", "yarn");
conf.set("yarn.resourcemanager.address", YARN_RESOURCE);
}
return conf;
}
}
输出路径应该是
hdfs://172.17.73.138:9000/class/output/rec001