每次我将路径作为新路径（someDir）;
public void writeToHDFS（Path path，long uniqueId，String data）{ FileSystem fs = path.getFileSystem（conf）; SequenceFile.Writer inputWriter = new SequenceFile.Writer（fs，conf， path，LongWritable.class，MyWritable.class）; inputWriter.append（new LongWritable（uniqueId ++），new MyWritable（data））; inputWriter.close（）;解决方案
目前没有办法通过API附加到现有的SequenceFile。当创建新的 SequenceFile.Writer 对象时，它不会追加到 Path 中的现有文件，而是覆盖它。查看我的早期问题。
正如Thomas指出的那样，如果您保留相同的 SequenceFile.Writer 对象，您将能够附加到该文件，直到您致电 close（）。
I am writing to hadoop file system. But everytime I append something, it overwrites the data instead of adding it to the existing data/file. The code which is doing this is provided below. This code is called again and again for different data. Is opening a new SequenceFile.Writer everytime a problem?
Each time I am getting the path as new Path("someDir");
public void writeToHDFS(Path path, long uniqueId, String data){ FileSystem fs = path.getFileSystem(conf); SequenceFile.Writer inputWriter = new SequenceFile.Writer(fs, conf, path, LongWritable.class, MyWritable.class); inputWriter.append(new LongWritable(uniqueId++), new MyWritable(data)); inputWriter.close(); }解决方案
There is currently no way to append to an existing SequenceFile through the API. When you make the new SequenceFile.Writer object, it will not append to an existing file at that Path, but instead overwrite it. See my earlier question.
As Thomas points out, if you keep the same SequenceFile.Writer object, you will be able to append to the file until you call close().
更多推荐
写入HDFS：文件被覆盖
发布评论