通过参数在Hadoop中初始化公共静态变量(Initialize public static variable in Hadoop through arguments)

我在Hadoop中更改公共静态变量时遇到问题。我试图从命令行将一些值作为参数传递给jar文件。

这是我的代码：

public class MyClass { public static long myvariable1 = 100; public static class Map extends Mapper<Object, Text, Text, Text> { public static long myvariabl2 = 200; public void map(Object key, Text value, Context context) throws IOException, InterruptedException { } } public static class Reduce extends Reducer<Text, Text, Text, Text> { public void reduce(Text key, Iterable<Text> values, Context context) throws IOException, InterruptedException { } } public static void main(String[] args) throws Exception { col_no = Long.parseLong(args[0]); Map.myvariable1 = Long.parseLong(args[1]); Map.myvariable2 = Long.parseLong(args[1]); other stuff here } }

但它不起作用，myvariable1和myvaribale2总是有100和200.我使用Hadoop 0.20.203和Ubuntu 10.04

I have a problem with changing public static variables in Hadoop. I am trying to pass some values as arguments to the jar file from command line.

here is my code:

public class MyClass { public static long myvariable1 = 100; public static class Map extends Mapper<Object, Text, Text, Text> { public static long myvariabl2 = 200; public void map(Object key, Text value, Context context) throws IOException, InterruptedException { } } public static class Reduce extends Reducer<Text, Text, Text, Text> { public void reduce(Text key, Iterable<Text> values, Context context) throws IOException, InterruptedException { } } public static void main(String[] args) throws Exception { col_no = Long.parseLong(args[0]); Map.myvariable1 = Long.parseLong(args[1]); Map.myvariable2 = Long.parseLong(args[1]); other stuff here } }

But it is not working, myvariable1 & myvaribale2 always have 100 & 200. I use Hadoop 0.20.203 with Ubuntu 10.04

最满意答案

为了获得相同的行为，您可以做的是将变量存储在用于启动作业的配置中。

public static class Map extends Mapper<Object, Text, Text, Text> { public void map(Object key, Text value, Context context) throws IOException, InterruptedException { Configuration conf = context.getConfiguration(); String var2String = conf.get("myvariable2"); long myvariable2 = Long.parseLong(var2String); //etc. } } public static void main(String[] args) throws Exception { col_no = Long.parseLong(args[0]); String myvariable1 = args[1]; String myvariable2 = args[1]; // add values to configuration Configuration conf = new Configuration(); conf.set("myvariable1", myvariable1); conf.set("myvariable2", myvariable2); //other stuff here }

What you can do to get the same behavior is to store your variables in the Configuration you use to launch the job.

public static class Map extends Mapper<Object, Text, Text, Text> { public void map(Object key, Text value, Context context) throws IOException, InterruptedException { Configuration conf = context.getConfiguration(); String var2String = conf.get("myvariable2"); long myvariable2 = Long.parseLong(var2String); //etc. } } public static void main(String[] args) throws Exception { col_no = Long.parseLong(args[0]); String myvariable1 = args[1]; String myvariable2 = args[1]; // add values to configuration Configuration conf = new Configuration(); conf.set("myvariable1", myvariable1); conf.set("myvariable2", myvariable2); //other stuff here }

更多推荐