并行处理Windows批处理文件(Parallel processing Windows batch file)

我有相当多的文件需要转换为不同的格式。转换是通过Java-JAR-File完成的，该文件将文件名作为参数。我现在有一个Windows批处理文件，它使用for循环遍历所有文件（有一个文件包含需要转换的所有文件的列表）

for /F %%i in (all_files.txt) do call java -cp %Classpath% de.xyz.Convert -xml %%i .\xml

现在，我想要这样做的机器有八个核心。文件数量约为360.000，我希望它花费尽可能少的时间，所以我想尽可能多地使用核心。我如何尽可能简单地使用多个内核？ Windows会自己做吗？

I have a pretty big number of files that need to be converted to a different format. The converting is done via a Java-JAR-File that gets takes the filename as a parameter. I now have a Windows batchfile that uses a for loop to loop through all the files (there is a file that contains a list of all files that need to be converted)

for /F %%i in (all_files.txt) do call java -cp %Classpath% de.xyz.Convert -xml %%i .\xml

Now the machine I want to do this on has eight cores. The number of files is about 360.000 and I would like it to take as little time as possible, so I'd like to use as many cores as possible. How would I go about using multiple cores as easy as possible? Is Windows going to be doing that on its own?

最满意答案

好吧，因为我之前没有真正做过，所以我把它搞砸了。它不是很好，我使用的lib是我创建的一个罐子，在2分钟后崩溃。希望你能够根据你的需要对它进行逆向工程。

package test; import java.io.IOException; import java.util.concurrent.BlockingQueue; import java.util.concurrent.ExecutorService; import java.util.concurrent.LinkedBlockingQueue; import java.util.concurrent.ThreadPoolExecutor; import java.util.concurrent.TimeUnit; public class Test { public static void main(String[] args) throws InterruptedException, IOException { BlockingQueue<Runnable> runnableQueue = new LinkedBlockingQueue<>(); ExecutorService executorServ = new ThreadPoolExecutor(8, 8, 1, TimeUnit.MINUTES, runnableQueue); runnableQueue.add(new RunCrash("Example")); // Add one for each file... executorServ.shutdown(); while(!executorServ.isTerminated()) { // running } } } class RunCrash implements Runnable { private String fileName; RunCrash(String fileName) { this.fileName = fileName; } @Override public void run() { System.out.println(fileName); try { crash.CrashMe.main(new String[]{fileName}); } catch (InterruptedException e) { // TODO Auto-generated catch block e.printStackTrace(); } } }

哦，你可以让主线程在其他完成之前死掉，我相信JVM会保留执行程序和相关的队列。 :)

Ok, because I hadn't actually done it before, I knocked this up. It's not great, and the lib I used was a jar I created to crash after 2 mins.. Hopefully you'll be able to reverse engineer this for your needs.

package test; import java.io.IOException; import java.util.concurrent.BlockingQueue; import java.util.concurrent.ExecutorService; import java.util.concurrent.LinkedBlockingQueue; import java.util.concurrent.ThreadPoolExecutor; import java.util.concurrent.TimeUnit; public class Test { public static void main(String[] args) throws InterruptedException, IOException { BlockingQueue<Runnable> runnableQueue = new LinkedBlockingQueue<>(); ExecutorService executorServ = new ThreadPoolExecutor(8, 8, 1, TimeUnit.MINUTES, runnableQueue); runnableQueue.add(new RunCrash("Example")); // Add one for each file... executorServ.shutdown(); while(!executorServ.isTerminated()) { // running } } } class RunCrash implements Runnable { private String fileName; RunCrash(String fileName) { this.fileName = fileName; } @Override public void run() { System.out.println(fileName); try { crash.CrashMe.main(new String[]{fileName}); } catch (InterruptedException e) { // TODO Auto-generated catch block e.printStackTrace(); } } }

Oh, you can let the main thread die before the others finish, I believe the JVM will keep the executor and associated queue. :)

更多推荐