我有相当多的文件需要转换为不同的格式。 转换是通过Java-JAR-File完成的,该文件将文件名作为参数。 我现在有一个Windows批处理文件,它使用for循环遍历所有文件(有一个文件包含需要转换的所有文件的列表)
for /F %%i in (all_files.txt) do call java -cp %Classpath% de.xyz.Convert -xml %%i .\xml现在,我想要这样做的机器有八个核心。 文件数量约为360.000,我希望它花费尽可能少的时间,所以我想尽可能多地使用核心。 我如何尽可能简单地使用多个内核? Windows会自己做吗?
I have a pretty big number of files that need to be converted to a different format. The converting is done via a Java-JAR-File that gets takes the filename as a parameter. I now have a Windows batchfile that uses a for loop to loop through all the files (there is a file that contains a list of all files that need to be converted)
for /F %%i in (all_files.txt) do call java -cp %Classpath% de.xyz.Convert -xml %%i .\xmlNow the machine I want to do this on has eight cores. The number of files is about 360.000 and I would like it to take as little time as possible, so I'd like to use as many cores as possible. How would I go about using multiple cores as easy as possible? Is Windows going to be doing that on its own?
最满意答案
好吧,因为我之前没有真正做过,所以我把它搞砸了。 它不是很好,我使用的lib是我创建的一个罐子,在2分钟后崩溃。希望你能够根据你的需要对它进行逆向工程。
package test; import java.io.IOException; import java.util.concurrent.BlockingQueue; import java.util.concurrent.ExecutorService; import java.util.concurrent.LinkedBlockingQueue; import java.util.concurrent.ThreadPoolExecutor; import java.util.concurrent.TimeUnit; public class Test { public static void main(String[] args) throws InterruptedException, IOException { BlockingQueue<Runnable> runnableQueue = new LinkedBlockingQueue<>(); ExecutorService executorServ = new ThreadPoolExecutor(8, 8, 1, TimeUnit.MINUTES, runnableQueue); runnableQueue.add(new RunCrash("Example")); // Add one for each file... executorServ.shutdown(); while(!executorServ.isTerminated()) { // running } } } class RunCrash implements Runnable { private String fileName; RunCrash(String fileName) { this.fileName = fileName; } @Override public void run() { System.out.println(fileName); try { crash.CrashMe.main(new String[]{fileName}); } catch (InterruptedException e) { // TODO Auto-generated catch block e.printStackTrace(); } } }哦,你可以让主线程在其他完成之前死掉,我相信JVM会保留执行程序和相关的队列。 :)
Ok, because I hadn't actually done it before, I knocked this up. It's not great, and the lib I used was a jar I created to crash after 2 mins.. Hopefully you'll be able to reverse engineer this for your needs.
package test; import java.io.IOException; import java.util.concurrent.BlockingQueue; import java.util.concurrent.ExecutorService; import java.util.concurrent.LinkedBlockingQueue; import java.util.concurrent.ThreadPoolExecutor; import java.util.concurrent.TimeUnit; public class Test { public static void main(String[] args) throws InterruptedException, IOException { BlockingQueue<Runnable> runnableQueue = new LinkedBlockingQueue<>(); ExecutorService executorServ = new ThreadPoolExecutor(8, 8, 1, TimeUnit.MINUTES, runnableQueue); runnableQueue.add(new RunCrash("Example")); // Add one for each file... executorServ.shutdown(); while(!executorServ.isTerminated()) { // running } } } class RunCrash implements Runnable { private String fileName; RunCrash(String fileName) { this.fileName = fileName; } @Override public void run() { System.out.println(fileName); try { crash.CrashMe.main(new String[]{fileName}); } catch (InterruptedException e) { // TODO Auto-generated catch block e.printStackTrace(); } } }Oh, you can let the main thread die before the others finish, I believe the JVM will keep the executor and associated queue. :)
更多推荐
发布评论