multithreading - What is the most efficient way to write to multiple files on disk simultaneously in Java? -


i have system has n threads (1 < n < 10). each thread generates data, , data each thread needs written it's own file (so, n threads, n files). files huge , written hard drive (not ssd), , in cases raid array.

i have chosen strategy of writing data each thread it's own buffer, , having single thread read buffer contents , write files (each buffer have it's own file). reasoning writing disk multiple threads less efficient (more disk head movement) , result in poor performance.

my questions:

q1) assumption correct, or should write directly each thread?

q2) if assumptions correct, there libraries this?

thanks

i've implemented @ school in java teaching purposes , i've come conclusion better write in parallel on multiple threads. however, depends on number of threads, synchronization complexity, , size of buffer needs written.

and more: cannot sure cpu run of threads in parallel, if create many threads, nothing. used threadpoolexecutor anyway.

you should implement both methods , simulate tests , log times , you'll come conclusion. may more efficient 1 way or other until reach limit. then, may become more efficient other way.

this interesting problem. i'll test big files when home.


Comments

Popular posts from this blog

javascript - Using jquery append to add option values into a select element not working -

Android soft keyboard reverts to default keyboard on orientation change -

jquery - javascript onscroll fade same class but with different div -