Stop almost x CPU-intensive jobs

I've obtained concerning 50 folders of information to procedure, and also I've obtained a ruby script that refines a folder's documents (which folder it refines is based upon a.yml arrangement documents). And also a computer system with 4 CPUs in it.

I would certainly such as to have the ability to start up the 50 procedures, yet have just 4 of them proactively going for at any time, and also have the various other 46 stopped. As soon as among the procedures coatings, I would certainly such as among the stopped procedures to come to be un-paused, till all 50 are finished. In this way, I can do

./super_script.rb > folder_1_log.txt
*edit config.yml*
./super_script.rb > folder_2_log.txt
*edit config.yml*

And also focus on something else till the handling's done.

Is it feasible to do this? Exist some terms wherefore I'm desiring that I can google?

(An additional choice would certainly be to make super_script with the ability of multi-threading - possibly I'm a scaredy-cat for not taking that strategy)

(The os is Ubuntu Linux, and also a lot of the CPU time isn't occupied by super_script. rb, yet by various other ruby programs it calls using system () )

2019-05-06 00:54:45
Source Share
Answers: 1

Here's a bash script that seems it does something near what you intend to do-- it launches a variety of procedures in parallel yet sees to it that no greater than n are ever before going for the very same time.

On the various other hand, if what you're doing is disk-bound, as opposed to CPU-bound (I'm asking due to the fact that you claim you've obtained "50 folders of information to procedure" ), after that you might in fact be far better off running every one of your procedures serially to stay clear of opinion for the disk in between the procedures.

2019-05-07 17:39:40