I’ve learned about some options I didn’t know that allow for running lines parallel right within the R script. In that article, the author refers to how I was conceiving the workflow as “poor man’s parallel.” 🙂 So, I might combine the newer technique by running the models parallel within the script, and creating an array job submission of some kind for each data set.
Now, I have to plan for resources requested. Does each script run get 1 core, 1 node, etc.? I found a suggestion that if memory is an issue, setting the number of threads to 2 is a good idea. R-INLA runs with OpenMP and can eat up a lot of resources. Sometimes that means it converges, and sometimes that means it crashes.