Running parallel jobs with files outside of AFS

The $TMPDIR  is set when the job is executed points to a temporary directory on the counting machines where job files can be written. Accordingly, in the script file, there should be instructions for copying the desired files from directories that are not part of AFS and which are accessible directly only on interactive machines lxpub01-lxpub05, and on counting machines – only available in the absolute path.
To work with / scrc /, you must have a script file in your home directory to run the task using MPI, and / scrc / u / username – the files (program texts or executable and / or input files).
Example of a script file for running parallel tasks with files outside of AFS:

#PBS -l walltime=00:20:00
#PBS -l cput=00:05:00
#PBS -l nodes=4
# We move to the temporary directory on the computer:
cd $TMPDIR
if test $? -ne 0 ; then
echo “ERROR 1: can not cd $TMPDIR”
ret=1
fi
# Copying executable and input files:
# (Replace directory / scrc / u / username / and filenames – with your own)
scp -p2 lxpub01:/scrc/u/tsap/test .
scp -p2 lxpub01:/scrc/u/tsap/in.dat .
# copy files to only one machine,
# And below to the rest:

for h in `cat $PBS_NODEFILE | sort | uniq` ; do
test X”`hostname -f`” = X”$h” && continue
ssh -1x $h “mkdir -p $TMPDIR”
scp -p2 test $h:$TMPDIR
scp -p2 in.dat $h:$TMPDIR
done

# Check whether everything turned out.
# (File names are also replaced with their own)
if test ! -e $TMPDIR/test ; then
echo “ERROR 2: no $TMPDIR/test”

ret=1
fi
if test ! -e $TMPDIR/in.dat ; then
echo “ERROR 3: no $TMPDIR/in.dat”
ret=1

fi
test $ret -ne 0 && exit 1
# Run for execution
mpiexec ./test
# Copying the output file out.dat
# (Also replace the filename and address with / scrc / with your own)
if [ -f out.dat ]; then
scp -p2 out.dat lxpub01:/scrc/u/tsap/.
fi