Running tests

Added by Remko over 5 years ago

I love the way the CMake build does everything out of source. That's definitely a good thing.

What I don't like though is that the whole test suite now first needs to be copied to the build directory for the tests to run.
That, I think, is a waste of effort and disk space.

I see totally that at present there is no alternative. In fact, with the option of running several scripts simultaneously, it is not safe either. Two scripts in the same directory could potentially produce a temporary file with the same name, and more likely they both might like to create or update a gmt.conf file, where one could clobber the other.

I suggest, after the whole transition is done, we rethink this. We should probably use the isolation method in all scripts (gmt_init/remove_tmpdir) and have the PS files coming out be placed in the build directory.

That requires a bit of rewriting all the scripts, and make them maybe a bit less readable for the average user. Maybe we can find a mechanism in which the bulk of the script is enclosed between some symbols so that outside we can manipulate source and destination directory and the gmt_init/remove_tmpdir stuff.

Thoughts?


Replies (2)

RE: Running tests - Added by Florian over 5 years ago

I love the way the CMake build does everything out of source. That's definitely a good thing.

What I don't like though is that the whole test suite now first needs to be copied to the build directory for the tests to run.
That, I think, is a waste of effort and disk space.

I totally agree with you and I've been thinking about that since a long time. The problem is less the time of copying the files but more the extra space needed. CMake is quite smart and does copy a file only once or if a file in src is changed.

I see totally that at present there is no alternative. In fact, with the option of running several scripts simultaneously, it is not safe either. Two scripts in the same directory could potentially produce a temporary file with the same name, and more likely they both might like to create or update a gmt.conf file, where one could clobber the other.

Exactly, multiple GMT processes usually cannot coexist in the same working directory because they would mangle the common .gmtcommands file. Right now it is only possible to run GMT in parallel in the tests and examples subdirs. Integrity of the tmp files is ensured by a lockfile. I have been thinking about appending the PPID (parent process ID) or a user specified tag to the tmp files like .gmtcommands to avoid this. E.g., .gmtcommands.51715, where 51715 would be the PID of the executing process, usually the shell. This is especially useful for people working in common directories on shared network drives. But this will clutter the drive with lots of tmp files that the user would then have to remove.

I suggest, after the whole transition is done, we rethink this. We should probably use the isolation method in all scripts (gmt_init/remove_tmpdir) and have the PS files coming out be placed in the build directory.

Sure, isolation mode can easily be implemented. Although it needs some work: all scripts would have to be updated to use absolute paths to resources in ${GMT_SOURCE_DIR}/... and output files in ${GMT_BINARY_DIR}/... Then you really could run as many GMT processes as you machine can handle.

That requires a bit of rewriting all the scripts, and make them maybe a bit less readable for the average user. Maybe we can find a mechanism in which the bulk of the script is enclosed between some symbols so that outside we can manipulate source and destination directory and the gmt_init/remove_tmpdir stuff.

You still have to access the resources by absolute path. So this means rewriting the scripts. However, you can rely on properly set up environment variables, e.g.,

ps=front.ps
psxy -R-2/12/-2/22 -JM1i -O -K -W1p -Gred -Sf0.4i/0.1i+b t.txt -X0.25i -Y0.4i >> $ps
would become
ps=${OUTPUT_DIR}front.ps
psxy -R-2/12/-2/22 -JM1i -O -K -W1p -Gred -Sf0.4i/0.1i+b ${RESOURCE_DIR}t.txt -X0.25i -Y0.4i >> $ps
If OUTPUT_DIR and RESOURCE_DIR are variables that CMake sets prior to executing the test then from the user's view nothing has changed.

RE: Running tests - Added by Remko over 5 years ago

OK. That's a plan. I'll put it in as an "issue" and assign it to myself.

(1-2/2)