GMT Modern mode, 2.0

Added by Paul 6 months ago

Given our experimentation with the earlier demo modern mode and learning about its benefits and remaining shortcomings, here is an updated proposal 2.0:

Modern mode initiation

  1. A modern gmt session starts with the command gmt begin. This step accomplishes several tasks:
    1. Obtains the parent process ID (PPID).
    2. Creates a subdirectory gmt5.PPID in the current temp dir (OS-dependent) where all hidden GMT files are written.
    3. Writes a clean gmt.conf to this directory; no gmt.history exists at this point.
    4. Enables modern mode; this is the only way to enable this mode from the command line.
    5. PostScript plots are written to file gmt.ps0 in /tmp/gmt5.PPID.
    6. All interactions with gmt.conf and gmt.history take place in this unique directory.
  2. A modern session ends with gmt end, which removes /tmp/gmt5.PPID.
  3. All gmt calls bracketed by begin - end will know they are running in modern mode by the presence of /tmp/gmt5.PPID. There will be no GMT_RUNMODE gmt.conf setting to misuse.

The use of the session-specific temporary directory serves the same purpose as the classic isolation mode but transcends a particular OS [The implementation of isolation mode was strictly UNIX], is always enabled and does not require the user to do anything extra (beyond using gmt begin - end).

Rules regarding plot construction (-O -K)

These remain unrecognized options under modern mode. As before, the first plotter is identified by the lack of a gmt.ps0 file, while subsequent plotters will append to this existing file. The plot is finalized when gmt psconvert is called, at which point an output filename is required.

Rules regarding the region setting (-R)

Modern mode recognizes that the concept of "region" has always had two meanings in GMT:

  1. Specifying the map/plot domain. Apart from being a valid region, there are no further requirement on the values of the domain limits.
  2. Specifying the domain and organization of an equidistant grid in concert with conforming increments and node registration.

Yet, in classic mode there is no distinction between these meanings as far as GMT history goes. This will change in modern mode. For data processing modules that define and create grids the -R rules regarding the history are as follows:

  1. The first data-processing module must specify the grid region, its increment(s), and optionally node registration. These three settings will be stored in gmt.history's new -RG slot (for grid domain).
  2. Subsequent data processing will default to the history in -RG unless overridden on the command line. Any or all of the three components can be overridden on the command line and -RG will be updated accordingly.

Next, here are the -R rules for plot-producing modules:

  1. A new plot will require a complete -R specification on the command line. This setting will be stored in gmt.history's new -RP slot (for plot region).
  2. Subsequent plot overlays will default to the history in -RP unless overridden on the command line (which also resets the history of -RP).
  3. Two new flavors of -R will be available to plot modules:
    1. The -Re (for exact) option will determine a tight region from the input data (tables or grids).
    2. The -Ra (for auto) will take the region from -Re and round it outwards using a suitable increment.

We have had something like an implicit -Re in classic mode, for instance when a plotting module takes its plot region from the grid domain. Now this is explicit and extends to all data sets. The -Ra mode is new and will allow generation of maps with reasonable boundaries much easier.

Benefits

  1. The use of the gmt5.PPID temporary directory extends the benefit of isolation mode to all of GMT, across all platforms.
  2. Plot modules gain new capabilities for selecting the plot region with -Re and -Ra.
  3. The gmt begin and end pair encapsulates a single GMT workflow [http://gmt.soest.hawaii.edu/projects/gmt/wiki/GWF] and greatly reduces the potential of rouge gmt history values. It provides needed structure for new (and old) GMT users.
  4. The region history of grid generation can inform about plot regions but not the other way around.
  5. There can be no confusion between classic and modern mode since users cannot inadvertently switch to modern by messing with gmt.conf.

Replies (11)

RE: GMT Modern mode, 2.0 - Added by Remko 6 months ago

Suggestions for modification and questions

  1. I would suggest /tmp/gmt.PPID instead of /tmp/PPID ... this makes it easier to find those temporary directories in case they are left behind, and avoids clobbering by other programs that may have created /tmp/PID in the parent script.
  2. What would the temporary directory be named when calling the API from a program? In that case it should be /tmp/gmt.PID (the PID of the program), not?
  3. We could potentially create PS files named gmt_000.ps, gmt_001.ps, etc., for the different layers. This makes it faster to change the header (only modification to gmt_000.ps). Of course, all gmt_???.ps need to be piped to ghostscript, which may not be trivial to do in an OS-independent way.

RE: GMT Modern mode, 2.0 - Added by Paul 6 months ago

  1. Excellent, agreed, and updated.
  2. Probably. Getting the PPID can be tricky in that case (think Matlab) since there may be many threads and PPID can change. So yes, likely PID. We will lean more when trying that.

RE: GMT Modern mode, 2.0 - Added by Paul 6 months ago

I swear there was no 3 earlier!
3. I fail to see the utility of this. Remember, modern mode adds restrictions to simplify life and we are not allowing people to look under the hood to pull out a particular layer. That being said, we did discuss at one point the ability to save a partially baked PostScript cake so that a plotting sequence could jump-start to that point without having to rerun lots of GMT commands (i.e., a case where many plots share the same tedious background map, made once). That capability I think we can accommodate with a special psconvert option to "get" and "put" the half-baked cake in/out of the temp directory.

RE: GMT Modern mode, 2.0 - Added by Remko 6 months ago

There was no no 3 before, indeed. I added it later.
Fair enough, so forget about it (I did a strike-through on that one).

RE: GMT Modern mode, 2.0 - Added by Remko 6 months ago

But then, one more thing:

4. Instead of /tmp/, it would be better to use $TMPDIR/, which is defined on some OSes differently from /tmp. If not defined, then default to /tmp/

RE: GMT Modern mode, 2.0 - Added by Paul 6 months ago

Yes, we actually use API→tmpdir that is set depending on OS, etc. I was just using /tmp as a shorthand.

RE: GMT Modern mode, 2.0 - Added by Leonardo 5 months ago

Hi Paul, this is great!
So the Python side will also need to start with a "gmt.begin()" and "gmt.end()" right?
We could get rid of "gmt.show()" by patching our "gmt.end()" to pull up a png and return it to the notebook.
Just to be clear, there is still need for "gmt psconvert" before "end", correct?

RE: GMT Modern mode, 2.0 - Added by Leonardo 5 months ago

Actually, we could even to away with "begin" and "end" altogether in Python.
That could be replaced by a context manager (https://docs.python.org/3/reference/datamodel.html#context-managers):

with gmt.figure():
gmt.pscoast(...)
...
gmt.psconvert(...)

The "with" statement will handle calling "begin" and "end" when entering and exiting.

RE: GMT Modern mode, 2.0 - Added by Paul 5 months ago

Yes, but remember not everything may be resulting in a figure. Some people may be making a grid only, so we should not think of begin/end to necessarily be graphics related. They just encapsulate a specific workflow I think.

RE: GMT Modern mode, 2.0 - Added by Leonardo 5 months ago

Right, for figure generation we could use "gmt.figure()" (which need to produce a figure for the notebook in the end) but if not, then "with gmt.begin()" could be used (without needing a gmt.end() later).

RE: GMT Modern mode, 2.0 - Added by Paul 5 months ago

That seems reasonable.

(1-11/11)