Proposal for a gmt movie module

Added by Paul 9 months ago

Motivation

We want a super-friendly and simple way to create GMT animations, and it should work similarly across command-line and API extensions to Matlab, Julia, and Python. This will be implemented as a new module "gmt movie".

Goals

It needs to hide ALL the usual book-keeping and loops from the user. It should be flexible enough to handle optional pre-calculations (e.g., perform initial calculations that produce grids or tables or other items needed by the main script and maybe build a background plot) and optional post-calculation (i.e., produce a foreground overlay). These optional plot layers should be regular PostScript plots so the user can examine them as any other plot. gmt movie will need to gather parameters from the user (to define movie size and length) as well as a main script, and the optional pre- and post-scripts. It will also need to run the movie loop internally and take advantage of OpenMP. Finally, it will build the movie or animated gif and clean up. All bash scripts involved must be using gmt modern mode.

Synopsis

This document focuses on how the gmt movie module works in a command line setting. Implementations in Python, etc. will differ but it is likely the use of bash scripts will translate into functions in such environments.

Proposed syntax:

gmt movie <mainscript> -F<format> -N<prefix> -T<times> -W<dims>
    [-A<rate>] [-E] [-G<color>] [-Q[<frame>]] [-Sb<background>] [-Sf<foreground>] [-V]

Required parameters:

    -F Video format, either mp4 (mp4 video), gif (animated gif), or none [just pngs]
    -N The prefix for all movie-related products and directories
    -T Number of frames in the animation OR file with parameters for each frame
       If file then column values will be accessible in the main script (see below).
       Note: The file may not exist until after pre-script has run!
    -W Sets the frame dimensions from known sizes (HD, 360p) or width x hight x dpi

<mainscript> is a bash shell that makes the main plot for a single frame

Optional parameters:

    -A frame rate of movie [24 per second]
    -E Erase directory with frame images after completing movie
    -G Canvas color [white]
    -Q No movie, just plot png frame 0 (or given frame) [make movie]
       This is useful for debugging and testing if you have the right layout
    -Sb Give name of optional background-producing script [none]
    -Sf Give name of optional foreground-producing script [none]
    -V report progress and final ffmpeg/gm command

An example [This is anim02 redux]

  1. Pre-script pre.sh: (Making list of angles/directions and a cpt to use in loop; no background plot is generated)
    gmt begin
        gmt math -T0/360/10 T 180 ADD = angles.txt
        gmt makecpt -Crainbow -T500/4500 > main.cpt
    gmt end
    
  2. Main-script main.sh: (just making a single frame plot using preset frame variables)
    gmt begin
        width=`gmt math -Q ${GMT_MOVIE_WIDTH} 0.5i SUB =`
        gmt grdimage @tut_relief.nc -I+a${GMT_MOVIE_VAL1}+nt2 -JM${width} -Cmain.cpt -BWSne -B1 -Xc -Yc
        gmt psxy -Sc0.8i -Gwhite -Wthin <<< "256.25 35.6" 
        gmt psxy -Sv0.1i+e -Gred -Wthick <<< "256.25 35.6 ${GMT_MOVIE_VAL2} 0.37i" 
    gmt end
    
  3. Post-script: not used in this animation.

We then run the main command:

gmt movie main.sh -W3.5ix4.167ix72 -Fgif -Nanim02 -Tangles.txt -Sbpre.sh -A6

gmt movie will use its arguments to build the actual scripts that are run under the hood. The gmt movie scripts here are 10 lines in total (of which 4 are begin/end), whereas the anim02.sh in classic mode is 35 lines long and convoluted. Basically, gmt movie "inserts" all those missing lines that deals with the loop, frame increments, ripping to PNG etc., etc.

How it works

  1. The setup stage calculates canvas dimensions, number of frames, dpi and other job settings first. These are written to file movie_init.sh. The parameters written are these (there may be others as well):
        $GMT_MOVIE_WIDTH      # The width of your paper canvas [in plot units (c|i|p)]
        $GMT_MOVIE_HEIGHT     # The height of your paper canvas [in c|i|p]
        $GMT_MOVIE_DPU        # DPU of the frame plot
    

    To simplify cleanup we create a subdirectory called GMT_MOVIE_DIR and basically make that the current directory. We ensure that all products are moved to its
    parent directory then remove the build dir completely.
  2. The optional background script is used to build the prescript.sh: Start with "source movie_init.sh" then copy/paste the rest of the script. We then add a rm -f prescript.sh at the end gmt movie then runs prescript.sh via a system call. If a background plot is requested then the modern mode MUST start with "gmt begin background ps" so that a PS plot by that name is built in the current directory.
  3. The optional foreground script is used to build postscript.sh: Start with "source movie_init.sh" then copy/paste the rest of the script. We then add a rm -f postscript.sh at the end gmt movie then runs postscript.sh via a system call. The modern mode must start with "gmt begin foreground ps" so that a PS plot by that name is built in the current directory.
  4. gmt movie now produces one shell include file per frame. These files are called movie_frame_######.sh and contain all parameters specific to the corresponding frame. These are items like
        $GMT_MOVIE_FRAME             # Current frame number 0-(nframes-1)
        $GMT_MOVIE_NFRAMES           # Total number of frames
        $GMT_MOVIE_NAME              # Current frame name prefix
        $GMT_MOVIE_VAL1, $GMT_MOVIE_VAL2, ...     # Per-frame parameters from columns via -T (if used)
    
  5. gmt movie now builds the actual loop script from <mainscript>: Start with "source movie_init.sh" and "source movie_frame_$1.sh", then mkdir $GMT_MOVIE_NAME; cd $GMT_MOVIE_NAME then append the rest of <mainscript> to loopscript.sh, with this wrinkle: After the required "gmt begin" line we insert "gmt figure $GMT_MOVIE_NAME png E${GMT_MOVIE_PLOT_DPU}" then at the end we append "mv -f $GMT_MOVIE_NAME.png .." then append "cd .." then append "rm -f $GMT_MOVIE_NAME movie_frame_$1.sh" This ensures that the frame png will have the correct serial name and be of the correct dimensions and placed in user dir. it also makes sure that any temporary file created by mainscript (e.g., t.txt) will be secure in its own frame directory ($GMT_MOVIE_NAME).
  6. gmt movie now sets up a loop with system call to loopscript.sh with frame as arg. These are launched as long as there are more frames and there are available cores. We take advantage of all available cores. This way, many instances of loopscript.sh will run simultaneously and accept different frame numbers as argument in order to source the correct parameter file. These jobs will not interfere since their frame and hidden directory names are separate and unique.
  7. gmt movie module will detect if static background and/or foreground plots are produced. If that is the case then the gmt figure command in the main script will add suitable -M options for psconvert so that these layers can be included in the rasterization.
  8. gmt movie waits for all frames to complete. It is then time to produce the MP4 or animated GIF from the frames (if requested). Again, this is done via a system call to the relevant tool (gm or ffmpeg or none). After that we delete movie_init.sh, background.ps and foreground.ps (if present) and then report back the name of the products.

Events

Events are plot overlays that are intermittent. As a simple example, consider wanting to add a label that should only be visible for frames 24 through 50. One possible implementation of this would be -E<events.txt>, where the event file contains one or more events (one per line) in the format

type  start  stop snippet

where type is either F (frame numbers) or an integer ? > 0 which refers to the GMT_MOVIE_VAL? value. This determines what values are checked against the start and stop values. In the label case above it may look like this:
F 24 50 echo SUBDUCTION | pstext -F+f24p+cTR

whereas if a MOVIE_VAL was used then perhaps start and stop are floating point limits on time, distance, or whatever. In the context of the movie scripts it may be simpler to have move.c determine if an event should occur on a per frame basis and define EVENT? as 1 or 0 in the corresponding parameter file, and let the main script have one or more "if (event) <do snippet> end" statements appended at the end of main.sh, depending on scripting language.

Scripting languages

At present, we have implemented support for Bourne and Bourne Again shells, the C-shell syntax, and DOS BAT files.