Bug #588

gmt2kml (GMT5) can no longer read binary files

Added by Remko over 2 years ago. Updated over 2 years ago.

Status:In ProgressStart date:2014-07-14
Priority:NormalDue date:
Assignee:Paul% Done:

90%

Category:-
Target version:Candidate for next bugfix release
Affected version:5.1.x Platform:Mac OS X

Description

gmt2kml used to be able to read binary (netCDF) files (in GMT4). That is no longer the case because it forces text input.

There is a snippet of code referring to binary data, but I don't get it. Why doesn't gmt2kml use a more generic opening of files?

    /* If binary input then call GMT_gmtconvert to handle that and read in as ascii, else do as below.
       Or open as GMT_IS_DATASET if binary and TEXTSET otherwise, then handle the differenes below
       by setting n_tables = (binary) ? D->n_tables : T->n_tables; same for nsegments, then
       final ifs on parsing text record or using coord as is. */

    if (GMT_Init_IO (API, GMT_IS_TEXTSET, Ctrl->F.geometry, GMT_IN, GMT_ADD_DEFAULT, 0, options) != GMT_OK) {    /* Establishes data input */
        Return (API->error);
    }
    if ((Din = GMT_Read_Data (API, GMT_IS_TEXTSET, GMT_IS_FILE, 0, GMT_READ_NORMAL, NULL, NULL, NULL)) == NULL) {
        Return (API->error);
    }

This was probably before the oldest revision in the repository (r7637), hence in GMT4. So I guess gmt2kml never worked with binary files in GMT5.
Since I don't know the API will enough, I assign this to Paul.

gmt2kml.gmt - GMT example script (316 Bytes) Remko, 2014-07-15 22:28

JA2_GPR_2PdP037_004_20090704_031541_20090704_041154.nc - NetCDF input file (330 KB) Remko, 2014-07-15 22:28

gmt2kml.kml - Result from GMT4 (321 KB) Remko, 2014-07-15 22:28

Associated revisions

Revision 13359
Added by Paul over 2 years ago

Addressing issue #588

Revision 13360
Added by Paul over 2 years ago

Fix issue #588 with regards to CPT values and honoring -R

History

#1 Updated by Paul over 2 years ago

  • Status changed from New to Feedback

gmt2kml has to be able to deal with text inputs such as special labels and what not that the user may specify. For a binary file none of those apply of course, but that also means we either must have two separate paths through the program (see GMT4 solution) or do something different. I may be able to have a check for binary or column selections and then read in a dataset, else we read in a text set, and then abstract away a few things such as n_segments = (binary_file) ? D→n_segments : Di→n_segments; etc etc. It can be done.
But it may be simpler to read in the datatable and convert it to a text table and move on from there.

#2 Updated by Paul over 2 years ago

  • Status changed from Feedback to Resolved
  • % Done changed from 0 to 100

Alright, I made those changes by reading binary files as a dataset, and then deal with the differences. Seems to work, please test with netcdf file etc. In r13357.

#3 Updated by Remko over 2 years ago

I'm afraid it has not been resolved. See the attached script and input file. That one works in GMT4, but crashes on GMT5. Here the GMT5 output:

+ makecpt -Crainbow -T-0.2/0.2/0.01
+ gmt2kml 'JA2_GPR_2PdP037_004_20090704_031541_20090704_041154.nc?lon/lat/ssha' -Aa0 -Fs -Cmy_rainbow.cpt -N -R-180/-100/0/80 -K
ERROR: Caught signal number 11 (Segmentation fault) at
0   libsystem_c.dylib                   0x00007fff97821eca flockfile + 18
1   ???                                 0xffffffffffff0068 0x0 + 18446744073709486184
Stack backtrace:
0   libgmt.5.2.0.dylib                  0x0000000101c1ca6b sig_handler + 347
1   libsystem_platform.dylib            0x00007fff998955aa _sigtramp + 26
2   ???                                 0x0000000000000000 0x0 + 0
3   libsystem_c.dylib                   0x00007fff978228dc fgets + 41
4   libgmt.5.2.0.dylib                  0x0000000101c58bc2 GMT_ascii_textinput + 322
5   libgmt.5.2.0.dylib                  0x0000000101c616a8 GMT_read_texttable + 472
6   libgmt.5.2.0.dylib                  0x0000000101c23278 GMTAPI_Import_Textset + 1224
7   libgmt.5.2.0.dylib                  0x0000000101c26a01 GMTAPI_Import_Data + 305
8   libgmt.5.2.0.dylib                  0x0000000101c29ca4 GMT_Get_Data + 452
9   libgmt.5.2.0.dylib                  0x0000000101c29fb9 GMT_Read_Data + 633
10  libgmt.5.2.0.dylib                  0x0000000101d60492 GMT_gmt2kml + 4914
11  libgmt.5.2.0.dylib                  0x0000000101c2d9e3 GMT_Call_Module + 835
12  gmt2kml                             0x0000000101c0f7dc main + 1228
13  libdyld.dylib                       0x00007fff980c45fd start + 1
+ gmt2kml 'JA2_GPR_2PdP037_004_20090704_031541_20090704_041154.nc?lon/lat/ssha' -Aax2e5 -E -Fl -R -O
ERROR: Caught signal number 11 (Segmentation fault) at
0   libsystem_c.dylib                   0x00007fff97821eca flockfile + 18
1   ???                                 0xffffffffffff0068 0x0 + 18446744073709486184
Stack backtrace:
0   libgmt.5.2.0.dylib                  0x00000001004e6a6b sig_handler + 347
1   libsystem_platform.dylib            0x00007fff998955aa _sigtramp + 26
2   ???                                 0x0000000000000000 0x0 + 0
3   libsystem_c.dylib                   0x00007fff978228dc fgets + 41
4   libgmt.5.2.0.dylib                  0x0000000100522bc2 GMT_ascii_textinput + 322
5   libgmt.5.2.0.dylib                  0x000000010052b6a8 GMT_read_texttable + 472
6   libgmt.5.2.0.dylib                  0x00000001004ed278 GMTAPI_Import_Textset + 1224
7   libgmt.5.2.0.dylib                  0x00000001004f0a01 GMTAPI_Import_Data + 305
8   libgmt.5.2.0.dylib                  0x00000001004f3ca4 GMT_Get_Data + 452
9   libgmt.5.2.0.dylib                  0x00000001004f3fb9 GMT_Read_Data + 633
10  libgmt.5.2.0.dylib                  0x000000010062a492 GMT_gmt2kml + 4914
11  libgmt.5.2.0.dylib                  0x00000001004f79e3 GMT_Call_Module + 835
12  gmt2kml                             0x00000001004e07dc main + 1228
13  libdyld.dylib                       0x00007fff980c45fd start + 1

#4 Updated by Paul over 2 years ago

  • % Done changed from 50 to 80

Sorry, did not run your test - somehow I forgot. I have fixed things so it now recognizes the netcdf file. It reads the three columns correctly I think. But kml not the same (yet). Also 11 pm here and up early tomorrow so I can checking in what I have in case you have a time to look further. In r13359.

#5 Updated by Remko over 2 years ago

Three things are still wrong:
  1. Symbols don't get the colours they out to have. This is might indicate that the third column is stored as NaN in the first call to gmt2kml in the example script. From the second call I at least get a line at non-zero altitude.
  2. The -R option seems to have no affect.
  3. When adding the -L option (-Llon,lat,ssha) all data elements are empty:
                                                 <ExtendedData>
                                                            <Data name = "lon">
                                                            <value>
    
                                                            </value>
                                                            </Data>
                                                            <Data name = "lat">
                                                            <value>
    
                                                            </value>
                                                            </Data>
                                                            <Data name = "ssha">
                                                            <value>
    
                                                            </value>
                                                            </Data>
                                                    </ExtendedData>
    

#6 Updated by Paul over 2 years ago

Thanks. will try to have a look tomorrow; today has other priorities...

#7 Updated by Paul over 2 years ago

  • % Done changed from 80 to 90

Both the -R issue and the failure to use the CPT colors have been fixed. As for -L, the documentation says this option can be used with ascii data and that we expect the named columns to simply follow the regular columns in the file in the given order. So not working for you is a feature. I agree that with netCDF it might be nice to specify -L with arbitrary variables but then we would have to read them as well and it becomes much more of an issue. It would be much easier to accommodate your particular case in which -L lists the same variables as you are reading in anyway. We would just have to determine which order they were given when read and then sort things out. But it would require an if (netcdfdata) test. In r13360.

#8 Updated by Remko over 2 years ago

It's not a big priority for me either, but -L used to work, with netCDF data in GMT4. But I can circumvent this by doing a gmtconvert first. Not much of a big deal.

Also available in: Atom PDF