Feature #656

Improved model specification for trend1d, trend2d, grdtrend

Added by Paul almost 7 years ago. Updated almost 6 years ago.

Status:In ProgressStart date:2014-12-10
Priority:NormalDue date:
Assignee:Paul% Done:


Target version:Candidate for next minor release


These modules fit trends to 1-D, 2-D, and grids. However, they are limited in what models to fit. For instance, we have received requests to allow trend1d to fit a seasonal signal plus trend but we cannot currently do this. This issue tries to define how the models should be specified to allow for more flexibility.

trend1d: Currently takes -N[f]<n>[r], for polynomial of n terms, prepend f for Fourier series instead; append r for robust.

Suggestion: -N[f|s|c]<list-of-terms>[,[f|s|c]<list-of-terms>,...][r]

Here, list-of-terms is such that we include all x^n, for all n in the list-of-terms. E.g., -N0,2 would fit m_0 + m_2 * x^2, while -N0-3 would fit m_0 + m_1*x + m_2 * x^2 + m_3 * x^3. If list-of-terms starts with f then we fit the Fourier terms c_n * cos (2*pi*n*x/T) + s_n * sin (2*pi*n*x/T), where T is the length of the input series. If c or s is used instead of f then we only add the cosine or sine term. A trailing r means robust fit.
FInal example: -N0-1,f1 would fit model y(x) = m_0 + m_1 * x + c_1 * cos (2*pi*x/T) + s_1 * sin (2*pi*x/T). Note: Either 0 or f0 can be used to include the constant m_0.

trend2 and grdtrend: Currently takes -N<n_model>[r], where n_model is in the 1-10 range (10 means full bicubic model). There is no f modifier.

I suggest trend2d and grdtrend should have the same syntax as trend1d, with the understanding that an n like 3 would mean "include x^3, x^2y y^2x, and y^3". This means we would not be able to tinker with models that exclude some of these terms (which we cannot do in the current version anyway), Here, f,c,s would include Fourier terms in both x and y.

Example: -N0-2,f1 would fit z(x,y) =m_0 + m_1*x + m_2*y + m_3*x^2 + m_4*x*y + m_5*y^2 + cx_1 * cos (2*pi*x/X) + sx_1 * sin (2*pi*x/X) + cy_1 * cos (2*pi*y/Y) + sy_1 * sin (2*pi*y/Y)

where X and Y are the length of the x and y dimensions. This proposal has several benefits:
  • Dramatically generalizes these modules and allow for needed flexibility.
  • It also extends the Fourier building blocks to 2-D.
  • Keeps the syntax consistent across three modules
  • Compact notation for the 2-D cases

If anyone wants the 2-D fits to be able to exclude cross-terms etc then one could consider -N[x|y||xy][f|s|c]<list-of-terms>[,[f|s|c]<list-of-terms>,...][r] but this would be messier.


#1 Updated by Paul over 6 years ago

To make it easier to maintain backwards compatibility we should consider these changes:
  1. Require p for polynomial instead of it being a default basis.
  2. Use +r rather than just r to indicate robust fit.
  3. Accept +x<period> (and +y<period> for trend2d and grdtrend) to set the fundamental period(s) [data domain].

#2 Updated by Paul over 6 years ago

The above handles the Fourier series in a flexible way. In order to fully allow for complete control over which basis are used for polynomials we could consider listing the basis in different ways:
  • -Np2 would mean polynomial of order two and if 2-D would include x^2, y^2 and xy, i.e., the scheme above.
  • -Nxx,yy would only select x^2 and y^2 but not x*y. We would accept things like xxxyy or x3y2. When x and y are used the p is not required.

#3 Updated by Paul almost 6 years ago

  • Status changed from New to In Progress
  • Assignee set to Paul

The 1-D method outlined above for trend1d was implemented in 5.2.1. Work remains to explore the 2-D portion to make sure it is not overly complex.

Also available in: Atom PDF