Dear John
Thanks for your posting. I have changed the subject so as not to mix it up
with the discussion about the format for maintaining the document.
The rules expect that a test file is provided to accompany a change to the
conventions, but this has rarely or never been done in practice. We could
improve the procedure by insisting on this, when it's relevant.
I agree with you that provisional status has a cost in complication and has
not given a benefit. The idea of provisional status is that it should be easy
to reverse provisional changes, I suppose, but we have not written down how
this would be done in practice, because we've not needed to. Therefore I would
be happy if we abolished provisional status. That means if a change is found
to be wrong, it would have to be reversed by opening a new ticket.
Perhaps we could make a new procedure to help with this, should it ever be
needed. If a ticket is agreed to reverse a previous change which was found
to be wrong, it could mean that a new version of the convention is released
straight away, with the error corrected, so that it doesn't have to wait to
go live, and that the current and any other versions of the convention which
contain the erroneous change would then be deprecated.
Best wishes
Jonathan
----- Forwarded message from John Graybeal <jbgraybeal at mindspring.com> -----
> Date: Wed, 28 Jan 2015 11:07:56 -0800
> From: John Graybeal <jbgraybeal at mindspring.com>
> To: Jonathan Gregory <j.m.gregory at reading.ac.uk>, CF Metadata List
> <cf-metadata at cgd.ucar.edu>
> Subject: Re: [CF-metadata] Editing/publishing workflow update
> X-Mailer: Apple Mail (2.1878.6)
>
> Jonathan, thanks for this update, just the information I was looking for. I agree this is a good opportunity for this discussion, and a good one to start on the list.
>
> In what follows, I explore the need for a provisional period, and advocate its elimination. Instead we can add a simple requirement for an example data file (binary and NCML), demonstrating the proposed change before it is approved. (An example application using the data file could be encouraged, but in most cases would not be important to recognize the validity.)
>
> Why should we make this adjustment?
>
> (1) There are definite technical and social costs to the provisional period. Technical, to implement and maintain provisional text, and to describe how it works. (And have this discussion every few years.) Social, in the creation of a burden of understanding for new users, who see a needlessly complicated presentation of the standard; and then a non-trivial explanation of the presentation's meaning; and finally have to consider themselves how important it is to consider the provisional parts in their own work. And how often would we tell that user to avoid a provisional change?
>
> (2) The statement that it has not proved necessary to date -- in how many years and CF versions? -- is a powerful consideration. (Compare to CF's insistence on only considering standard names *that are already needed.*)
>
> But for the possible edge cases to come, let's take the alternate case, where a new specification is adopted and proves unworkable, and see what the worst case scenario is.
>
> Given that a few applications will simultaneously implement the change, one or more of them will find the problem, and report it. There are a three possibilities: the problem can be tolerated; the problem must be fixed but the fix advances the change; or the problem must be fixed by reverting the change.
>
> The vast majority will certainly be in the first two categories. Indeed, both have been experienced already in minor forms, as CF improves each cycle. Prompt attention will certainly be given in the second case, so the cost, inconvenience, and damage are minimized (and in any case will be equally incurred with a provisional period).
>
> In the third case, what are the costs without a provisional period? A few groups have unexpectedly lost some time -- but that will happen with the provisional period too. The community/standard must backtrack, issuing a revision that undoes the change -- but this too comes with the provisional period. There might be an arbitrary number of data sets following the flawed interim specification, but this can happen with the provisional case too (and they may get the advantage of the change without the impact of its flaws anyway). While there are potentially squirrelly edge cases here, they are way, way down in the weeds, and are not actually avoided in any practical sense by the provisional period.
>
> (3) It doesn't really help. From this above scenarios, I see the benefits of the provisional period, even in the most unlikely and pathological case of oversights and errors leading to squirrelly outcomes, are minimal at best.
>
> John
>
>
>
>
>
> On Jan 28, 2015, at 10:05, Jonathan Gregory <j.m.gregory at reading.ac.uk> wrote:
>
> > Dear John, Seth and all.
> >
> > ----- Forwarded message from John Graybeal <jbgraybeal at mindspring.com> -----
> >
> >> On Jan 27, 2015, at 08:31, Jonathan Gregory <j.m.gregory at reading.ac.uk> wrote:
> >>> ...all changes ever since the first version are still shown as provisional because we have no rule for accepting them as permanent
> >>
> >> This seems at odds with Seth's observations about when changes appear to go away (with each new release).
> >
> > I haven't checked what the documents show, but the intention is that all
> > changes should still be provisional. I think that's Jeff's understanding too.
> >
> >> I'm pretty sure the intent was also to validate the changes -- i.e., that they were not considered fully approved until they had been around long enough to inspire confidence based on use.
> >
> > That's right. The rules say that two applications must have demonstrated that
> > they work. See http://cfconventions.org/rules.html. However we have never
> > organised this process so it's never happened. I agree that's unsatisfactory.
> > Although this thread is about document markup, it's a good time to review the
> > rules in this regard. In fact I think we discussed it before, a few months ago.
> >
> > During the whole time CF has existed, we have never needed to reverse a change
> > because it turned out to be erroneous or problematic. I suppose this is because
> > we are always thorough in agreeing them in the first place. However it is still
> > sensible to have some provisional period, just in case, and perhaps we can
> > presume that if there is a problem, someone will raise it, rather than having
> > to set in place a verification process. I presume we should stick to releases
> > of the doc. We could update it continually with trac tickets but I think that
> > would be more difficult for users of the convention to keep track of.
> >
> > Bearing all that in mind, my initial suggestion would be that a change is
> > marked as provisional in the version in which it is first made and any
> > subsequent versions which are released less than a year after the change is
> > first made. This will mean checking the history to see which provisional
> > changes become permanent whenever a new version is compiled, and that's a bit
> > of work. Since it's more than a year since we last had a new version, that
> > would means all changes that have previously been made would be permanent in
> > 1.7. The only provisional changes would be those newly made in 1.7.
> >
> > Changing the rules is something that the conventions committee would have to
> > decide about, but it'd be good if we can agree something on this email list.
> > What do you think?
> >
> > Best wishes
> >
> > Jonathan
> > _______________________________________________
> > CF-metadata mailing list
> > CF-metadata at cgd.ucar.edu
> > http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
>
> _______________________________________________
> CF-metadata mailing list
> CF-metadata at cgd.ucar.edu
> http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
----- End forwarded message -----
Received on Wed Feb 04 2015 - 07:11:33 GMT
