Possible Bug in Editing Master Sources

RootsMagic
21

Aha! But that means that once you have used the “Reset to defaults” function for a citation, you can never really go back to the default behavior. You are stuck with using the “Reset to defaults” function with that citation forever.

It still raises the question of how my citation got into the problem in the first place, given that I never use the Customize Sentence feature. I’m going to have to look at my CitationTable and see how many of the citations show the use of the Customize Sentence feature. Maybe tonight. I’m still trying to get out the door and away from the computer for the rest of the day.

22

You may have missed my edit in which I describe that the way to actually reset to defaults is to customise the sentence as an empty sentence. That erases whatever is stored in CitationTable.Footnote, etc., allowing the template generated sentence to be outputted in Preview and Reports.

1 Like
23

I’ve also just confirmed that this bug applies to Citation Details for one of my custom, lumped templates, too, so it is not only an issue for the Master Source nor only for your specific example. It also apples to free Form sources. All one has to do is open the Customise dialog and the override entry field is populated with the generated sentence. I think that was true in RM7 but the interaction between the forms and the database tables was a little more logical. I’m finding RM9 is quite confusing as to what actions on which form interact with the other form or the table. For example, the Edit Citation Footnote Preview may be responding to changes in Source or Citation fields but the Customise form may be showing what was generated in the same Edit session if it had been opened prior to further changes in variables.

It would take some very methodical testing to track what happens where at each step in the various paths one could follow but I’m reasonably confident that, at the heart of the problem, is a mistake in the logic of “Reset to defaults”, perhaps even a misinterpretation of what “defaults” means. It’s as though a new programmer has misunderstood that it means that one is supposed to populate the CitationTable.Footnote field with the template-generated sentence when the rest of the system ignores generating a sentence from the template when that field is non-empty.

24

I think it timely to recap my findings so far to correct something and expand on others.

  1. The issue is not solely with the editing of Master Sources but also with the editing of Citations in RM9.
  2. The issue is identical in RM8.
  3. The “Customize” citation feature for Footnote, Short Footnote, Bibliography sentences was introduced in RM8 and has undergone no apparent change since.
  4. There is no explanation of how to use “Customize” citation sentences in the RM8 Wiki or the RM9 Help; I do not know if there is in the book (perhaps someone could post a scan).
  5. It could be argued that “Customize” citation sentences is working as designed and is not buggy. However, its “Reset to Defaults” functions illogically and opposite to the Customize feature for fact/event sentences in both RM8 and RM9 (and in RM7) on which its user interface is modelled.
  6. It seems to be another half-baked new feature with good intentions but seriously flawed and possibly misunderstood and unreported by those who have been caught in the same trap as you!

Compare the behaviour of Customize between RM9 Fact Sentence and RM9 Source sentences:

  1. Both show a preview of the output sentences with all the template variables referenced by the sentence template expanded to their stored values.
  2. The preview sentence is taken respectively from the database’s EventTable.Sentence content or the CitationTable.Footnote (CitationTable also has ShortFootnote and Bibliography columns but I will just use Footnote henceforth) if it is non-empty else it is taken from the respective sentence template defined in the FactTypeTable or the SourceTemplateTable.
  3. The sentence generated by the sentence template is what we understand to be the “default”. Here’s where things go off the rails.
  4. Both the EventTable.Sentence and the CitationTable.Footnote for a new record in each remain empty until specific operations are undertaken in their respective “Customize”, thus Preview and Reports output the “default” sentence generated using the FactType or SourceTemplate sentence template.
  5. Variance #1: On first opening, EventSentence.Customize form shows the FactTypeTable.Sentence template with unexpanded variables; Citation.Customize form shows the expanded SourceTemplateTable.FootnoteSentence, i.e., variable names are replaced by the stored values.
  6. Variance #2: Without making any edits, OK and Reset to default(s) do the following: EventTable.Sentence is left empty so the default sentence continues to be output in preview and reports varying according to changes in variable values but CitationTable.Footnote is overwritten with the expanded sentence that is insensitive to changes in the values of variables - it has become static.
  7. Variance #3: The Citation sentence preview cannot expand from variable names even if they are manually entered in the Customize form; the [ and ] characters are treated as literals. The Event sentence preview functions correctly with manually entered variable names (provided they match exactly with the names defined by hard coding in the software).
  8. Variance #4: While RM Inc might argue that Citation.Customize,Reset to Defaults or OK does produce the same default sentence that an empty CitationTable.Footnote field would allow at that point in time, it creates the probability of a divergence among different Citations of the same Source when changes are made to the Master Source values after the customization or reset of one and that of another.
  9. To restore the Citations to using the dynamic, up-to-date sentences requires one to Customize, delete the contents and OK. DO NOT USE Reset to defaults. For those capable of SQLite, a simple query can clear all customizations from the CitationTable by emptying the Footnote, ShortFootnote and Bibliography columns.

So that’s the mess it is now. To make Citation Customize function as logically as Fact Customize requires:

  1. Preview must expand template variables found in the CitationTable.Footnote…
  2. Customize must receive the default, unexpanded sentence template from the SourceTemplateTable for editing and generate previews therefrom and with subsequent edits in the same window.
  3. Reset to defaults must empty the CitationTable.Footnote, ShortFootnote, Bibliography fields.
  4. Help must be elaborated to describe how to use Customize.

Those are the bare minimum requirements. Because SourceTemplates have a vast number of different variable names, the UI needs further enhancements, e.g., a way of viewing the available variable names and their definitions from within the Customize Editor. The Citation Manager/Editor should show whether a Citation has any customised sentences. I’m sure there are others that escape my now exhausted mind!

3 Likes
25

Thank you for this wonderful summary of the situation and all the time you had to put into the research. I do hope the problem is addressed.

1 Like
26

First step is getting getting it acknowledged by @rzamor1 .

Your question as to how your example got into the static state when you “never customize” is probably answered as “finger misapplication likely due to misleading UI”! Just as we have found it confusing as to what causes some variable to be saved and what causes some display to be updated which led you to skip the OK button after “Reset to defaults” in the video and your years of exposure to “Reset to default” in Fact Sentences conditioning you to think it would behave the same way for Citations led you to apply it, it seems highly likely that sometime in the past you hit OK instead of Cancel or X, or maybe you used Reset, trusting it to do what you rightfully expected. Earlier today, I was puzzled that SQLite was not showing a change I expected from Customizing a Fact sentence until I realised I had to switch to a different fact in RM9 to trigger the save of the custom sentence…

1 Like
27

I checked my database with SQLite, and I only have the one person with the problem. And it’s the person I was playing with to try to replicate the “parentheses become quotes in source template name fields” problem. The only rational explanation is that I clicked on the Customize tool by accident (or on purpose without remembering I did it) as a part of that investigation. I do realize that never is a long time. So my “never” probably needs to be qualified to be “never on purpose” or “never in my normal use of RM”.

28

Yep, that’s a specific instance of a general principle that from the user’s point of view, it’s often very fuzzy exactly when or if a newly entered piece of data has been saved to the database.

29

Well, that pretty much shoots down another hypothesis that I was going to explore: that your past observation about identical endnotes not being consolidated in RM9 as they used to be in RM7 might be related to a mix of default and customised Citations of the same Source. I may explore that anyway along with how Merge All Duplicate Citations manages such a mix that all happen to produce the same sentence.

30

[Edited to correct link]

This Customize Citation feature introduced in RM8 is one that slipped by me unnoticed or unrecognised for what it might have responded to. It’s not brought out explicitly in either RM8 or RM9 ‘new features’ lists. There is something called ‘live edits’ which is a euphemism that did not mean anything to me. When thinking about what one might use it for, it dawned on me that it might actually be a response to one or another enhancement request that you or I made 'way back in RM4!

I remember posting about using any source template to construct a draft set of sentences that I would copy into the Free Form citation (preferably an enhanced FF like my !MyFreeForm3 template) and how it would be expedited if that procedure was supported by the UI. While I have not found that discussion in my archive of old enhancement requests, I did find a related post of yours from 2011:
A Radical Idea for Source Templates and Sharing Facts

What ‘Customize’ does is similar, if one exercises the option to use it, but it completely eliminates any template variables from its output while I had in mind that the FreeForm [Footnote], [ShortFootnote], [Bibliography] and [Page] variables would continue to control output, plus the [ShortPage] variable if my custom FreeForm template was adopted. In a way, it goes even further, by allowing the output Footnote, Short Footnote and Bibliography sentences to be truly arbitrary, completely under user control. This may lend itself well to footnoting that is commentary, not citation, and maybe other things.

I’m surprised (or maybe I missed it) that there has been absolutely no discussion that I can recall on this powerful, albeit immature, feature. That it does not support the source template language may not be as critical as I thought (it would bypass that hurdle for many users), but I do think it is critical that the ‘Reset to defaults’ followed by OK should clear the override fields in the CitationTable so that the current template sentence is what is outputted, not the text from the last time it was generated.

What do you think?

31

The link to A Radical Idea for Source Template and Sharing Facts does not work for me, so I’m trying to remember what I said.

I do know that over the years I have developed some antipathy towards RM’s style of source templates (even though I use them heavily and happily!), and also antipathy towards the style of doing footnotes and and endnotes where there is a left-part and a right-part to each footnote. The left-part and right-part thing seems to be global to all of genealogy and is not specific to RM. For example, the left-part and right-part thing manifests itself in GEDCOM via the SOUR tag which is the left-part and the PAGE tag which is the right-part. It manifests itself in RM’s free form template as Footnote which is the left-part and Page Number which is the right-part.

In this sense, RM’s free form templates mirror exactly with the GEDCOM tags of SOUR and PAGE. Except that the RM terminology is totally illogical. In effect, in RM we have Footnote = Footnote + Page Number where the term “footnote” means two different things at the same time. At least GEDCOM has the decency to call it Footnote = SOUR + PAGE where there is no contradictory meaning.

One influence in my thinking is that Web sites that provide document images or other genealogical information generally speaking will provide you with a completed footnote that you can just copy and paste into your genealogy software as is. Except that because of things like source templates and the left-part and right-part thing, genealogy software generally speaking has no place to paste such completed footnotes. At a minimum, you as the user have to break up the already completed footnote into a left-part and right-part to paste into your genealogy software. At a maximum, you as the user have to break up the already completed footnote into individual source template fields. This does not seem to me to be a good situation.

RM8 and RM9’s support for the Customize Citation feature certainly offers a solution to this dilemma. However, it seems to me that there is little documentation for how it really works. For example, if you do a Customize of a citation and then export a GEDCOM, how much of the customized citation goes into SOUR and how much goes into PAGE. I haven’t tested, but I suspect it’s 100% SOUR and 0% PAGE so that you have a completely split citation. That’s not necessarily a bad thing, but it’s something that should be understood if that’s the way it works. And if that’s the way it works, it doesn’t force the user who just wants to paste a completed footnote into RM to split up the footnote before pasting it. Rather, it simply forces the user to become an extreme source splitter for that particular footnote.

I apologize to Bruce in advance if I’m misremembering something. But I seem to remember one video he made in the earliest days of RM8 that mentioned this feature briefly. What I remember is not a discussion of being able paste a completed footnote directly into RM using the Customize feature. Rather, what I remember is a discussion about how the Customize feature could be used if the user simply couldn’t get a footnote to look exactly right - like if there were two many or two few spaces, or too many or too few commas, or a completely missing comma. I don’t remember the Customize feature being presented as a radical overthrow of the whole left-part and right-part model and as flying in the face of RM’s support for source templates.

I do agree that Customize => Reset To Defaults must clear all customization and restore control to the source template. I think failure to do so is either a programming bug or else a serious design flaw. Otherwise, the user is completely trapped into the customization and can never get back to the use of the source template for that particular footnote. For my one citation with the problem, I cleared all the customization using SQLite. And I cleared all three customized fields - Footnote, Short Footnote, and Bibliography. I didn’t take the time to explore whether simply clearing the Footnote field would have been sufficient.

I don’t want to get too off subject, but I remain an extreme source splitter in RM9 even though RM9’s reusable citations should render moot my original reasons for becoming an extreme source splitter. However, there are several things that have made me cautious about adopting RM9’s reusable citations in place of my own extremely split citations. One concern that’s at the back of my mind is the following. With my extremely split citations, there is no left-part and right-part. There is only one part. Well, I have to store the whole footnote as the left-part, but I don’t have to decide which source template variables go in which part. If I switch to using RM’s reusable citations instead of using my own completely split citations, then I will have to deal with the left-part and right-part issue. I’m not sure yet that I want to do that.

32

Sorry, this should work:

33

Will GEDCOM v7 be a path forward in improving source/citation transport between apps?