Investigating Long Endnote Lists in RM8

RootsMagic
1

I have been investigating the problem where RM8’s list of endnotes can be so much longer than RM7’s list. The simple explanation is that RM8’s feature to Reuse Endnote Numbers must have a bug. The feature works sometimes and not sometimes. I have not yet discovered what the pattern is where it sometimes works and sometimes doesn’t.

Well, I do know that if you import an RM7 database and don’t run the Merge Duplicate Citations process, the duplicate citations each get their own unique endnote number in a report. The same thing happens if you Memorize a citation in RM8 and then do Paste/Copy rather than a Paste/Reuse and if the citations created via Paste/Copy are never modified. I think the Reuse Endnote Numbers feature should work even in these cases. But rather than testing the text of the citation it seems to be testing the citation numbers in the CItationTable which will all be different for unmerged citations. That’s just a guess, but that’s one thing that seems to be happening.

But I always do a Merge Duplicate Citations, and still I get bunches of duplicate citations that won’t use the same endnote number despite turning on the Reuse Endnote Numbers feature. This is when I run large reports from my real database. But so far, I cannot reproduce the problem with a small database containing only a few people. With a small database, the Reuse Endnote Numbers seems to be working correctly.

None of my first guesses as to the cause of the problem have been correct. In my small test database, endnote numbers are reused properly if the citations have different quality indicators or if I deliberately paste the same citation to the same fact multiple times. So I will continue looking for other causes that might be repeatable.

2 Likes
2

I’ve seen the same problem Jerry and have also been trying to figure out why this might be so. I have a very small database (1,300 people) but have noticed the duplicate citation issue despite selecting the Reuse Endnote option - just can’t seem to replicate it now to display! Please let me know if you come up with any other insights (and I’ll do the same!) :slight_smile:

3

What is an EndNote? How do you find them?

4

An endnote is a citation sentence.

When you run a report, the citation sentences for each page can appear at the bottom of each page. If that’s where you print them, they are called footnotes. Another option is that citation sentences for all the pages can appear together at the end of the report. If that’s where you print them, they are called endnotes. They are the same sentences no matter where they are printed or what they are called. It can be a little confusing. RM calls the citation sentences footnote sentences when you are making them. It calls them footnote sentences or endnote sentences when you are printing them, depending on where you print them.

If you are working with sources and citations in RM, you will see that RM is making the citation sentences for you from the source data you typing in. That’s the real purpose of all that source template stuff in RM. The source templates produce citation sentences for you so you don’t have to produce them yourselves. You put your source information into the source template and the source template produces the citation sentences. But at this point in the RM user interface, RM doesn’t call them citation sentences. It calls them footnote sentences even though they may end up being printed as footnote sentences or as endnote sentences.

The “reuse” option citation sentences is called Reuse Duplicate Endnotes. Duplicate citation sentences are always printed multiple times as footnotes and cannot be reused. Duplicate citation sentences may be reused and printed only a single time as endnotes. That’s why there is a Reuse Duplicate Endnotes option and there is not a Reuse Duplicate Footnotes option.

You would like to avoid duplicate citation sentences in reports. If the same census page serves as a source for four members of the same family, then there will be a superscript citation number somewhere in the report for each of the four family members. The superscript citation numbers also appear at the front of each footnote or endnote in the lists at the bottom of the page or the end of the report. In the worst case scenario, each family member will have a different superscript number for the census citation and the citation sentence will be duplicated four times as either a footnote or an endnote. In the best case scenario, each family number will have the same superscript number for the census citation and the citation sentence will appear only one time when you are printing endnotes.

That’s what the Reuse Endnote Numbers option is all about. It’s an option for reports. It’s not an option for the sources and citations themselves. The option is under endnotes in the options box under things like Publish => Narrative Reports or Publish => Family Group Sheet or an other report. On my computer the options box for printing won’t fit on my screen so I have to scroll down to see the endnote options.

In the abstract, I prefer footnotes to endnotes so that my reader can simply look at the bottom of the current page to see where each piece of source information came from rather than having to look at the end of the report. But compared to most papers such as you might have to write in a high school or college English class, a genealogy report contains a gazillion citations. As a result, footnotes can easily occupy 1/2 to 2/3 of each page. Also, footnotes provide no opportunity to reuse the citation number and endnotes do provide such an opportunity. For those reasons, I always specify the endnote option rather than the footnote option when I run any of RM’s reports.

2 Likes
5

I have now made some progress on identifying the causes of my endnote lists in RM8 being much longer than in RM7 for the same report.

In the first place, if you Memorize a citation in RM8 and Paste/Copy it rather than doing a Paste/Reuse, the citation cannot be reused by the Reuse Duplicate Endnotes option in reports. The option is not looking at the text of the endnotes to see if they are duplicate. Rather, it is looking at the internal table structure of the RM8 database. If the citations are separate citations in the RM8 database, they are not reused by the Reuse Duplicate Endnotes option in reports even if the citations are otherwise identical.

In the second place, otherwise duplicate citations come into RM8 from RM7 in the Paste/Copy state rather than in the Paste/Reuse state. Therefore, it is really important after importing an RM7 database into RM8 to run the Sources => THREE DOTS => Merge All Duplicate Citations process. This process converts all citations that are truly duplicate in all respects from the Paste/Copy state to the Paste/Reuse state.

I have been running the Sources => THREE DOTS => Merge All Duplicate Citations process every single time I import my RM7 database into a new RM8 database ever since the beginning of the RM8 Community Preview. Nevertheless I still have lots of citations that cannot be reused by the Reuse Duplicate Endnotes option in reports.

What is happening is that sometimes citations that appear to be duplicate will not merge automatically because of very subtle and sometimes invisible differences in the citations. They can be forced to merge on a one citation at a time basis in RM8 by the Sources => Citations => THREE DOTS => Merge Citations process.

There are several problems with doing it this way. One is that I have to redo it every time I import from RM7 into RM8, which is still several times a week for me. A second is that I have lots of citations that have this problem. A third is that you are almost merging blind. You have to do a lot of slipping and sliding in the sources => Citations => THREE DOTS => Merge Citations process to find out where the “almost duplicate” citations are being used. Even with the slipping and sliding, sometimes it can be very hard or even impossible from the RM8 user interface to see what the differences really are.

I suspect that such minor differences don’t really matter. If two citations that won’t merge automatically are so similar that you can’t tell the difference, your best strategy is probably just to merge them and to ignore any invisible differences. But for the time being, my strategy is to use RM8 to identify citations that won’t merge and to make whatever changes to the same citations in RM7 are necessary to make them merge automatically the next time I import them from RM7 to RM8. I will post more details about my process for cleaning up my citations in RM7 in a future message.

In the meantime, I should report that the Reuse Duplicate Endnotes option in RM8’s reports does work correctly when its definition of “Duplicate” is understood. It does not mean duplicate endnote sentences. It means duplicate citations in RM8’s internal database structure. This in turn means that you have to use Paste/Reuse for citations rather than Paste/Copy nearly all the time and that you have to merge your duplicate citations any time you import from RM7 into RM8. Otherwise, the Reuse Duplicate Endnotes option will not work and you will have humongously long endnote lists in your reports.

2 Likes
6

Thanks so much for sharing all the work that you have done in this space Jerry and the outcomes. I really appreciate all your insightful posts and I suspect they assist a lot of us RM users!

It’s bugging me, but I can’t find the report where the endnotes were duplicating, despite selecting the Reuse End Note option. I suspect it’s, like you say, that even though the Merge Duplicate Citations was ran, that there was some little inconsistency that meant that, what on the face of it seemed to be identical citations, were in fact not, and so they were repeated in the Endnotes.

@Marshtown – Jerry has provided a very full response to your question about Endnotes. I’ve just watched the latest Webinair (on Sources, Citations and Documentation) and Bruce also provides an explanation on Endnotes along with a discussion on footnotes, short footnotes and the bibliography. I can’t identify exactly where this discussion took place in the video (saw this post after I’d watched the webinair) but it’s well worth watching the whole lot anyway!

2 Likes
7

I would suggest that the “Reuse Endnote Numbers” feature be recoded to the extent necessary so that it functions as well in RM8 as it does in RM7. Also suggesting that this thread be changed from a “question” to a “feature-request”. The suggestion is based on the assumption that a feature-request gets more attention from the RM8 code writers.

8

I just now submitted a separate short message that has just the request for this feature without including all the attendant detail.

1 Like
10

Not to highjack my own thread, but there can be a problem with the idea of always merging identical citations. It has come up in other threads that citations which are downloaded from Ancestry via TreeShare sometimes (or often?) are identical in text even though they are for a totally different image, and hence for a completely different source.

In one sense, the problem is in Ancestry and not in RM because Ancestry has collections that are not indexed in such a way that they can generate unique citations for each image. But in another sense, the problem is also in RM because RM is treating citations as the same for merging purposes even though they are tagged with completely different images and are really completely different sources. It seems to me that RM really must address this problem in some sort of reasonable fashion. And the flip side is that if RM users cannot merge duplicate citations because of the Ancestry problem then the same users will end up with enorously long endnote lists.