GEDCOM from FTM to RM9

RootsMagic
1

This post is a summary description of those TAG elements of GEDCOM and corresponding data within an FTM created GEDCOM file that have been discovered to be rejected or misinterpreted following import of the file into RM9. My test database has 3000 individuals and 850 families.

I hope this information is useful and hope it can be added to with the experience of others. I do no claim it is exhaustive, bound to have omissions and errors.

With reference to GEDCOM Standard, I refer to version is v5.5.1 published in 2019.

A. Source of GEDCOM file: FTM 2019 implemented on Mac.

  1. Generated (exported) file is in standard GEDCOM v5.5.1 lineage linked format.
  2. Includes links to media files.
  3. Living people are not privatized.
  4. Includes privatized facts and notes.
  5. Places are geocoded.
  6. Includes citations, sources, repositories, notes.

The following FTM GEDCOM Standard compliance discrepancies have been discovered within generated file that might cause rejections (and data loss) on RM9 import:

Custom TAGS
FSID - Data payload is Family Search identifier for an individual within INDI record.
_INIT - LDS initiatory ordinance individual event structure within INDI record.
_LINK - Data payload is URL for web link within citation SOUR structure.
_MILT - Military service individual event structure within INDI record.
_FREL -Subordinate within child CHLD of family FAM record, payload is relationship to father.
_MREL - Subordinate within child CHLD of family FAM record, payload is relationship to mother.
_DATE
_PRIV
_FOOT

Use of EVEN for individual custom attributes instead of FACT tag within INDI record.

Use of EVEN for individual nickname instead of NICK within formal name piece structure. E.g. 1 EVEN boris / 2 TYPE Nickname.

Overloaded use of PUBL publication data payload within source SOUR record. Multiple tag:data pairs possible separated by semi-colon. E.g. PUBL Name: ; Location: ; Date: .

Notes With Tables - Tables within notes are replaced with plain text, single row per table cell on export.

Research Notes Flagged as Private - FTM individual research notes are identified as private with use of _PRIV tag in note record.

B. Target of GEDCOM file: RM9 implemented on Mac.

  1. Have chosen to import raw (unmodified) GEDCOM file produced by FTM.
  2. Have chosen not to add an additional source for imported data for import.
  3. A new family database is created within RM9 with imported data.
  4. RM9 produced a corresponding .LST file which is a log of exceptions raised on import of file.
  5. An inspection of the .LST file and the subsequently created database via RM9 is basis for identifying potential data import issues such as data loss and misinterpretation.

Findings from analysis of .LST file:

  • Geocoding of places with GEDCOM Standard MAP and subordinate LATI and LONG tags for PLAC identified as “Unknown info (line n…)” are rejected.
  • Custom FTM tag FSID for individual family search identifier identified as “Unknown info (line n…)” is rejected.
  • Standard tag RESN restriction notice identified as “Unknown info (line n…)” is rejected.
  • Standard tag FORM within multimedia OBJE record identified as “Unknown info (line n…)” is rejected. Note that import file had this as “2 FORM xxx” at correct hierarchical level definition as per GEDCOM v5.5.1 standard.
  • Custom FTM tag _DATE associated with OBJE record identified as “Unknown info (line n…)” is rejected.

Findings from analysis of new database via RM9:

  • There are many examples of duplicate citations in the created database. This is due to fact that in original GEDCOM file, the same citation is duplicated if it had been linked to multiple facts within FTM, e.g. an individual baptism and birth. Additionally, there are duplicate links between citations and a referenced multimedia image file. Import logic of RM9 does not check for duplicate citations.
  • [Separate issue] After perusing images, RM9 aborts with no warning message. Repeatable.
  • GEDCOM file with use of EVEN for individual custom attributes (instead of FACT tag) interpreted as facts and added to individual detail information.
  • Per previous point, use of EVEN for individual nickname has meant creation of a fact for Nickname appearing in Person Fact list yet when looking at Person Name window Nickname is empty.
  • Places window shows empty longitude and latitude as a consequence of import rejection.
  • No family search identifier shown with person facts understandably so given a consequence of import rejection.
  • All sources imported into Source Type: Free Form. As a consequence, publisher information in brackets is appended to Source Name in Footnote and Bibliography. The tag:data pairs of publisher information are left in tact.
  • Military fact corresponding to FTM custom tag _MILT appears to have been correctly interpreted and a corresponding fact with associated information was added to corresponding individuals.
  • There are examples of Research Note for a Citation for a fact being truncated as Edit Note display does not show full text as per original GEDCOM file. In FTM this information was known as Citation Detail which was translated into PAGE tag and payload for a citation. (Is there meant to be a size limit on PAGE payload length? Not specified in GEDCOM Standard.)
  • LDS Initiatory Ordinance data corresponding to FTM custom tag _INIT appears to have been correctly interpreted and a corresponding fact with associated information was added to corresponding individuals.
1 Like
2

Mac FTM gedcom to RM9 and back to FTM via gedcom was very messed up clearly demonstrating that FTM to RM9 is a one way trip with serious data mangling.

I am staying with FTM but watching for a stable useable RM9 mac version (9.5?) down the road. Everything I do is just simpler and easier in FTM which does not crash or require many clicks down/up.

3

Reality is we have both reached the same pragmatic outcome. For others, some have reached similar conclusions and are staying with RM7. Newbies are still finding their way, possibly struggling. So, upon reflection I do wonder why I’ve started to contribute to this forum, knowing what I know and using another tool for my core efforts. Do you ponder the same? I do use multiple tools and think it critical that I be able to move my data between them with minimal / no data loss. I deliberately work to avoid vendor tool lock-in - and that is not easy. I share information on this forum hopefully to make a positive contribution for making better informed use of RM9+ and perhaps developers will take notice and improve things - lots of great feedback and ideas in this forum from others for them to harvest.

4

The big problem is that it really has never been fully possible to avoid possible loss when moving between tools. Way back in the early days, GEDCOM was supposed to make this possible until the week after it was released and some genius software developer decided to come up with some new and novel was of stretching the standard. Then the next developer decider to stretch it a different direction, then presto! Possible losses galore.

5

Of course you are absolutely correct. A compound effect of developers going in different directions, let alone stagnation of standards and no formality to compliance governance. Recent efforts with renewed GEDCOM 7 are still early days/years. As a consequence I have had to invest in building my own tools to support / enable fulfil objectives - it took some time, but have developed my own GEDCOM parser import/export. Not the typical path for everyday tool users. However, I look forward / hope for a time when I can abandon that line of attack.

6

This forum has some very helpful and knowledgeable contributors (ie JerryBryan) that pass on information germane to other software. You just have to ignore the nasty ones and those deeply into writing SQL tools. Rootsmagic is the only genealogy program with such users.

7

Actually it isn’t. There is at least one other software package where several users are heavily into writing plugins to add functionality (might even be two). Fortunately, it doesn’t do a native Mac version.

8

@Rooty @kfunk

I recall reading a review of latest tools available for genealogy. In the intro, the author mentioned he was tracking over 1100 tools in the market place. That is just one indicator of how much things have changed over past decades and support of growth in popularity of our hobby. As to SQL tools, there is a place for them especially when an apps underpinning database is “visible” and “accessible” with their use. Hence a reason why RM was / is popular. But the new UI … jury is still out I think. Appreciate your feedback, thank you.

9

First, this is an amazing amount of work on a tedious topic!

One point: There is a size limit on PAGE payload length. PAGE payload is defined as <WHERE_WITHIN_SOURCE> which has the size specification {Size=1:248}.

1 Like
10

Thank you. And great find on what I should have been well aware of in the GEDCOM 5.5.1 standard for PAGE payload size. In v7 of GEDCOM there appears to be no size limit which is where I got muddled.

11

Yes, impressive analysis. I experienced the truncation of overly verbose Ancestry citations with its GEDCOM export in the RM5 era and reported it to Ancestry.com, to no effect.
Ancestry.com and RootsMagic 5 #ancestrycom #sources #update #gedcom – SQLite Tools for RootsMagic

12

Had a look at SQLite Tools info re. RM5 era etc. Impressive efforts, and yes verbose citations from Ancestry (and FamilySearch) are ugly.

13

Another benefit of using a mac where users expect software to just work as advertised and required by the operating system.

14

There is no benefit to using a Mac outside of graphics…and obviously the operating system doesn’t require software to behave in a certain otherwise RM8 and RM9 would not run, and we know how many Mac compliance issues that they have.

15

The Mac vs Windows tete a tete is really a distraction. You could lump in Linux varieties, Raspberry Pi too if you want to be more inclusive of cross platform considerations.

Things like: ease of use, accessibility, streamlined workflow, navigability, understandability, data relevance etc. that fit within the discipline of user experience design (besides reliability, accuracy, integrity, and other “ilities”) is where a good discussion is needed. The decisions made for user experience design is at the heart of my criticism for the new RM8/9. E.g. why don’t we have menus? Alignment to general accepted guidelines for cross platform UI could address a lot of issues.

Good programming does not mean good user experience design. The two are separate disciplines.

1 Like
16

I have been griping about the sorry excuse for a UI since the earliest preview videos for RM8
I believe I went so far as to call it cartoonist. There are several handfuls of cross platform frameworks out there which would allow not only Mac/Win development, but Android and iOS development from the same codebase too. However none of them must be able to work with Delphi, which is what RM is developed in.

17

Delphi? Wow, that is a blast from Borland past, and under different ownership now I assume. So, I’d speculate then that a dev decision was made to use a bespoke Delphi UI framework as a perceived shortcut for enabling prioritised cross platform (Windows / Mac) implementation? If that is correct, I can then understand why the app has a constrained user experience, programming logic being retrofitted into that framework, and reliance on framework supplier to fix performance, bugs and deliver missing capability (e.g. keyboard support) in that framework. Judging by delays in bringing v8/9 market, that’s got have bitten in the proverbial rear end. If any of preceding is accurate, then I do wonder of about future.

18

image

19

A voice from the dark ages? For average users the mac operating system and integrated hardware offer many benefits. The surprising thing to me is that RM8 and 9 run at all despite violating so many OS conventions.

20

I’m uncertain what that means but we believe that all versions of the RootsMagic app after RM3 have been developed on versions of Embarcadero Delphi. Somewhere, I’ve seen a post describing the decision to switch from C++ to Delphi but it does not come up in Google. Here’s a blog post that describes what they went through for RM4. I daresay they encountered much the same sort of problem with the libraries they used in RM4-RM7 when going cross-platform in RM8/9.

1 Like