Excessive time for GEDCOM export

While I’m not on the latest version of RM 10, the reason is I’ve been trying to export my whole database to a GEDCOM since 10:41 this morning. While I do have 25,000+ people, 6 hours seems pretty excessive for an export, particularly considering the size of the file (407.3 MB) is less than half the size if the export I did in January (980.1 MB). That indicates I’m looking at maybe 6 more hours until it’s done.

Maybe I’m being overly demanding, but 12 hours to create a GEDCOM seems a bit harsh.

Doesn’t matter what version you are on, 6 hours is too excessive and something is wrong…

I just made a gedcom of a file with abt 12,500 people in it-- took abt a minute but mine is 12.8 MB-- so do you have a massive amount of media attached to this file? If so, depending on what you are going to use the gedcom for, perhaps try making it without the media

That does seem excessive. Have you checked what is happening with the process in task manager. Is it actually still running?

Also the obvious questions. How long has it been since you ran the database tools to rebuild the indices and compact the database?

There’s quite a bit of media as much of the facts are from Ancestry birth and death notices.

As far as database tools I run them every time I get some type of RM error, which is every other day.

Other things to look at which have caught me up in the past include:

  • the disk space available.
  • are you writing to a local drive?
  • a cloud location?
  • a USB thumb drive?
  • are you running any other programs which consume large chunks of memory or other resources?
  • security suites/virus scanners?

Other than a link to each media file, the number of media has no impact on the size of the GEDCOM. Each link is likely only 100 or so characters so the overall impact is small.

Will it let you do a drag n drop still?

I have a 32GB Macbook Pro M2 with 700GB available on a 2TB SSD. Nothing is running except for RM. At 7:35 PM it’s up to 466MB written. CPU, Memory, and disk space aren’t the issue.

Being a technologst since the 70’s most of the current virus suites are a scam so I don’t use any. I can see that RM is using 100% of one CPU, so it’s not multitasking.

Not sure, still hung on the export.

In the three hours between your two posts reporting the size of the GEDCOM file it has grown only by 60MB. At 100% CPU all this time, there is something seriously wrong. I’d kill the process, reboot, run tools. Try drag’n’drop everyone as suggested by Renee but that, too, is a GEDCOM export-import so it could get mired the same way. If it doesn’t, maybe the export from the original file will work this time, too. If that export bogs down, then you will have to switch to your stripped copy (drag’n’drop is lossy).

There are some changes to the drag n drop in RM10. It copies the data directly over, vs exporting and importing.

2 Likes

That’s good news. Now we should see lossless or less lossy transfers.

So Books aren’t lost?

I was able to save the Books using SQLite when I hit the 32768 bug on places. That was a few months ago so a drag n drop “worked” then. I had to do something because the entire things I’m doing now is using Publications so losing that makes using RM kinda worthless.

At an FYI it’s still running, up to 553MB. I need to let it finish because I don’t want to chance losing another day on the export.

You have problems within your database if this is happening. I’d open trouble ticket with support and send them the database.

Unless you can reproduce the error consistently it’s a waste of everyone’s time to open a ticket. I’ve been getting random errors in RM since day one years ago on version 7, most frequently when using TreeShare or FamilySearch. If I can reproduce it I open a ticket with the steps and send them a copy of the database, otherwise I just run DB Tools and move on.

Some more info, I’ll need to submit a ticket as this is an obvious RM Mac OS specific bug. I suspect it’s related to the Ancestry download issue, and specific to the SQLite version on Mac OS.

What I’ve found:

a) RM 9 Database on Mac OS 14.5 is 972.6 MB. GEDCOM successfully exported on January 26, 2024 was 980.1 MB. Export started yesterday ran over 24 hours and only got to 853.1 MB before I killed it.
b) RM 10 database on Mac OS 14.5 is 971.2 MB, GEDCOM export is going at about 50MB an hour. There is no export progress bar on RM 9.1.6 but on RM 10 one popped up after abput an hour with 510 people displayed . DB Tools ran successfully before (on RM 9) and after conversion.
c) RM 10 database on Windows 11 created from transferred RM 9.1.6 database is 948 MB. After conversion DB tools ran successfully. Export progress bar is showing, but not refreshing. No idea how far it’s gotten in the last hour.
d) Looking back through my database backups I see the backup size doubled between 5/14 and 5/16, between what I assume was the 9.1.4 to 9.1.5 update. RM 9.1.3 would have been the version that performed the last successful full export in January.
My observation is this process is not using any threading at all as on both Mac OS and Windows it takes quite some time to bring the RM window back, and would explain the lack of refresh on the progress window.

Renee, I copied the database to another Mac and attempted to drag n drop to an empty database. It’s been running since yesterday and only through a a few thousand people. I’m going to try it on a WIndows machine now as I’ve given up waiting on the GEDCOM export on that machine.

Found some interesting stats trying to research this problem.

Restored database from 11/23:

Sources 3196
Citations. 30078
Media Items 32290
Media Links 30944

After Drag N Drop and update via GEDCOM to bring current:

Sources 3196
Citations. 115312
Media Items 16708
Media Links 1952983

After conversion to RM10 and Drag N Drop to an empty DB that took two days:

Sources 3339
Citations. 114514
Media Items 16278
Media Links 1933892

Notice the huge increase in Citations and Media Links. Where they came from is completely unknown at this point.

The huge increase in citations and medialinks in the RM9 drag’n’drop (a GEDCOM export-import) is because it unmerges formerly merged duplicate citations. Every use of a citation having a medialink becomes a duplicate citation with a new medialink.

With RM10, that may not happen for drag’n’drop because it is a more direct database to database procedure but will continue for Export in order to comply with GEDCOM.