I just upgraded to RM10 today, importing one of my trees (about 14,000 individuals). It is taking forever (4 hours and counting), mostly because of frequent ‘Connection Errors’. It seems it will finish (now 60% complete with Downloading media items), but very annoying because every time the error happens it grabs focus away from anything else I’m doing on the computer (including typing this).
Is there anything to be done? I have a couple of other trees of similar size to import. I have not tried RM9 (still installed) but I don’t remember it having such a problem.
Perhaps that is the case, but at any rate, it finally imported to RM.
User performance with browser should be mostly isolated from API access performance. FTM rarely has issues updating and downloading trees.
Web performance has been bad for at least a year or two, and it has nothing to do with holiday traffic. I have to continually refresh, try again. For the last week, I often have to click Ignore button multiple times for a tree hint before it works.
Something else must be going on with your computer as I have been downloading/uploading 170,000 with no issues (45min or so) multiple times in the past 2 weeks.
Are you on Windows or Mac? I’m on Mac. There’s nothing wrong with it.
I think I had similar issue with a previous version of RM and it had to do with too many requests being made, causing the service to bail out. I had to throttle the connection (with proxy?) to avoid this.
But I’m not planning to be downloading trees that often, so as long as it eventually works, I’m not interested in getting to root cause or trying to rediscover the workaround I used.
Since I use both Windows and Mac, but mainly Mac the performance issue with Ancestry isn’t the network or the OS, but the SQLIte used on the Mac side. Ancestry processing results in a lot of SQLite activity, including a secondary database in %userprofile%/RootsMagic/AMT/DT_%.DT8.
I don’t understand why you think the performance issue is the SQLite used on the Mac side. RM on Windows does the same thing and FTM2019 databases are encrypted SQLite so there must be SQLite procedures involved in its interface to the Ancestry API.
Because I’ve traced the performance issue to the SQLite database access. The issue isn’t SQLIte itself, but the RM integration with it. I suspect it’s the use of RMNOCASE and the specific implementation of that on the Mac, but that’s just a suspicion.
I’ve proven that the /AMT/DT_%.DT8 file which is only used for Ancestry is part of the problem. I’ve taken the same database, roughly 25,000 people, attempted to upload from a MacStudio with 64GB memory and a 2TB SSD and it times out, leaving a partially created *.DT8 file. Using a GeekOM Intel box with 32GB and 1TB SSD it upload in about 30 minutes. Same network, same switch, so it’s obviously not the network.
Then, you take the *.DT8 file from the windows machine along with database and move it to the MacStudio and Ancestry works just fine. Rename the *.DT8 file and Ancestry stops working on the Mac, do the same on the GeekOM and it recreates the file.
Pretty much proof positive the problem lies in the database maintenance specific to Ancestry on the Mac.
OK, not SQLite itself but the RM program and MacOS and their interaction between the network and SQLite. The partially completed DT8 file is just a symptom and evidence of the point at which the process broke down.
I know it’s not an Apple Silicon vs Intel issue because I’ve reproduced the problem on Intel Macs also. I know it’s specific to the *.DT8 file because you can have Treeshare working on a machine just fine, rename the *.DT8 file and it breaks, delete the corrupted *.DT8 file, rename the old one, and Valla! it works again.
That is likely the WAL (write ahead log) file you are renaming/deleting and it contains a session of numerous operations to be carried out “subsequently” upon the “associated” SQLite database (to catch back up) …both, when the CPU and network operations are least busy and upon termination of the processes … to finalize writing all changes to the “associated” database. The reason it works when transferring the .DT8 and the .RMTREE to the Mac is because, immediately upon reopening a SQLite database, there’s a check for that WAL file to begin/resume incorporating the remaining data into the database to maintain ACID (Atomic, Consistent, Isolated, Durable) compliance claimed by SQLite.
Possibly, but the fact is that file (actually just another SQLite DB) is managed better/faster on Windows than MacOS. If you delete/rename on Windows it’s recreated without issue, not true on MacOS.
If you have a large number of people in the tree then whatever inefficiency is there causes a timeout on the Ancestry connection. One the database is built that problem doesn’t exist.
There are six tables in the DT8 database, CitationTable, ConfigTable, MediaTable, PersonTable, and SourceTable. These have the same row counts as their corresponding table in the RMTREE database, with a different structure. All except ConfigTable have ID, Hash, Deleted, ModifyDate and Details. PersonTable also has Given, Surname, BirthDate, DeathDate. ConfigTable only has one row, the last date/time of TreeShare.
Clearly it’s a mapping database to track changes, but doesn’t appear to have any content other than identifiers.