Macbook Air M2 24GB ram 512 SSD takes 15 seconds to open RM10 with a 400 person database which is shocking considering that my M2 chip is 10+X faster than Intel. FTM takes 2 seconds and a 1.3GB scanner program 5 seconds.
Time machine currently auto backups approximately every 60-90 minutes waiting until enough changes have occurred. The first backup is everything with subsequent ones just being any new or changed files. You can force an immediate backup if desired. Over time backups are culled to one a week. TM is very useful and easy to browse for missing items etc. However it is at physical risk for drive failure, flood, fire and theft unlike a cloud backup.
It happened again. When I initially reported this problem I was unaware of the need to click the āokayā or āupdateā box when altering a fact, so I assumed my problems were related to that. However what Iām reporting now has nothing to do with that. I was using the FamilySearch/RootsMagic interface. I copied seven children to a mother and father already in my RM file. I rearranged them by birthday, and I did a backup. I then went to Ancestry TreeShare and they didnāt show up as requiring export to Ancestry. This surprised me so I went back in RM to the parents to whom I had copied these children, and the seven children were gone!
I re-copied them from FS, rearranged them by birthdate, performed an update, then went to TreeShare again. This time I copied only the first of the seven, then checked my famly in RM, and all seven children were still there, so I went back to TreeShare and copied the other six children. This time none of them disappeared from RM.
By the way, I saw in TreeShare something Iāve seen before, which is more a curiosity of how TS works. It showed one of the parents, and basically asked me to agree to copying the last three of the seven children. I had already copied the first of the seven, so I expected to have to okay migration of the other six, but it only asked about three of the six. I canāt explain that, but I think Iāve seen this before.
The important thing is that the seven children disappeared on the RM side and I only discovered it because TreeShare showed nothing to copy to Ancestry. On the second time through it was okay.
Were you adding the children under Family Members and actually saw each one transfer to the RM side afterwards.
What you are saying happened without closing RM seems impossible. I donāt know what is going on that would be overwriting your database. Things just donāt disappear like that.
Yes, I was adding them from the list of children beneath the parent and spouse, and theyād appear individually on the left side.
I believe this is a difficult-to-recreate phenomenon but I think itās happened to me at least three times. Do the coders of RM see the reports in this forum? I know when I did IBM mainframe systems work I appreciated being informed of difficult problems because I could let my brain ponder them.
As Renee said, it seems pretty impossible if the above is really the sequence You performed. Pretty hard for the programmers to āponderā past that.
My assumption is that thereās a problem when getting data from the Family Search connection. In some instances it appears that the data isnāt imported to RM even though it displayed on the screen. I would like a report of this to get to the programmers.
Then I would suggest sending an email to support@rootsmagic.com and in this email, diescribe what you see as the issue, give any steps to recreate it and attach a backup of the file you are using (without media). You will want to give enough detail that they can at least recreate the issue because if they canāt, it is a real beast to find some of those problems.
I think the newly found bug caused by RM 10ās Relationship Chart report is indeed responsible for most, if not all, of the unexplained losses you were plagued with and it had nothing to do with FamilySearch or TreeShare or cloud-synchingā¦ It is an unexpected quirk caused by the programmer switching SQLite away from atomic transactions that update the database disk file on the fly to a batch process for which transactions are accumulated in-memory but have to be committed to the drive en masse by an explicit command. The report generator seems to start the batch mode but fails to complete it and so new data happily builds up in the in-memory database without being written to disk and is lost on closing the database. It is also lost after running the Chart with some other operations such as Compact and maybe, from your description, by TreeShare. Hopefully, the coming update will enable you to gain confidence and Iām sure you will tread carefully.
Sorry for all the attention I gave to OneDrive as the possible cause.
Thank you TomH. My first report of this was on July 4. If knowledgeable programmers took my report seriously they might have asked themselves whether thereās a process which creates a temporary database which needs to be updated to the permanent database. Whenever I dealt with the so-called experts who evaluate problem reports I could tell they didnāt believe me when I described what I saw. I was afraid they were never informing Bruce or others of the reported problems. Thatās why I sent a physical letter to Bruce. Not sure if he ever saw the letter. I basically gave up on RM. I hated that I had to do that, but nothing I experienced gave me confidence that my reports were being given serious consideration.
Sadly, I think nearly everybody misdiagnosed this problem, both on the RM staff and in the user community. I certainly did. The problem gave every appearance of being one of two things. One main suspect was a user error, where for example the user had multiple databases and had entered the data into the wrong database. The other main suspect was that the userās database was located in a folder that was being synced with a cloud sync service, or even being synced to multiple machines or multiple disks on a home network. I know it took a very long time before I personally was convinced that the problem was a bug in RM rather than a user error.
In retrospect, your note from July 5th included all the information needed to replicate and diagnose the problem. Itās just that RMās code normally updates the RM database in what is called auto-commit mode, and auto-commit mode is virtually foolproof. Auto-commit mode truly saves as you go and it requires no further attention by the programmer after a transaction to update the database is initiated. Just to be clear, this auto-commit mode thing is something the programmer deals with. It has nothing to do with any user options or anything the user does or does not do.
It had never occurred to me that RMās code to produce Relationship Charts would update the database. After all, itās only producing a report. And it certainly had not occurred to me that RMās code to produce Relationship Charts would turn off the RM databaseās auto-commit mode and then fail to commit the userās changes in manual mode. I think that everybody that works with RM in any capacity just sort of expected RM to save as you go. So hopefully, we have a lesson learned.
I have no idea if RMās fix is going to be to make sure that auto-commit mode always stays on, or if it is going to be to be sure that changes are committed properly when auto-commit mode needs to be turned off temporarily. But in any case, I hope that they do a thorough search of the code to be sure that there are not any other places in the code which have the same problem.
RM10 released 6/19/2024.
I started using RM10 on 6/23/2024.
I reported the problem on 7/4/2024.
On July 5 I wrote āMy initial assumption was that RM was adding the new people into a copy of the DB that it had in memory, and that somehow that never got written to the hard drive.ā
On July 24 another user reported in this issue that he was having the same problem.
On August 7 rzamor1 wrote āItās not a RM10 issue because we would have multiple reports if that was the case.ā
I sent a letter to Bruce on November 11, 2024.
I would not be surprised if a manual-start for a bunch of transactions does not exist somewhere else in the system. But we have seen only rarely where someone has reported a āCannot start transaction while another is in progressā error message. That could occur if someone ran such a procedure after Relationship Chart failed to commit its set.
Years ago, we showed how some outboard SQLite procedures that mimicked RM procedures outran them by an order or two of magnitude by using manually started batch transactions and encouraged the developers to apply the technique. However, I, too, wonder why it might be wanted in this chart process. Especially as the timing of the appearance of the journal file comes when the report is closed, potentially after the chart is generated. That would suggest it was triggered when RM saves the report settings for recall the next time it is opened. There would be no need for a manually started batch for such a trivial update.
A 2nd local drive backup is a good idea but consider adding an iDrive cloud backup. for $10/yr you get automatic cloud backup both of incremental file changes and once/day the whole backup set. No physical risks.
There are certainly situations where performance can be much better if a particular set of operations is batched into a single transaction with a manual BEGIN and COMMIT rather than using auto-commit. So Iām not suggesting that RM should always depend on auto-commit. Itās just that with an bug such as this, I think itās a best practice to check to see the same bug might exist elsewhere in the code.
It seems likely that RM is relying mostly on auto-commit. If that were not the case, then users might have been bombarded with āCannot start transaction while another is in progressā messages after creating their Relationship Charts. I do not remember seeing any reports of such messages in the threads concerning the lost data problem. Thatās pretty strong evidence that RM is relying mostly on auto-commit.