What/where is .rmtree-journal?

I am using RM8 with the app and data in Dropbox (not shared). After shutting down and backing up the data file, the next time I start RM8 I get the following message from Dropbox: "Remove “xxxx.rmtree-journal” from everyone’s Dropbox account and all devices? If you move this file to $Deleted, it will no longer be shared with anyone, and won’t be available in Dropbox or on any devices.

I always cancel this pop-up, and never click on Move out of Dropbox; and check a box saying don’t ask this again (but it always does).

I have read in community messages that RM8 does not have a change journal. Also a search of Dropbox and my PC (W11) shows that such a file name does not exist (even searching “hidden” files).

Anyone know what this file is, where it is, and what happens if I delete it?

Don’t edit the RM database live while in the cloud. You risk damaging your database beyond repair. Store the backups there.

2 Likes

Every time the database engine in RM writes to the database file it first writes the values of the affected records to a journal file. If writing to the database encounters a problem that prevents completion, the database engine restores the file to its pre-written state by restoring the values from the journal. The journal file is deleted on successful writing or rollback of the database.

Because the database file is updated atomically (short intervals, speedy, high frequency), so too are the transient journal files. Both the rapidly changing database file and the bursty journal files trigger DropBox to sync them to the cloud. Every sync carries a risk of a clash and even corruption. Working a database file stored in an actively synced Dropbox folder risks even more serious issues than the recurring message you are seeing.

The only cloud sync service that I am reasonably confident can safely accommodate a working RM database file without manually pausing sync is Microsoft OneDrive which appears to autopause syncing of .rmtree and .rmgc files when they are open in the RM app. That said, I do not know if the same protection is afforded the journal files.

Thanks, Tom, for the complete explanation.

I started using Dropbox (as it seems others are as well) when I started using RM7 a few years ago, upon the recommendation of RM at that time as the best way to share the database with other devices. I never encountered those messages or had any issues with the database or apps (Android) until RM8. Something must be different.

Anyway, it looks like I switch to MS One Drive, not a problem for me. But I guess I also give up on having access to the database on my phone and tablet unless RM is ever going to develop an Android app (7x more users than iOS)–no more answering questions about the family on the fly. (An interesting aside to me is that Apple users apparently embrace the detested MS universe and ignore iCloud.)

Oh, I just discovered as I was looking for the best way to move my files to One Drive that RM8 still has an export function to Dropbox (so that the file “can be downloaded by the RootsMagic mobile app”), but not to OneDrive or any other cloud service. As I recall, when I initially started using Dropbox with RM7, that export function created a special app folder in Dropbox , which is still there and in use by RM8.

So, what is the best way to move to One Drive? Can the files merely be copied from Dropbox and then opened using the RM open file process?

Also, perhaps the Export to Dropbox function should be removed from RM8 to avoid continuing confusion. Or is this intended to only store backups that need to be manually refreshed every time the database is updated? At least some additional documentation explaining the whole sharing subject would be helpful. Would save a lot of people a lot of time trying to figure it out and having extended conversations in the community.

The mobile app for iPads and iPhones still works and the Export to Dropbox can still be used to move files to Dropbox for them, they really can’t take it out.

As for moving your file, easiest way would probably be copying and pasting the file to where you want it. Then open the file in Rootsmagic.

Yes, but it is highly recommended by RM personnel that Dropbox not be used for the live database. So why not an export to OneDrive?

Support recommends that you don’t have your live file in ANY cloud storage that may try to sync it while it is open, not just Dropbox. The Export to Dropbox serves a very SPECIFIC function for a very SPECIFIC reason. It is not for general use to move your files to Dropbox so you can work on them. The mobile app required your file to be in Dropbox in order to find it. The export routine made a copy of your database file and placed it directly in Dropbox for you so you didn’t have to manually move it. The copy that was placed in Dropbox is NOT your actual working file.

Personally, my working file has lived in Google Drive for 10 years or so. However I turn off file syncing while I am working on it. When I am done, I turn syncing back on. This takes some degree of discipline to remember to turn it off and back on, however, I have done it so long, it is now second nature.

Thanks, Ken.

That is what I have surmised at this point. Nevertheless, I (and apparently many others) have spent a lot of time unnecessarily researching this question had this information been included in the documentation. Also to be included would be that fact that if only backup/exported files are to be in the cloud, one must remember to re-export every time the file is updated if they want a current version available on a mobile device–and that is no less difficult than to remember to pause syncing every time you edit the file, which during active research would be every day.

And, as to my other point, why is the Android system ignored when it constitutes 87% of the mobile operating systems in the world v. 13% iOS? I know every iPhone user loves it, they are a very small minority of mobile users. Just saying.

I suspect it is because RM did not actually write the mobile versions. I think they contracted them out. Updates to Android appear to have broken things with Dropbox and RM so at this point, the laid Android to rest. I would not expect to see much movement on updated mobile releases until sometime after RM8 gets all if its issues ironed out. Maybe there is no plan to update mobile and when the Apple version breaks, it may well be discontinued too. I don’t really know.

I read this discussion with interest. I am a Mac user so it seems there may an inherent fault with RM8 with respect to cloud storage and Macs. Contrary to a post above I have my Macs connected to iCloud and very happy with this. However this means that all folders are backed up all the time and to switch this off on a Mac deletes your local info for anything not already on the Mac at that time which is a real pain and works system wide! So I won’t be turning on and off syncing. So my question would be does working on a Mac connected to iCloud (most of us I would imagine) endanger our RM database?

Mac users love iCloud for its intended purpose of synching Apple content among apple devices. If you want access to your RM data file, forget the 3rd party iOS RM app and use the Ancestry app. The latter is editable, continually updated and handles multiple ancestry trees easily. Best displayed on an iPad vs a phone.

Bad idea to put your master database in any cloud service for obvious synching reasons as mentioned by others.

Unless it auto-pauses sync while RM has the file open, yes.

After reading all comments and thinking about this more, I am curious about what makes you confident that MS OneDrive pauses before syncing RM. I use OneDrive extensively with many applications and I don’t see any delay in syncing. For example, with MS Excel or Word files in OneDrive with AutoSave set to on, it syncs so fast I don’t see any delay in the sync. Virtually instantaneous. I also see no delay in syncing RM files, in OneDrive or Dropbox. The only cloud drive I see any delay with is Google Drive, which I have not used with RM7 or RM8.

So how do you see or experience a sync delay?

Here’s my first observation:
(#FileSync on #OneDrive looks more promising)
corroborated by @thejerrybryan

Thinking about this myself, I set about an experiment today. I had my RootsMagic 8 database open on my Mac. At the same time I opened the folder containing the rmtree file open on my iPad. It had the time stamp 12:48pm, which is when I last opened the database. I had about 20 names to tidy up, which I did. I noticed that the time stamp on the file on the iPad stayed at 12:48pm until the moment I closed the database on my Mac, then it more or less instantly changed to the current time. I don’t know if this is any help to anyone else, but it looks like iCloud does not sync open files until you close them.

1 Like

My observation is not that OneDrive “pauses” before syncing an RM database. Rather, it is that OneDrive recognizes that an RM file is an SQLite database that is open and therefore does not sync the database until it is closed. That may seem like a distinction without a difference, but see below for more details.

Indeed, this very second I hovered over the OneDrive icon in the Windows system tray and it reported that OneDrive was active and up to date. It definitely was not paused. I immediately clicked rather than hovered on the same OneDrive icon and a OneDrive status window opened. It reported two sync issues with OneDrive where OneDrive was not able to sync a file. One sync issue was that the file jerryrm7.rmgc was in use and could not be synced and the other sync issue was the file jerryrm8.rmtree was in use and could not be synced.

That simply means that my RM7 database is open and that my RM8 database also is open. OneDrive is saying it cannot sync either one of those databases until after I close them. That’s very different than me pausing OneDrive or OneDrive pausing itself. Which is to say that OneDrive continues to sync all other files in my OneDrive folder provided that they are not SQLite databases as are my RM7 and RM8 files.

The same thing happens with any file that is open for output. OneDrive will not try to sync a file while it is open for output. I can’t speak for every piece of software in the world, but many apps such as Microsoft Excel do not keep the current file open for output while you are updating the file. Rather, they open the file for output only during a File=>Save operation. The rest of the time that you are working on the file and updating it, what you are working on and updating is a copy of the file in memory, not the copy of the file on the disk and the file does not remain open continuously for output. And for most files most of the time, the File=>Save operation is extremely brief. So after the File=>Save operation and while you are still working on the file, OneDrive will do a sync. The sync will usually be so fast you never notice it happened. And in any case, there is no conflict with your editing operation in an Excel file and the OneDrive sync operation.

Essentially what OneDrive is doing is treating an SQLite database as if it’s open for output the whole time it’s open. By contrast, Dropbox does not treat an SQLite database as if open for output the whole time it’s open. The result is that Dropbox tries to sync an RM database anytime the database is changed. And an RM database can be changed by trivial user activity that doesn’t really change anything significant. Simply highlighting a different person can be a “change” to an RM database. The result of using Dropbox with an open RM database will be excessive disk I/O, excessive network activity between your computer and the Dropbox servers, and in worse case corruption of your RM database.

I used Dropbox with RM for years safely, but only because I paused Dropbox manually any time I used RM. But I now keep my RM databases in my OneDrive folder without pausing OneDrive and without any problems whatsoever.

I can’t speak to how iCloud, IDrive, Google Drive or any other similar service interacts with RM databases because I don’t use them. They may interact with RM databases more like OneDrive or more like Dropbox. Or there may be some secret option #3 about how they interact with RM databases.

By the way, I don’t think Microsoft’s OneDrive developers are any smarter than are the Dropbox developers. Rather, Microsoft’s Visual Studio product starting using SQLite databases in the Visual Studio 2015 version. I think what happened is that Microsoft discovered a conflict between Visual Studio 2015 and OneDrive and that they fixed the conflict by having OneDrive recognize when SQLite databases are open and not syncing them until they close. I don’t know if OneDrive does anything special for Microsoft Access databases or for any other databases.

I’m a heavy user of Visual Studio. I upgraded from Visual Studio 2010 to Visual Studio 2019 in 2019. At that time, I discovered that I needed to move my Visual Studio projects out of Dropbox and into OneDrive. Visual Studio was even more sensitive to performance issues and data corruption issues from Dropbox than was RM. And I wanted my Visual Studio files synced while I was working so I didn’t want to pause Dropbox. Moving my Visual Studio files to OneDrive was a perfect solution and it also works well for RM files.

It’s pretty clear to me that Dropbox pretty well tries to sync all files continuously all the time. I don’t know for sure but I suspect that Dropbox doesn’t play well with Microsoft Access database files or any other database files. And as I said, I don’t know how the sync products that I have not used handle SQLite databases or any other kinds of databases that are updated continuously rather than going through a File=>Save.

1 Like