I’m working on a (large) family tree on a Windows computer at home and a MacBook in the summer house.
Whenever moving from one of the sites to the other, I copy the *.rmtree to a place in the sky, and take it down and use it. When I come back to the Windows computer, and use the edited file first time, I do an Integrity Test, and always get a long list like this:
Integrity check results:
row 4 missing from index idxCitationName
row 184 missing from index idxCitationName
row 758 missing from index idxCitationName
… … …
I reindex the file and carry on working as if everything is good and well taken care of.
My question is: Do I loose data, or will the re-indexing handle the problems perfectly???
I’m working on a file with 13,000+ people, and cannot check if every data bit is still in its right place.
RM is based on a relational database called SQLite. Relational databases have a feature called collating sequences that are either built-in to the database or which can be added on to a database. RM uses a special collating sequence called RMNOCASE which it adds on to each RM database (my terminology of “adds on” may not be 100% technically accurate, but essentially that’s what happens).
There is some sort of difference in behavior between RMNOCASE on Windows and on a Mac. This issue has been discussed previously at RMNOCASE differences between Mac and Window If Reindexing is the permanent solution to this difference, then the need to Reindex when moving RM databases between Windows and Mac should be well documented. It may be documented somewhere, but I don’t remember seeing it.
RM has stated that the database file is the same in both operating systems. This may not be completely accurate or perhaps the cloud storage step is modifying the file. Seems simpler to just put the database on a flash drive formated as ExFat Master boot record which both Windows and Mac can read. Neither can read drives formatted in the others normal manner.
That is irrelevant. The database file is the same except for differences in table indexes based on the RM proprietary, secret RMNOCASE collation sequence embedded in the application. Either it is different source code between Windows and Apple OS or the development compiles the same source code differently. Regardless, one always has to Rebuild Indexes or Compact Database on receiving a database file from the different OS.
I would suspect that it’s the latter, although I’m not sure the distinction is very meaningful.
As an example, I’m very familiar with C++ libraries where the same code can be compiled for a UNIX vs. a Windows vs. a Mac environment. The libraries use standard C++ features such as the #IF directive. These kinds of directives are common in programming libraries, even when the code is not actually in C++. As a result, code can compile as if there were different source code for different run time environments even though the different source code is actually stored in the same source code files. The #IF directives are usually so embedded in the programming libraries that the distinctions in the source code are usually transparent to the developers using the libraries except possibly for an initial setting of the #IF directives.
Wow! That is a terrible problem for anyone with a foot in both operating systems for some reason. RM tech support has said that the database file was the same for both systems. If correct that should be highlighted in the wiki and manual.
I profess/confess that I’m not a programmer and therefore I do not know the ins and outs of collations and string comparisons.
Is it not possible for a collation sequence in SQLite to be identical and the results of comparisons internal to SQLite to be identical regardless of the operating system?
Why would OS functions even come into play? If SQLite source is C, shouldn’t the compiled code produce the same result regardless of OS?
Is there any reason for that to be different if the collation sequence is a custom one loaded as an extension or included in a custom compilation of SQLite versus the ones built into the pre-compiled binaries for MacOS and Windows?
I’m not a programmer either. My apologies. I did not revisit the forum after posting. It’s obvious I inadvertently quoted the wrong half of your postulation, then pointed out the machine language variance generated from Delphi (effectively Pascal-ish) source code, depending on whether native to Windows or cross-compiled for Mac OS (Unix) & Android.
Aside from default and supported Locale (language) and character encoding conventions as input, I don’t know whether the operating system architecture differences accounted for in their respective compiled SQLite library files (.DLL for Windows versus .SO or .DYLIB for Mac OS), have capabilities that affect actual database file contents. As I understand it, Collation, in and of itself, is for external result-set data manipulation/display.
I can’t say.
Surely, the functions in an Apple-provided compilation of SQLite library files would depend on toolchain, compile-time parameters and the resultant code optimizations for OS version, architecture & system features, as far as any potential for some slight differences. Complicated stuff requires much deeper knowledge than I possess, for sure.
Not sure. Is there an equivalent Mac OS construct equivalent or similar to unifuzz.dll?
The author does have to get around the Windows function you cited that he ran into in the C source. And he hasn’t published an update supporting his goal of broad compatibility.
I want to again raise the question of what will RM inc do about this?
Could RM inc please document the RMNOCASE collation so that is can be recreated for all of us SQLite hackers?
My guess is that RM’s implementation of RMNOCASE is calling an OS function which gives slightly different results on Windows and MacOS. I think that was the point of the referenced GitHub code: unifuzz by MooreDan. That code takes the source code (from opensource Linux wine) that does the actual character comparison (that seems different in Mac and Win).
Apparently, the tables needed to do the compare are larger that SQLite itself, so that’s why SQLite decided not to ship then by default.