I am creating a group for people who are counted on the various censuses, but I’m finding anomalies when I use the “OR” condition.
I create criteria that goes:
Census date equals 1880
I get 72 people.
I create criteria that goes:
Census (family) date equals 1880
I get 90 people.
I combine them, I get 172
So far, so good.
I create criteria that goes:
Census date equals 1880
OR
Census (family) date equals 1880
I get 124 people. It misses 48 people.
I did the same test with the 1870 census, and it selected people who were not even alive in 1870, much less on the census.
There is a flaw somewhere in the logic.
Using two conditions and the “OR” conjunction worked in RMG9, but is having a problem in 10.
I agree that there is or are problems. While I have not corroborated your OR concern, I have encountered an error with the Census (fam) criterion that I suspect is a programming error in Groups. The error I suspect is that it has confused the FamlyID (MRIN) with the PersonID (RIN). So a Census (family) event is ‘owned’ by a FamilyID with which the PersonID has been directly matched when it should have been matched to the FatherID in the couple having that FamilyID…
In my case, the one instance of the Census (fam) event containing 1871 in the date put one person in the group who was born much later and has no Census fact. The Fact List report of people having the Census (fam) report, correctly lists the husband of the one couple having the fact. (Whether they should also list the wife as having the event is a quite different matter which adds more complications and then there is the issue of shared events!)
Well, this type of thing can get messy when you have multiple Primary keys with different owner types (Family, Individual, Media) You are going to have same ID# in more than one table. Never was a big fan of the design of some of those tables are it can get confusing (behind scenes there are multiple joins needing inner, outer, left etc joins in diff combinations
Indeed, that is my case, having inspected the FamilyTable with sqlite3. The RIN of the wrong person listed in the group is 41. The FamilyID (or MRIN) of 41 contains the FatherID=97 and MotherID=92 whose respective Edit Person windows show the Census (fam) event.
I checked the Marriage events, the likely most commonly used family-type fact, and it does not have this problem. It is most likely confined to Census (fam) assuming the closeness of its name to the individual Census fact type could be at the root of the error.
It is possible that the misses are entirely due to the Census (fam) bug I’ve identified in Group criteria, not the OR operation. First of all, any people in your database having both a Census date of 1871 AND a Census (fam) date of 1871 will count as only 1 under the OR logic. Because the criterion for Census (fam) points to the wrong person, it may be pointing to some (possibly as many as 48) who satisfy the Census criterion and are thus counted once.
Hopefully you are aware that RM’s “family” facts apply only to the couple and not to the children. Think of the Census (fam) fact as working just like the Marriage fact - only for the couple. For this reason, many or most RM users do not use the Census (fam) fact.
That being said, if there is a bug in the logic searching for Census (fam), then the bug would also be there searching for Marriage. For example, what if you searched for Birth>Date>Before>1880 OR Marriage>Date>Before>1880. I don’t know what would happen, but I would expect it to work just like it says. Namely, it should such for individuals who have a Birth fact before 1880 and it also should search for individuals how have a Marriage fact before 1880. And in searching for Marriage facts, it will need to search for couples with a Marriage fact before 1880 and then return both individuals. It can’t really return the couple as a unit.
Well, I just now ran the Birth>Date>Before>1880 OR Marriage>Date>Before>1880 search. I’m probably missing something obvious, but on it’s face it seems to have returned the expected people.
I tested with Birth>Date>Equals>1900 OR Death>Date>Equals>1880 It’s a strange looking query, but it returns the expected people.
I tested with Birth>Date>Equals>1900 OR Marriage>Date>Equals>1880 It’s an equally strange looking query, but it should return a population of people matching the query. But it returns all kinds of strange people - like people with no Birth Fact or Marriage fact at all.
I tested with Marriage>Date>Equals>1880 It’s a perfectly reasonable looking query, and it seems to return all the people that had a Marriage fact in 1880 - both spouses.
So in my case, I seem to need both the OR condition and a couple fact to see the problem.
As I reported in my post preceding the one to which you had replied, I checked the Marriage fact type to see if it applied to all or more family-type facts and found what you found. The bug is not affecting all family-type facts; the only other I have checked is Census (fam) which we know has this particular bug. I suspect it is the only one but that has not been confirmed by anyone.
I have not found the problem anywhere except when doing Census records. If I do a simple query of “Census/Date/Equals/1880”, I get the expected results. If I do a simple query of “Census (Family)/Date/Equals/1880”, I ALSO get the expected results. When I try the more complex query of “Census/Date/Equals/1880” OR “Census (Family)/Date/Equals/1880”, I get errors, either a large group missing, or a number of extras that do not belong.
For the record, I have about five people in my database who have multiple Census records for the same year, and only one who has a single and family record in the same year, and I have accounted for those exceptions.
I get the right count when I do the same complex query in RMG 9, so the problem is new to V10, and related to doing a complex query.
I would also note that the complex query involves two different fields (Census and Census(Family)) having the same value.
On further examination and reinstalling RM10 on my notebook plus upgrading to RM10 on my laptop, checking and double-checking, I think I can make these observations:
There is a bug related to the Boolean operators OR and AND in Person Search - Advanced and in Groups when using “Criteria - select or clear people based on field values”.
It affects family-type events such as Marriage, Census (fam), Residence (fam), Divorce…
The error is that it returns individuals whose Record Number (RIN) corresponds to the Marriage RIN visible only in the Marriage List report with the corresponding option enabled. Note that MRIN is a misnomer for the record number of a table in the database that identifies the couple having a family-type event or are the parents of another person.
The error arises when a family-type event follows the Boolean operator, not when it precedes the operator…
My initial reaction is that therefore this should be a trivial bug to fix. But it could be a little tricky. The query for Advanced Search has to link directly to PersonTable/NameTable for individual facts and to FamilyTable for family facts, and from there link to both spouses. And a single set of Advanced Search criteria could certainly include both individual facts and family facts. I’m confident the developers’ SQL skills vastly exceed mine and that they will get this sorted out right away.
I’m only guessing as usual about RM internals, but my sense is that RM7 did advanced search queries by going through PersonTable one person at a time and querying the criteria just for that person. That’s why it was a little slow but not painfully slow, and the progress indicator worked pretty well.
I can’t even begin to guess about how the Advanced Search worked internally in the beginnings of RM8. It was terribly slow - much slower than RM7. But it was optimized starting with 9.1.1 and has been much, much, much faster ever since. I would even go so far as to suggest that for many or most Advanced Searches that it’s a good bit faster now than it was in RM7.
So here is my guess. My guess is that’s the optimized search is not going through the the PersonTable one person at a time. Rather, I suspect that it’s constructing a single SQLite query corresponding to the user’s criteria and just executing that one query. The progress indicator no longer seems to work or to work very well, which could be a clue that it’s now constructing and executing a single query to do all the work. Well, a single query that’s all for individual facts is fairly easy. And a single query that’s all for family facts is fairly easy. But a single SQLite query that has to incorporate both types of facts in the same query is not so easy. It’s not rocket science and it’s very doable. But it’s not so easy. I know because I stumble every time I write a query against EventTable that needs to report on both individual and family fact types at the same time.
That’s an interesting historical synopsis (much of which had escaped my memory) and analytical speculation. You are probably right about constructing more complete queries in sql; it’s what I’ve advocated and demonstrated time and again since co’founding sqlitetoolsforrootsmagic.com.
Hopefully, we will see 10.0.2 soon with that error corrected.
I have added a number of family-related facts/events (Immigration, Emigration, Residence, Miscellaneous), a few completely new single & family facts (Deportation, Migration), and one or two couples-only events (Did not marry, Re-marriage). I have found that all Couples facts (including custom ones) have the same problem as you describe above.
Well, 10.0.2.0 has just come out, and we can rest assured that those individuals with thousands of DNA matches will not be as slow; we can be safe in the knowledge that diacritics will show up correctly in places where they did not before; we know that we now have a newly-built Ctrl+Shift+U function, and myriad of unstated cosmetic issues and minor bugs.
But this particular issue, involving a serious failure of the Grouping function, remains uncorrected, after having first been reported two months ago. Is there some sort of specific priority given these issues? I would have thought that this one, its source having been identified, and solution having been suggested by users, would be a fairly quick fix.
Since version 8 was introduced, I’m running something like zero-for-ever on RMG correcting the bugs I’ve identified:
Reports STILL have a bug where they cannot understand a name suffix that does not conform to a small list (non-English suffixes, primarily). This has been a problem since RMG 8.
Apparently considered by RMG to be a feature instead of a bug, they wiped out the Windows <Ctrl+Shift+Right> and <Ctrl+Shift+Left> functions in favor of adding a redundant function (they already had two means of doing so, but apparently wanted to give our Mac users another);
and this one, which prevents me from being able to create a number of groups.
I ran my somewhat strange query of Birth>Date>Equals>1900 OR Marriage>Date>Equals>1880 that failed so terribly in 10.0.1.0 again just now in 10.0.2.0 and it ran perfectly this time. Could you be more specific about the exact query that is still failing in 10.0.2.0?
Sorry for the delay. I was experiencing intermittent power outages three weeks ago, and had my reply killed by a sudden loss of power. Other things intruded, and I forgot about this. Apologies.
You are correct. When I first tested, I actually (and unknowingly) inserted an error into my criteria. It works fine now.
But they still do not have the name suffixes on the Narrative reports correct yet. Something you won’t notice unless you have a non-standard suffix (as in having French-Canadian ancestors).
But I was mistaken when I thought they still had not fixed the Search criteria.