Saving data VERY slow on large database

I am trying to figure this out – but I can’t figure out where in the software I can check for whether it is 32 or 64 bit.

I am, at least, encouraged to know that it isn’t “just me” and that there is a problem here to be resolved, either by this community or by RM itself, which is probably my next step.

I am going to submit the issue to RM for analysis. If and when I learn anything further I will post back here. If anyone has any other ideas I would welcome them. I have to admit that the distinctions between CPU bound or I/O bound and the details discussed following that conversation are beyond the limits of my computer knowledge at this point.

I may be missing something, but I can’t find a place inside of RM to figure this out.

Outside of RM, if you have an icon somewhere for starting RM, you can right click the icon and then click Properties. Look at where the program is installed. If the high level folder is C:\Program Files, then you have the 64 bit version. If the high level folder is C:\Program Files (x86) the you have the 32 bit version.

On the main window at the upper right, click on the 3-dots and select About RootsMagic. Up pops a window and under the copyright line is the path to where it is installed. If it says “C:\Program Files (x86).…” it is 32 bit. If the “(x86)” is absent, it is 64-bit.

I just started to have a look at this and, to my amazement, RM9 is invisible in Task Manager on my Win 10 Home laptop. I had RM7 open at the same time and it was there. And I have SQLiteSpy and SQLite Studio open on the database of the instance from which I opened the second instance on a large database. Maybe it’s time to reboot - I think I’ve had RM9 and the others open on the same db for days.

That db is 190k people and 500MB. I don’t see much, if any, difference whether I add a Birth fact to somebody without it or do absolutely nothing in the Edit Person screen. On exiting back to the main view, it is unresponsive for 5-6 seconds with no group filter, about double that (12-13s) with a filter. Slow…

The RM7 version of that database (identical except for the structural differences between the RM versions) is almost instantaneously ready to respond on exiting Edit Person.

Restarting RM9 caused it to reappear in Task Manager- how could an active app disappear?!

Simply exiting from the Edit Person screen with no edits resulted in 0 disk activity but up to 30% CPU over the duration the screen was unresponsive. Same CPU level if there had been edits; any disk utilisation was over before the CPU% peaked.

Deleting a fact in the Edit Person screen did result in a very low amount of disk utilisation while in Edit Person. None further on exiting indicating that the database was updated immediately on the user action. Adding a fact type did the same. Editing a fact could result in disk activity while Edit Person was open if the check mark was clicked, else it awaited the closing of the EP window.

If you have a four processor core machine, the 30% represents maxing out one processor core and then using a bit more of a second core. It’s like it’s updating something - either something that doesn’t need to be updated or else it’s updating something extremely inefficiently. It may not be the exact same thing, but it sounds like the same kind of CPU lockup that has been reported when using Advanced Search or when filtering by a group. But you are doing neither.

I keep thinking of things to try. What if the sidebar is set to something other than the Index tab? To tell you the truth, I seldom use the Index tab because I mostly use the Family Tab instead. So by not using Advanced Search and by not using the Index tab, the only place any more where I can see the problem is when using People List View which has the same filtering capability as does the sidebar Index tab.

True for the 5-6s results. Sidebar index filtered doubles the duration.

Agreed there is a gross inefficiency. My suspicion is that nothing much can be done about it in the source code because it’s the compiler that’s at fault.

This is not necessarily correct. Processors don’t just fill up a core and spill over to the next core. Depending on what you are running on the machine, you may have processes running across all 4 cores that total 30% and none of them are maxed.

I have oh so fond memories of my colleges days when we were writing a prime number sieve to run across all 4 cores in our computers and how much fun we had when the professor broke out the machine that was running two huge nVidia GPU’s, each with 12GB of RAM and 3000+ cores.

That prof was kind of a parallel computing geek.

@kfunk it takes some rather sophisticated techniques to program something to run parallel. I’m pretty sure that RM isn’t at that level of sophistication.

I agree that a core doesn’t get to 100% then spill over into another one. Sort of like the electrons around a nucleus. The lower energy electron shells don’t necessarily need to be fully filled before the higher energy shell start capturing electrons.

Don’t I know it, even the simple prime number sieve to more than a little work to spread across and then reassemble. If RM were up to that level, just think how easy it could make things such as dynamically updating relationship lists and groups…or maybe that would just be overkill.

…as for the electrons, thanks for bringing up another horrible two Chem I/ Chem II classes that I took. Fortunately it was at a community college so it probably wasn’t as intense as it could have been. Fortunately, I have lived many years of my life and found that knowing (or not knowing) how the shells filled up didn’t consume much of the limited processor/memory that is located inside my head!

This is what Win 10 Resource Monitor shows on my ageing i5 2-core, 4-processors machine:


The blip on the left is opening the Edit Person; the hump on the right is closing the unedited Edit Person window. They would seem to follow your description.

1 Like

Just for amusement, I fired you Resource Monitor to check my much smaller file (46K people, 68MB). My machine is a rather aged i7 quad core desktop with 16GB RAM. The first bump on CPU2 is opening the edit person screen, the second bump is closing it with no edits. It looks like there was maybe negligible usage of some of the other cores but nothing compared to CPU2. I may need to paste together a much bigger file to see what happens.

I checked using this method and I am running the 64-bit version. I am sending a backup to RM today so that they can review.

1 Like

I pointed out your comments to RM as they make the most sense to me as being consistent with the problem. I really don’t understand computer programming/coding but I agree that the program is doing “something” when I close the edit person screen which may be making very inefficient use of my processor and thus freezing it for anywhere from 5-15 seconds depending on what I did.

UPDATE: After considerable back-and-forth with Diana of RM, here is the conclusion –

Good Afternoon Justin

RM9 works different from other RootsMagic programs. It opens alot of different windows, and closes those windows before it goes to the next screen. I have your file. You have alot of data in this file. I did see it is a little slower than other files I have tested, but thing out of the ordinary.

I am sorry I do not have a answer on how to make the program faster.

Diana
RootsMagic Support

Clearly this has not solved my problem. I am now wondering which of these options makes the most sense for me:

  1. Go back to Ancestral Quest and abandon my hope of a program that syncs seamlessly with my Ancestry.com tree;
  2. Take a “shot in the dark” at a newer computer with more RAM and see if that helps;
  3. Explore whether to use RM 7 as I understand it has something of a different build, but I don’t know if I would still have the Ancestry syncing;
  4. Live with a very unsatisfactory database entry experience.

Any thoughts or suggestions are welcome. If anyone has a faster processor and would be willing to try to run my database to see if the problem exists, I’d be happy to send you a backup copy to play with. Let me know.

Justin

Call me a doubting Thomas but I suspect throwing a new computer at the problem isn’t going to solve this.

RM7 does still TreeShare with Ancestry and so far it still does it well. This may be subject to change and if so, there wouldn’t be a fix for it. I don’t recall if you have RM7 already but if not, you will not be able to buy a license for it any longer. I don’t think the Essentials version allows TreeShare beyond the initial download.

I don’t have a faster processor but do have an observation that I found surprising, having just replaced my Dell Inspiron 5559 system HDD with a SSD. I’m delighted at the speed improvement across bootup to a usable state, to being able to play video files on my external monitor without jerking or even freezing. These operations have gone from largely disk-bound to CPU-bound. But for RootsMagic, not much has changed.
image
The wide orange hump on this graph from Resource Monitor shows the CPU is busy, but not overly so, when RM is switched back from the Edit Person screen to having selected someone different from the Index and seeing the change in the Family View. That hasn’t changed from before the upgrade of the system drive - today, it even seems a second or so slower at 6-7s without the index being filtered.

While I’d be willing to have a look at your database, I suspect my results will be similar to yours given that my system is Processor Intel(R) Core™ i5-6200U CPU @ 2.30GHz, 2401 Mhz, 2 Core(s), 4 Logical Processor(s), 16GB RAM, 1TB SSD. My test database is half-the population of yours.

Would you get an order-of-magnitude improvement from a contemporary high-end processor? I don’t know but that’s the improvement you would want.

Here is some background about “CPU bound” vs. “I/O bound” and also some more thoughts about this problem.

When your computer is actually computing instead of sitting idle, it’s mostly doing one of two things. It’s either doing some sort of calculations, or else it’s reading and writing data from your disk. There can be other devices from which your computer can be reading and writing data, but your disk is usually the main device you are concerned about when considering performance. Often, your computer is doing some sort of calculations and it’s reading and writing data from your disk simultaneously.

So let’s consider some task that your computer might need to do, such as exit from the Edit Person screen while you are running RM9. And let’s suppose that it typically takes 10 seconds to exit from the edit person screen. Well, what might your computer be doing during those 10 seconds? If your computer is mostly doing calculations during those 10 seconds, then it’s CPU bound. If your computer is mostly reading and writing from disk during those 10 seconds, then it’s I/O bound. But calculating and reading and writing from the disk are the only things it can be doing during those 10 seconds.

“CPU” is an old fashioned term meaning Central Processing Unit. It’s the part of your computer that’s doing the actual calculations. These days we speak of things like processor cores instead of a central processing unit, but it’s the same thing. “I/O” is an old fashioned term meaning Input and Output. You can have more than one disk, and it’s possible for one disk to be much faster or slower than another disk. Things like your keyboard and your mouse and your screen are also I/O devices, but they don’t usually have much impact on performance unless you are gaming, in which case your screen can slow down your game by being slow. Also, your Internet connection is an I/O device. Your Internet connection can be slow, but a slow Internet is not likely to have any impact on things like exiting RM’s Edit Person screen.

Something like exiting RM’s Edit Person screen should take a tiny fraction of a second. Modern computers are really fast. Modern SDD disks are really fast. If you still have an HDD disk, that can sometimes slow your computer down, but not an SDD disk. So when we talk about CPU bound and I/O bound, we are really asking what on earth is RM doing during that 10 second delay when exiting from the Edit Person screen. I don’t know, but it either needs not to do it or it needs to do it much more efficiently. The same problem exists when doing an Advanced Search. The same problem exists when filtering People List View by a group. The same problem exists when filtering the sidebar Index by a group. The same problem does NOT exist when creating a group in the first place.

By the way, 10 seconds to exit the Edit Person screen is an eternity in computer time. If it’s 5 seconds, that’s still an eternity. If it’s 2 seconds, it’s still an eternity. All it should be doing is writing the data you updated to disk. Doing so should take much less than a second. But it’s obviously doing other things as well. What are those other things? Do they really need to be done every time you exit from Edit Person? If so, can they be done much more efficiently?