My question concerns uploading a RM 8 tree that was formerly downloaded from Ancestry so contains all of the source images.
When I uploaded the RM8 file I selected to upload the media to the Ancestry Tree. This resulted in a very long upload time, (The upload yesterday took 1 hrs 15 mins with frequent notifications that RM had lost the connection but the upload seemed to keep going and eventually everything appeared in the tree on Ancestry), as all of the 5392 source images, formerly downloaded from Ancestry, being uploaded as well as any new or existing “people images” that I had added to the RM 8 file. I should add that I have not moved the people images to a dedicated folder I have links in the people sheet to the images from other folders on my PC. I did this to avoid having duplicates. The source images are held in a separate folder created during the download and it is this folder that is designated in Settings/Folder Setting as the Media folder (D:\Family History - Roots Magic\Rootsmagic - Import from Ancestry\J and M From Ancestry 30-11-2021 1815_media).
So after that long winded explanation my question is how do I only upload the “people images” that we have added to the RM 8 file and not the 5392 images that have been created during the Ancestry download (incidentally I assume that these images are not required and that Ancestry will go off and find the source image if I click on the source?).
Is it a question of changing the media folder and actually moving the people images into that folder? Do most people move the images that they use in RM8 into a separate folder?
And if I change the media folder do I not lose access to the source images?
Again sorry for the length of this and thanks for your help.
I’m not sure I totally understand the question. So you create a new RM database by using TreeShare to download your Ancestry tree to a new and empty RM database. So far so good. Second, you add some additional media files to your RM database and you don’t move or rename any media files that came down originally from Ancestry. Again, so far, so good.
At this point, are you wishing simply to “sync” the changes you made in RM back up to the same Ancestry tree, or are you trying to “copy” everything that is now in RM to a new Ancestry tree that will be a second Ancestry tree?
I’m going to go out on a limb and guess that you don’t really want to make a second Ancestry tree and that all you wish to do is to “sync” the changes you made in RM back up to the same ancestry tree. In this case, the “sync” is not automatic and probably shouldn’t be called a sync at all. Instead, TreeShare will list the differences between your RM database and your Ancestry tree. These differences will include everything you have added yourself to RM include media files. You will be given an opportunity to approve each change, one a one change at a time basis. The list of media files that were initially downloaded from Ancestry to RM will not be on the change list. You will not be given the opportunity to upload the downloaded media files because they are already there ancestry. And the downloaded media files therefore will not become duplicated in Ancestry.
Jerry,
thanks for your response. I am trying to create a new Ancestry tree based on a combination on the correct data in each tree . The reality is that neither of the trees, the RM one which was imported from Family Historian when I started with RM or the Ancestry one has a complete set of information, different data elements are missing. I elected to use the downloaded Ancestry tree as the base for the new tree as that has all the sources and links in it. I renamed the tree and kept renaming (Move/Rename in RM parlance) it after I had entered some data so that I had copied. I also did regular backups with differing names. What I did in RM was to do a compare between the 2 trees which threw up a load of inconsistencies. So then I had each tree open in a separate monitor and starting with the relatives closest to us go through and add missing elements either using Drag and Drop when it worked or by typing the data in. I did add some new images, photos etc to the new RM tree. I found that trying to sync using Treeshare did not work for me, although I still have a backup of the original connected tree so I could go back to that.
I did managed to create a new tree in Ancestry so now have the original tree, a second one that took ages to upload and has all the media and a version of that second tree but with no media which uploaded much more quickly.
I am very much experimenting with the best methodology, and trying to ascertain the best way to proceed. I intend to remove all of my “test” Ancestry trees and just leave possibly the new corrected tree and possible the original tree until I have convinced myself of the soundness of the new tree.
I hope that makes more sense, sorry for the confusion. If I could just crack the non uploading of the downloaded media I think I would be making progress.
As I type this it has occurred to me that perhaps I use the second uploaded tree (the one with the media files that took ages) and experiment with that using tree share. What confused me when I tried this is I only uploaded that file last night so the databases should be perfectly in sync (irrespective of whether the data is correct or not) so by that logic no-one should be a “changed person” and if I click on “only show changed people” then the list of people should disappear. If fact what happens is that the list appears to stay the same and records that I know I had not changed still remain. hence the reason for not finding the Treeshare very useful.
Aha, now that you have clarified the question, I see your conundrum.
I think your only choices when you finally make your new ancestry tree are to upload all of your media files or to upload none of your media files. I don’t see a way just to upload the files that didn’t come down from ancestry automatically. I could be wrong, but that’s what I think.
I do all my work in RM and only upload to ancestry to get WebHints. I have built small test trees in ancestry and downloaded from ancestry to RM. So I know how it works in both directions. I therefore am aware of a certain irony that when I upload my entire RM database to ancestry, if I include media then I’m uploading many files that originated in Ancestry and therefore that are already there. I don’t see a way around that conundrum and that irony.
When you are in Treeshare there is an Options button … click on that and there is a selection for “Clear everyone from changed list” … selecting that in effect says that there are no differences between Ancestry and RootsMagic. I notice that I periodically have to do this to start from a “clean slate” and not have Treeshare constantly flagging differences that I have no intention of resolving (for example sources or images I don’t want on Ancestry).
I’ve been noodling on this same issue for a while. I tested an approach that should work, but have held off implementing it since it’s convoluted (at best). I put all the family photos (ie pictures of people) that I might actual want to upload to a new ancestry tree in a different file location than all my source/citation images (ie media/photos). When I am ready to push a new tree to ancestry I make a copy of my db, and disconnect the copy from ancestry.com so that I can push a new tree from my copy db. Right now I don’t upload any images, so my family photos and non-ancestry.com source images aren’t included in my ancestry.com tree. The approach I am considering is to temporarily change my media folder names (except the photos folders). This will break all my media links except the images I actually want to push up. (As an aside I also have been doing some SQL stuff on the db copy to fix things like shared events and memorize/paste issues, so that the LinkAncestry table is as accurate as possible. My thanks to you, Tom, and all the others who contribute to https://sqlitetoolsforrootsmagic.com/ .)
Am also hoping this approach will prevent me from pushing up images that would accidentally violate ‘personal use’ copyright. (I have many source/citation images from other sites that allow personal use downloads. It seems to me that sharing these on ancestry.com would not meet the ‘personal use’ criteria.) Anyway, I did a small test and it seemed to do what I hoped. I’ve been hesitant to share this logic publicly for several reasons - it’s only good for a 1x push kind of approach, it would be very easy to muck things up, and I have not tested it at scale. With those caveats, I believe it should do what Michael is trying to accomplish. Since you guys are discussing this topic I’m interested in your thoughts.
edit - I should add that I tested this with rm7. Have not converted the sql queries to work with rm8 tables yet. I don’t expect rm8 to behave differently, except that copy/reuse should reduce the need for tweaks to the rm8 version of the LinkAncestry table. I still like to share facts and expect to continue to need to run scripts before uploading a clean tree to ancestry.
Thank you for the info on the Options button. But, I still do not think it is acting as I would expect.
So as a test I selected the “Clear everyone from changed list” button, then I added an additional photo to the record for myself. I then went back into Treeshare and selected “only show changed people” expecting that my name would appear. It did not. I then unselected “only show changed people” and my name was there.
When I opened my record it shows on the Rootsmagic side the very top row which contains my name as unhighlighted and the icon on the right of the RM side, which I assume is media, highlighted in pink. When I click on that icon a Compare media window appears showing the existing images and new image on the RM side and the existing images on the Ancestry side.
If I click on the right arrow on the RM side I get the option to update general information, and can then go through the process of updating the media on Ancestry and the new photo shows up on my record on Ancestry.
So I would have expected my name to appear on the changed people list after I changed my record by adding media, but it did not. So I have to ask what is the point of the changed people selection? It would seem that I have to keep a manual record of everything I have changed so might as well just upload the whole tree.
Kevin,
thank you for your response. I was considering a similar method.
Can I ask, do you copy all of your non ancestry images into one folder and reference that folder in the filename on the person sheet or do you just use the existing files location in that filename?
What I was thinking of doing is (and this is very much a thought process at present primarily with the safety of the data in mind):
First of all, outside of RM, make a copy of the RM Tree database and the media folder to a separate location, perhaps even to a different HDD
Within RM make a backup of the RM tree with media with a name that indicates which Ancestry Tree it is linked to.
Then do a move/rename and the new name has an indication of what Ancestry tree it is linked to.
Unlink the tree from Ancestry.
Do 1, 2 and 3 again, changing names as appropriate, just to ensure you can recover.
In settings/files either delete the reference to the folder that contains all of the Ancestry files, or use an empty folder.
Then try an upload and see what happens.
When I wanted to go back and work on the tree I would re-enter the information in settings files and use the linked tree, rather than the one I uploaded.
This is undoubtedly convoluted and could easily be fixed by RM by having additional media folders so that you could place Ancestry files in one folder and your own files into others and in the tree share for ancestry upload page allow the user to select which folders/files to upload from.
I will try the above in the next couple of days and report back and I would welcome any thoughts or cautions.
Your observation is correct that not every db change in RM or in Ancestry results in the person being flagged in the treeshare ‘Change List’. This is true in RM7 and RM8. I have not determined the actual criteria other than an observation that attaching a source to a new fact or event seems to always trigger treeshare to acknowledge the changed state. I have not explored how media gets tracked. Perhaps @thejerrybryan, @TomH or some of the other veteran RM users can provide additional details.
In answer to your question, my databases are small (around 1,000 people) and I keep ancestry and non ancestry images under a single common folder (one folder per database). Other folks like subfolders. It’s a matter of personal preference, # images, and how comfortable you are working with large folders. The only distinction I make is for images that I specifically want to upload to Ancestry. Usually, these are photos, but there are some source images too. Those images are in a separate folder (again, I designate 1 folder per RM database).
I am not sure I understand the part of your question about referencing the file locations in the file name but will try to answer. There’s no need to put the name or location of the folder in the file name. When you add an image, RM records the full path to the image in the Multimedia table. RM8 implemented a new feature that stores the media file location in relation other system file locations, for example the folder designated in the ‘Media Folder’ setting. (That’s what’s meant by storing the ‘relative path’. ) The impact of this in the scenario we’re describing is that dragging the media file to a different location may or may not break all the media links. The only sure way to break all the media links is to change the name of the folder(s) containing your media files.
I think the steps you outlined may be a bit overly cautious but should satisfy your primary concern of preserving data. I assume you are doing steps 3-7 to the new db and media you copied in step 1 and not your master db or media. Your step 6 may work “as is” or as noted above you may need to rename the actual folder(s) containing your media files in order to break all the media links. It’s hard to say more on this point without a deeper dive into how RM8 stores file paths. So, if changing the default media folder setting only gives partial success, change the name of the folder(s) containing the media files you don’t want uploaded. What’s really important is that you should never disconnect your ‘Master’ RM database from your ancestry tree because, once disconnected, it can never be re-connected. Always make a db copy and disconnect that if you want to do a 1X push of a new version of your tree up to ancestry.
Kevin,
thank you for your response. What I meant by filename was in fact what is normally called file path. However if you go to a RM 8 person sheet and click on one of the media files it opens up a Edit File Media panel and in that RM refers to the file path as filename, hence my use of that, potentially incorrect, term.
My reason for asking whether you keep a discrete folder with your tree photos is that I have many 10,000s of photos stored by year and subject so for example a folder for 1970 then subfolders beneath that for events so say 1970 - 04 - 13 - Mary Smith Christening, 1970 - 06 - 15 - Billy Bobs Wedding and so on, then within those subfolder I name the individual images with the folder name and then an index number so 1970 - 04 - 13 - Mary Smith Christening - 0010, 1970 - 04 - 13 - Mary Smith Christening - 0020, 1970 - 04 - 13 - Mary Smith Christening - 0030 and so on. This allows me to always know when an image had come from. I would rather just use these “original files” in RM rather than copy them to a RM specific folder
I have asked the RM support what triggers the person being flagged in the treeshare change list, it seems very remiss not to record every change, surely a change is a change!
You are correct in " I assume you are doing steps 3-7 to the new db and media you copied in step 1 and not your master db or media" I have the db backed up to 4 different locations just to be on the very safe side while I am experimenting!
I am assuming that restoring from backup maintains the link that was existing when the backup was created and does not effect the exiting file from which the backup was made?
Yes, restoring a RM backup file will restore the link to an ancestry tree if you’ve accidentally disconnect your ancestry tree from your rm file.
And thanks for explaining the logic of your file system. Now I understand your question. I copy any photo I want to use in genealogy into a dedicated media folder for photos.
Restoring from a backup with media, puts the backups of the media into a folder under the restored database file which has its media paths repointed there. I think I have that right - it used to.
I went through the steps I outlined earlier, I created a separate folder called Rootsmagic - Testing and move/renamed the existing database into that folder (I have realised that the move/rename does exactly what it says it and gets rid of the original database, however as I have everything backed up multiple times I can replace the original DB in the original folder by just copying back, I use software called Beyond Compare which I have used for years on my image folders)
I unlinked the tree from Ancestry.
I had created an empty sub folder called “Blank” under the folder where all of the "Ancestry Images had been placed, so in settings files I changed the Media Folder to this sub folder. I checked the Media tab and all of the images had a red cross and a warning that the image could not be found.
I then went through the upload and creation of the new Ancestry Tree I left the upload media tick box ticked. The whole upload took just over 8 mins compared to the 1hr 15mins that my original took!
The loading of the media files was about 1 minute. Afterwards I realised that some of the images that appeared in RM had been actually uploaded directly to Ancestry (I should perhaps explain that I recently took over responsibility for this project from another family member, so have been trying to understand not just RM but also the structure and locations of the tree) so they did not go up. So I will have to move all of the photos etc that were uploaded and downloaded to a different folder that I can reference, or perhaps just find them in the Ancestry download media folder.
In fact thinking about it and looking at the filename in the Media part of the Person Sheet the location is there, so why did it not load. The only difference I can see between the images of me that I loaded yesterday that did load and some of my Grandfather taken in the 1920s that were found online is that the “include in scrapbook” tickbox is not ticked. So I will have to do some more experimenting.
When I clicked on the source files in ancestry they appeared almost instantly. I don’t know whether Ancestry locates the sources and loads the images in the background or whether it happens when you click.
So all in all I think I am making progress, will try another upload after looking at the include in scrapbook issue, there is no mention of it needing to be ticked to upload in the RM8 Wiki.
My experience is that Sources and media attached only to the ‘person’ do not upload to ancestry via treeshare. Sources and media that have been attached to “Facts” do upload via treeshare. Good find on the ‘include in scrapbook’ setting. I thought this only impacted RM reports but it makes sense that it would carry over to exports.
I hope that that setting has no effect on exports. It never used to. If it is implemented, then there must be an option in the export and Upload dialogs to choose All or Scrapbook Only. That’s the only way it would make sense.
So life caught up with me for awhile so have not done much on Ancestry and RM8.
I have done a bit more testing and have found the following:
include in scrapbook. I have found this to be variable, so sometimes items marked with include in scrapbook do upload sometimes not. So finding it hard to understand whether or not this has any effect. I suspect that it does not.
Kevinm stated “Sources and media attached only to the ‘person’ do not upload to ancestry via treeshare”. I think that the reason for this is that they also do not cause the person to appear on the changed list. I found that if I change a Source and Media attached to the person and then go to Treeshare and go to that person it does show that there have been changes and them you can upload them! I am still waiting for a response to my question to Rootsmagic as to what triggers the Changed List. Perhaps they do not know?
Uploading Experiments: I think I have come up with a solution which for me works 100% of the time. In an earlier post I talked about creating an empty sub folder called “Blank” under the folder where all of the "Ancestry Images had been placed, then in settings files then changing the Media Folder to this sub folder, but not move any images. I checked the Media tab and all of the images had a red cross and a warning that the image could not be found. As stated earlier when I tried this it reduced the load time from 1hr 15 to around 8 mins. However, the next time I tried it the load time was back to over 1hr. I then discovered that RM8 had changed the location of the individual files (all 5388 of them) to the “blank” sub folder! So then I created a completely separate folder called “Media Temp Location”, and moved all of the 5388 files to that folder. I then uploaded to Ancestry and it was back to around 8 mins (the timings are: Media finished at 1:04, Tree finished at 7:34 and Collect changes finished at 8:26). After the upload is finished I copy the 5388 files back (which on my M.2 SSD takes about 30 secs) and when I click on the Media tab all of the files are found (no Red crosses). So as this worked 100% of the probably 10 times I have tried it. So based on this and the unreliability of the change list I am updating my data in RM 8 and doing lots of backups and then spending the 8 mins uploading to a new Ancestry tree and delete some of the previous test trees.
The issue that I am also having to how to deal with Ancestry hints. I have realised that I cannot trust the Treeshare to highlight any hints that I accept and incorporate into the Ancestry Tree, so that I can copy them to RM8. How do people deal with incorporating Ancestry hints into RM8 without using Treeshare. I did try doing some manually but that is a pain or at least the Census find that I tried yesterday was convoluted to put in mildly. Any suggestion gratefully received.
With regard to Media locations I have, for now, adopted kevinm’s method of having a separate dedicated media folder. One of the reasons for this is I found that when I tried to add .jpg files that were stored in directories that had very long paths RM8 would not add them. This was irrespective of whether I browsed to the location or dragged and dropped them. However, when I copied the files to the dedicated media folder I could add them by either method. So I wonder if RM8 has adopted the “Long File Names” discipline that Microsoft introduced in I think 2017. The registry in my PC has been set to use them and other applications work. I will ask Rootsmagic once they respond to my current questions.
I just do it manually, pain or no pain. That way I can name the downloaded images the way I want them named. I can store the downloaded images in the folders where I want them stored. And I can create sources and citations using my own source templates that look like I want them to look.
I’m probably just a glutton for punishment, but downloading the images and setting up the sources are not the most time consuming part of my workflow. I also do a complete transcription of any document I download. The transcription becomes a note both for a fact and for a research note for a citation. So by comparison, just downloading the image manually and entering the source manually is a relatively small part of the job.
I don’t use Treeshare. Every item of data in my database is entered manually. I don’t download data from anywhere but do sometimes turn on hints for Ancestry or FindMyPast. Usually I find that I have already found the information in the “hints”. I do of course download media which I save into my media folders/subfolders renamed as per my standard naming format.
I have had a reply from Rootsmagic regarding this issue:
Michael,
Sorry to take so long to get back to you. Sadly, the answer is with the Ancestry API that the upload of media is all or nothing. You would physically have to remove the media from the Media Galley so it doesn’t upload when creating a new Ancestry tree or adding a new person or source that has one. We have a request to be able to privatize media so it cannot be uploaded, and hopefully that will be possible one day. We don’t have a lot of control over the Ancestry API since Ancestry is the one that builds and maintains it.
So it looks like the only way to avoid loading 1000s of images is to use a workaround like the one I described above not ideal but it does work.