BT Cloud Backup unreliable

[RKR]

I am using BT Cloud 21.12.15 and have a problem with the Cloud Backup process being unreliable.

I have various folders on my computer set for "continuous" backup to the cloud (i.e. NOT "weekly" etc - I have checked this). I have recently been doing a lot of work creating new files and updating documents that it is very important are backed up to the cloud, but none of these have been uploaded even though the folder in question is definitely selected for continuous backup. I have not changed any settings at all recently; the backup used to work at least to some extent - I'm not convinced all my new files were being backed up, but as can be seen from the screenshot, it did backup some from the folder in question 5 days ago and also 9 days ago. There are also new documents in other folders that are also set for continuous backup that have NOT been backed up - I saved the screenshot used above into a folder that should be continuously backed up for instance and BT Cloud Backup has not  sprung into life, so it is not a problem with just one folder. The ONLY thing that DOES seem to be backed up - and this same item gets refreshed every time I restart BT Cloud to try and make it work - is ThumbCache.dat which is not something I actually particularly care about(!) As you can see I have 700GB free, so space should not be an issue.

 As a "workaround",  I am currently "uploading" (as opposed to automatic backup) the most critical folder in question just so that I have a backup of the most important items, but this is not in the correct place in my folder tree of previously backed up items, and doing this does not solve the underlying problem of why automatic backup has become so unreliable as to be practically useless - especially given that it used to work... I need to be able to be confident that ANY new item that is created within ANY folder that is set for continuous backup will in fact be backed up, ideally within the hour!

Many thanks for any help or insights regarding this problem.

[Earthling]

Personally I haven't seen any evidence at all of my continuous BT Cloud backups failing but I wouldn't put myself in the position of being wholly reliant on them either - I maintain local backups too.  I have found Windows File History to be the simplest reliable automatic file backup system that does not require any action at all on my part.  A decent HDD plugged into my hub holds file backups from each system on my LAN.  A NAS is another good solution of course - at a cost!

[RKR]

I do of course also back up locally, with two separate external HDD, one always connected for automatic backup, and one only connected during supervised manual backups to it so it cannot be a victim of a ransomware attack, but those local HDD backups would not be much use in the case of a house fire or burglary etc, which is why I pay extra for a faster broadband service than I can currently make use of, specifically in order to have BT Cloud storage. Given that I am paying for it, I really would like the automatic backup to be reliable, and would welcome any advice or insights, especially from moderators / those within BT itself, as to what the problem might be so that I can return to the situation I had before where new items would be backed up automatically - unfortunately it currently really isn't fit for purpose, and I have no idea what has changed from the situation I had before, to cause it to no longer want to back up my files. 

[RKR · Accepted solution]

In case anyone else finds this useful, I think I've solved the problem!

It appears that Windows 10 "Controlled Folder Access" had unilaterally decided to block access to my folders by the BT Cloud app, presumably about a fortnight ago, and as the backup is something that happens in the background, I wasn't around to see any notification about this - not that I'm convinced it gave one, as I could find no record of it in the list of attempted security "breaches", which is where I normally go to give specific apps permission to access folders! The way I discovered what the problem was, is when I experimented by trying to add a particular folder to the automatic backup list, and a fleeting pop up occurred saying access was being denied. I have now added the BT Cloud app to the "allowed" list [Search for the Microsoft article "Allow an app to access controlled folders" for instructions if you are unsure] and dozens of items from the last fortnight  are now being backed up, much to my relief, as I just couldn't get it to work in any other way, and now I completely understand why! As the Controlled Folder Access Allowed list specifies which numerical version of the BT Cloud app it is allowing to access my files, I may well need to add any future versions to the list manually if the BT Cloud app is updated, but at least I now know why there was a problem, and why that one totally unimportant file was still getting updated repeatedly, as it was in an unprotected folder. I hope this may help anyone else using Windows 10 for whom the BT Cloud automatic backup process malfunctions for no obvious reason!

[bluefin]

I'm seeing similar behaviour to your original issue (I think). I have Macrium creating backup files (split into 500MB pieces) daily in a dedicated folder which I've told BT Cloud to backup (not sync). There are dozens of these files, but they're just not going up to the cloud. "Recently updated" tells me that I had files uploaded to Cloud:

19 hours ago, 20 hours ago, 20 hours ago, 5 days ago, 5 days ago, 8 days ago, 8 days ago.

It's like it does 2 or 3  files then just doesn't bother any more. I quit the client and restart it, and it does a few more files then gives up again. There's not even any particular rhyme or reason for the ones it *does* backup. The Macrium files have a sequence number - last files backed up (most recent first) are 631, 544, 100, 440, 52, 180, 299.

I was hopeful when I saw your solution about controlled folder access, but when I checked it's not even turned on!

In previous attempts at fixing the problem I've uninstalled and reinstalled the client. I've turned on diagnostic logs, but can't see anything in the output that shows an obvious failure. I don't know what else to try.

[RKR]

@bluefin  I feel your pain - the "total block" of controlled folder access barring BT Cloud was a very real issue for me, and discovering it was definitely worthy of marking the problem "solved" on this forum, as I think it may be a complete solution for some, that it is really worth highlighting.

However, I still do have ongoing "unreliability" issues that sound similar to yours, i.e. (I quote) "It's like it does 2 or 3 files then just doesn't bother any more. I quit the client and restart it, and it does a few more files then gives up again. There's not even any particular rhyme or reason for the ones it *does* backup."

What I have noticed in my case, which may or may not have relevance for you, is that problems occur when the backup seems to get "fixated" with one particular large file (of say 300MB), and attempts to back that same file up Every. Single. Time. that I restart the Cloud Client, even though the file has not changed at all in months... It may be that the file in question at one point had its backup interrupted by someone pressing the "Pause Syncing" button, or some other reason, but that one file will be a perpetual log jam item that causes problems - the backup will try and do that file, *seem* to succeed, but refuse to do anything else, and then repeats this on every restart - until I move that one file to a folder that I am not backing up. Once it is no longer in its original place, loads of other files will immediately get backed up successfully and really rapidly. The files that cause the problem for me are usually not very important, so I have not attempted to put them back into their original folders, either with the same name, or with e.g. the suffix "a", to see if they then won't cause that glitch in the future, but that might be a worthwhile experiment for me to do when I am less busy. Maybe I could create a new folder that is still on the backup list where I put all the "troublemaker" files whenever they arise, and see if they get uploaded correctly from there, where cloud will *think* they are completely new files?!

Another factor may be that I do not have very fast broadband due to being 0.9miles from the green box, and not yet being offered fibre to premises, which I am desperate for. This means with an embarrassingly slow  upload speed, there is probably more opportunity for a larger file to cause a glitch than a smaller one? You may not have that same issue of slow broadband, but I did note that you said your files are 500MB - is there any way that you could experiment with the files being split into (say) 100MB pieces instead as a temporary measure, and see whether that improves things? I'm not suggesting that as a long term solution - splitting e.g. a video isn't going to be practical for everyone, and surely BT Cloud should be able to back up files of that size?! - but it might give a clue as to what the issue(s) we are experiencing really is/are, which we could then ask e.g. @SeanD to get investigated?

I look forward to hearing any updates from you here as if we combine our experiences maybe it will give enough data for a proper solution to be found. Here's hoping anyway(!)

[bluefin]

Thanks for getting back to me. 😊

On the speed front, I have FTTC and it's not particularly fast. I used to split the files into 1GB pieces and I think they took an hour or more each to upload (and the whole backup might take 4 or 5 days, in those halcyon days when it did seem to work!). I haven't noticed how long the 500MB pieces take, but I'd assume it's 30mins+. Unfortunately, 500MB is the smallest 'piece size' that Macrium allows me to set so I can't easily try something smaller. I might be able to do something with Zip as a trial, but I'll have to work on it.

Interesting idea about 'troublemaker' files - I'll have a look and see if the logs can tell me if there have been multiple attempts at anything. I'm now also wondering how it tracks what's been backed up, and maybe that's got corrupted. I would have thought that would have been 'fixed' the time I reinstalled the client, but maybe it didn't clean up properly and some corruption persisted across installs. Hmm, OK couple of things for me to look at then!

[RKR]

Thank you also for your response! A 52MB document of mine took about 20 minutes to back up earlier today, so your speed is quite a bit faster - unfortunately I then had to pause the backup process entirely as it was stopping other household members from using the Internet at all(!) After the weekend I might be able to restart backup without annoying everyone else, and then I will do some further investigations the first time I spot it not functioning correctly - frustratingly it WAS working pretty well when I had to pause it today to let other people use the Internet, but at least I was able to wait until it wasn't part way through a file, so maybe that will prevent a "troublemaker" file being created and the glitch occurring? If I do get a log jam I will try setting up a brand new backed up folder for the troublemaker(s) and see what happens. I will also try and find time to do a detailed comparison check between various of my most important folders in the Cloud and on my PC and see if all of their contents have been backed up or only some files. Hopefully by comparing notes we can pin down a bit more exactly what is causing the problem and the worrying general unreliability and unpredictability!

[RKR]

I was able to turn Backup on again overnight, and 18 items were backed up. When I went to bed it was attempting an important 1.5GB file, but this morning that is not in the list of backed up files on the desktop client, and nor is it visible in the web client folder tree.

I have also checked some other folders with photographs from August. The contents are definitely NOT complete - some photos are there, whilst others are not. The folder with September photos in has not yet been recognised and is not showing up on the web client, despite it being a subfolder just like the August one (which is in the same main folder) that HAS been recognised though the contents are concerningly incomplete.

I have not been able to "trigger" the Backup process to restart this morning.

Things I have tried include:

(1) Exiting BT Cloud from the desktop client and then restarting it from the start menu.

(2) Going into the web client, selecting a folder that I know should have content being uploaded continuously that hasn't been, turning off backup for that folder and then clicking save, then reopening the web client, turning the backup for that folder on again, checking it is still set to continuous and clicking save again, and then opening the web client for a third time to double check that all the boxes are ticked to select that I want them backed up.

(3) Doing a Restart for my entire computer, and not opening anything else so that there are no unavoidable competitors for usage of CPU, RAM or Broadband.

(4) Creating a new screenshot picture and saving it in a folder that should be being backed up continuously.

None of these have done anything at all - the desktop client says that it is syncing, but nothing is actually happening, and nothing is being uploaded to the cloud. I have double checked that it is NOT a "controlled folder access" issue again.

QUESTION(s) that it would be really helpful if @SeanD (who has been brilliant before in doing his best for those in the BT community, and who I trust to actually try to do something!) could answer (ideally after checking through all posts in this thankfully brief thread) to help us understand where the ongoing glitch might be, or if in fact it is just a limitation of the product:

(a) How does BT Cloud identify what needs to be uploaded to Cloud Storage? i.e. does it have to check through all of the (probably) tens of thousands of items in the hundreds of subfolders of the dozen or so folders I have selected to be backed up every time the Backup process is restarted?! If so, I could imagine it might take a while to identify new target files, but that would seem unworkable in a marketable product / service.

(b) Is the backup process driven by the web client or by the desktop client, and would my slow broadband speed affect how long it takes for the new files to be identified?

(c) The process for selecting which files do get backed up and which get missed seems entirely random, as @bluefin mentioned in a previous post. I am left concerned that my most important important items (e.g. a document with two month's research, and precious irreplaceable photos) are not being backed up at all, as they are NOT on the web client, whereas other things have been. Is there a way of fixing this? Ideally the backup process needs to be 100% reliable (which is what I am paying for!), which it definitely is NOT currently, but second best would be a way of "triggering" backups to ACTUALLY start when the desktop client is (apparently) just sitting there totally idle (as mine has been for the past 5 hours despite the fact that there are hundreds of unbacked up items in folders that I know Cloud 'knows' about as some of the other contents HAVE been backed up!); third best (a temporary workaround for peace of mind) would be a way of prioritising a particular folder or file for immediate backup to the cloud, enabling it to "jump the queue"...

If there are any sorts of logs, system information etc that would help I am more than happy to supply them to @SeanD  - I'll do almost ANYTHING if it means I get the working and reliable Cloud backup that I am already paying for, but am not receiving, to supplement my local backup options in case of fire or theft. Many thanks.

[RKR]

Just a screenshot to show how long ago it was that anything was backed up, despite BT Cloud Backup being on continuously since my last post 3 days ago:

You can tell by the file names when some of the documents and images were created that first got backed up 3 days ago - there are other files from a similar timeframe that have still not been backed up at all. The desktop client has the icon for "Syncing" - it *knows* it is not up to date, or the icon would change - so why is nothing happening?!

I am guessing that @SeanD is probably on a very well deserved vacation, so I will await his return to see if he can help me, @bluefin , and the other BT Cloud users with similar issues, to regain access to a functional backup system. I don't want to have to manually upload all my files to the cloud, where they are then not located correctly in my computer's folder tree for easy access, but in a separate cloud folder; all I want is for the automatic backup I am paying for to work(!) Many thanks in anticipation to @SeanD .