BT Cloud Backup unreliable

[RKR]

Still no backups happening...

I found this post by @Old_Deuteronomy  that looked like it might help:

on ‎08-07-2021 11h44

1,562 Views

Message 18 of 27

Re: BT Cloud 21.4.10

Hello together!

OK, here's a recommendation I got from BT that solved the problem of stalled sync and the inability to see the repositories on different computers.  I applied it to both my Windows  PC and laptop, but I can't confirm the Mac advice works OK.

Steps:

1. Make sure to exit the application with the Exit button

It's on the Systray menu

2. Open the Data folder the App

On Windows 10:

%userprofile%\AppData\Local\BTCloud-Data

On Mac

~/.local/share/BTCloud-Data/

Make a backup of all files and folders in that folder (best to zip them and save to Desktop)

Delete the .db files in this folder

3. After this the user will need to set up the backup and sync folders

Make sure that it's using the same backend mapping as before for the same folder

Note that the client will see that the sync folder already exists, so it will suggest a new one

user will need to manually choose the old sync folder!​

This is the direct copy of the advice. I found I had to do a small interpolation of them on my devices, but that was no problem.

I haven't tried it yet as I wasn't sure whether it is relevant to the CURRENT version of BT Cloud, and if it is I would need a touch more clarification about Step 3 (bearing in mind I am ONLY interested in BACKUP from my main PC and I don't use the Sync folder separately) - "After this the user will need to set up the backup and sync folders. Make sure that it's using the same backend mapping as before for the same folder" - is this set up in %userprofile%\AppData\Local\BTCloud-Data or is it in the settings within the popup from the desktop client...? If the  former, I wouldn't know how to do  that, but if (as I suspect) it is the latter, I'd need to make sure I had a detailed list of exactly which folders I currently have selected before deleting anything, and that process could be quite a pain, so if there is any other way around I'd prefer it!

I did find another helpful-looking post where someone tried to make their backup process malfunction, and eventually got it to restart, but I'll need to try and find that again before referencing it in a post!

[RKR]

Quoted below is the other post I found that looked helpful, by @JTS1 a year ago. It may have something of use in it for the problems detailed in this thread, but again, this may not apply to the latest version of BT Cloud.

"JTS1
Expert
on ‎07-07-2021 20h03

1,601 Views
Message 16 of 27
Re: BT Cloud 21.4.10
@Earthlingwrote:
Thanks John - just tried what you did by opening and saving a couple of files and they backed up inside a minute! No idea why nothing was happening earlier but looks like backup may now be working as it should.

I'm glad that you have had a successful outcome.

Since the the December version, where the BT Cloud problems began, I have been conscious that those having backup problems, were the members who had long term accounts that had accumulated large amount of big files. I was also conscious that many of them tried all sorts of ways to resolve their problems, but to no avail.

I therefore attempted, this week, to replicate the problem by trying to upload a folder containing many video files, the total volume of which was over 6GBs.

Initially, syncing progressed well, but it still wasn't completed. So during the process, I attempted to change the name of a totally different backup folder, added a single Sync file and deleted some of the files from the original folder containing file that were still in the sync process, to see how the BT Cloud would react.

I then turned my PC off and back on again.

I found that I had succeeded in stalling the syncing process. No matter what I did, I couldn't get the sync process to restart (just as described by many other users).

I then looked at my BT Cloud Logs and found that the BT Cloud recognised that sync was not completed, as the deleted files, which were named in the logs, were not there to be backed up. However, it ended in a loop, where it tried to find them, but couldn't, then tried to find them again, but couldn't, and so on.

I spent most of the week trying to get BT Cloud to start syncing, by pausing and unpausing, switching BT Cloud off & on again and shutting down and restarting. Nothing happened, despite repeating this over several day. Eventually, I decided to remove the entire backup folder that was causing the problem - still nothing and sync remained paused.

As a last resort, I unpinned it from my Backup Preferences and clicked save. Hey presto, after a dormant week, syncing started immediately.

Since then, I have added the same backup folder (and others) without any problems, so it is recoverable from the user's end.

JTS1"

Looking at my most recent BT Cloud Logs from today (with "COMPUTERNAME" in place of my actual computer name), I have the following repeated constantly hundreds of times:

2022-09-09 00:04:42.366 +01:00 [INF] CreateAttachTask DESKTOP-COMPUTERNAME finalizables status: "HasPendingRetryAttachJobs", { SourceContext: "Sncr.Cloud.Core.Workflows.AttachTaskFactory" }
2022-09-09 00:04:42.366 +01:00 [INF] InternalFinalizeSynchronization incomplete, waiting 5 seconds , { SourceContext: "Sncr.Cloud.Core.SyncService" }
2022-09-09 00:04:47.381 +01:00 [INF] InternalFinalizeSynchronization, { SourceContext: "Sncr.Cloud.Core.SyncService" }
2022-09-09 00:04:47.404 +01:00 [INF] CreateAttachTask DESKTOP-COMPUTERNAME finalizables status: "HasPendingRetryAttachJobs", { SourceContext: "Sncr.Cloud.Core.Workflows.AttachTaskFactory" }
2022-09-09 00:04:47.419 +01:00 [INF] InternalFinalizeSynchronization incomplete, waiting 5 seconds , { SourceContext: "Sncr.Cloud.Core.SyncService" }
2022-09-09 00:04:52.435 +01:00 [INF] InternalFinalizeSynchronization, { SourceContext: "Sncr.Cloud.Core.SyncService" }

This seems to show a closed loop is occurring because InternalFinalizeSynchronization is incomplete...

Going back 5 days to the last time when anything backed up successfully, I have found the logs for when the problem started. [Certain parts of file names etc changed to "ME" for privacy.]

Is there anyone out there @SeanD  @JTS1 or indeed anyone else who can help interpret this following excerpt from the Logs and/or tell me what to do to enable Backup to restart and stay working? I would be eternally grateful!

2022-09-04 11:24:52.074 +01:00 [DBG] SyncRoot found : Path: S:\Users\ME\Pictures\My Pictures\ME other BACKUP pictures, ServerPath: /S/Users/ME/Pictures/My Pictures/ME other BACKUP pictures, RepositoryName: DESKTOP-COMPUTERNAME, Active: True, Frozen: False, Synced: True, Frequency: Daily, TransferMode: Upstream, NextScan: 05/09/2022 08:39:23 +00:00, IsMetadataSet: True, { SourceContext: "Sncr.Cloud.Model.Workflow.States.Sync.SyncPreCheckState" }
2022-09-04 11:24:52.075 +01:00 [DBG] Transition to "Syncing" triggered by "Sync", { SourceContext: "Sncr.Cloud.Model.Workflow.Managers.SyncStateManager" }
2022-09-04 11:24:52.075 +01:00 [DBG] Leaving "PreCheck", { SourceContext: "Sncr.Cloud.Model.Workflow.Abstract.StateAbstract`2[[Sncr.Cloud.Abstractions.States.SyncStates, Sncr.Cloud.Abstractions, Version=21.12.15.0, Culture=neutral, PublicKeyToken=null],[Sncr.Cloud.Abstractions.States.SyncTriggers, Sncr.Cloud.Abstractions, Version=21.12.15.0, Culture=neutral, PublicKeyToken=null]]" }
2022-09-04 11:24:52.075 +01:00 [DBG] Entering "Syncing", { SourceContext: "Sncr.Cloud.Model.Workflow.Abstract.StateAbstract`2[[Sncr.Cloud.Abstractions.States.SyncStates, Sncr.Cloud.Abstractions, Version=21.12.15.0, Culture=neutral, PublicKeyToken=null],[Sncr.Cloud.Abstractions.States.SyncTriggers, Sncr.Cloud.Abstractions, Version=21.12.15.0, Culture=neutral, PublicKeyToken=null]]" }
2022-09-04 11:24:52.077 +01:00 [INF] CreateUserInfoTask, { SourceContext: "Sncr.Cloud.Core.SyncService" }
2022-09-04 11:24:52.080 +01:00 [DBG] RefreshToken called.., { SourceContext: "Sncr.Cloud.Core.AuthorizedHttpHandler" }
2022-09-04 11:24:52.412 +01:00 [DBG] AuthenticationStore Updating, { SourceContext: "Sncr.Cloud.Net.AuthenticationStore" }
2022-09-04 11:24:52.578 +01:00 [DBG] RefreshToken called.., { SourceContext: "Sncr.Cloud.Core.AuthorizedHttpHandler" }
2022-09-04 11:24:57.791 +01:00 [INF] SynchronizeServerMetadata, { SourceContext: "Sncr.Cloud.Core.SyncService" }
2022-09-04 11:24:57.791 +01:00 [DBG] GetRepositoriesFromServer, { SourceContext: "Sncr.Cloud.Core.DigitalVaultClient" }
2022-09-04 11:24:57.913 +01:00 [INF] ReconcileDownstreamChanges, { SourceContext: "Sncr.Cloud.Core.SyncService" }
2022-09-04 11:24:58.145 +01:00 [DBG] Reconcile downstream changes, { SourceContext: "Sncr.Cloud.Core.SyncService" }
2022-09-04 11:24:58.145 +01:00 [DBG] ReconcileDownstreamChanges all changes processed, marking as complete, { SourceContext: "Sncr.Cloud.Core.SyncService" }
2022-09-04 11:24:58.145 +01:00 [DBG] ReconcileDownstreamChanges Waiting for completion, { SourceContext: "Sncr.Cloud.Core.SyncService" }
2022-09-04 11:24:58.145 +01:00 [DBG] ReconcileDownstreamChanges finished, { SourceContext: "Sncr.Cloud.Core.SyncService" }
2022-09-04 11:24:58.145 +01:00 [INF] ReconcileDownstreamChanges finished with 187, { SourceContext: "Sncr.Cloud.Core.SyncService" }
2022-09-04 11:24:58.145 +01:00 [INF] FinalizeSynchronization, { SourceContext: "Sncr.Cloud.Core.SyncService" }
2022-09-04 11:24:58.145 +01:00 [INF] InternalFinalizeSynchronization, { SourceContext: "Sncr.Cloud.Core.SyncService" }
2022-09-04 11:24:58.150 +01:00 [INF] CreateAttachTask DESKTOP-COMPUTERNAME finalizables status: "HasPendingRetryAttachJobs", { SourceContext: "Sncr.Cloud.Core.Workflows.AttachTaskFactory" }
2022-09-04 11:24:58.155 +01:00 [INF] InternalFinalizeSynchronization incomplete, waiting 5 seconds , { SourceContext: "Sncr.Cloud.Core.SyncService" }
2022-09-04 11:25:03.168 +01:00 [INF] InternalFinalizeSynchronization, { SourceContext: "Sncr.Cloud.Core.SyncService" }
2022-09-04 11:25:03.177 +01:00 [INF] CreateAttachTask DESKTOP-COMPUTERNAME finalizables status: "HasPendingRetryAttachJobs", { SourceContext: "Sncr.Cloud.Core.Workflows.AttachTaskFactory" }
2022-09-04 11:25:03.190 +01:00 [INF] InternalFinalizeSynchronization incomplete, waiting 5 seconds , { SourceContext: "Sncr.Cloud.Core.SyncService" }
2022-09-04 11:25:08.190 +01:00 [INF] InternalFinalizeSynchronization, { SourceContext: "Sncr.Cloud.Core.SyncService" }
2022-09-04 11:25:08.195 +01:00 [INF] CreateAttachTask DESKTOP-COMPUTERNAME finalizables status: "HasPendingRetryAttachJobs", { SourceContext: "Sncr.Cloud.Core.Workflows.AttachTaskFactory" }
2022-09-04 11:25:08.200 +01:00 [INF] InternalFinalizeSynchronization incomplete, waiting 5 seconds , { SourceContext: "Sncr.Cloud.Core.SyncService" }

[RKR]

Just tagging another couple of moderators whom I have seen to be very helpful here in the BT Community, to see if this whole short thread of posts could be read, and anything done to help with the ongoing element of my problem (which is DIFFERENT to the single aspect of the problem which is marked as SOLVED, so PLEASE do continue to read on, despite the "solved" tag!) that prevents my BT Cloud backup from getting out of the loop it has been stuck in for the past 6 days during which time nothing has been backed up at all! I am hoping that the Log information I have provided will provide some clues. Many thanks in advance to @DavidM and @DanielS .

[DavidM]

Hi @RKR and thanks for the posts.

I'm sorry you're having problems with the cloud backup. I'd like to get our experts to take a closer look at this for you. I'll drop you over a private message now so you can get in touch.

Cheers

David

[RKR]

Hi @DavidM  and thank you so much for your reply. I have responded to your private message with the details you requested, and I do hope that the team will be able to resolve my issue, and by doing so, also that of @bluefin who seems to have similar problems to me with BT Cloud Backup being unreliable, and who has been on a similar journey to me in this thread on the forum. Many thanks again.

[VanDyneski]

Same issue here with my S10+ files  and photos over 7 days old not auto backed up and had to be done manually 

[bluefin]

I don't know if something has changed (not at my end, certainly), but it does appear that uploads are now happening consistently and continuously - none of that giving up after managing a few files.

So, hooray?

[RKR]

@bluefin  There IS a new version, which I think I was sent by the team investigating my specific case to trial a little before it was (I presume from what you said?) more generally released, and *most* of my files do now back up automatically - it took about a week of uploading before all the smaller "missing" files made it to the cloud, as so many of them hadn't uploaded before, but I am 99% confident that they are all now there. This is certainly MUCH better than before, but I do still have an issue with large files (over about 100MB) as these seem to "clog the pipe" and not only do they not upload automatically, but they also stop the small files uploading that get caught up behind them. I suspect this might be something to do with my very slow broadband speed?
Thankfully I only have a dozen or so of these large "problem" files that I need to back up, so as a workaround (both to ensure they ARE backed up, and that they don't prevent my other smaller files from being uploaded) I have moved them elsewhere on my PC, to a folder that I have NOT selected for automatic cloud backup, and then I have been uploading them manually overnight, at the rate of one large file per night - if I attempt this in the daytime nobody in the house can use the Internet because the upload takes up all the bandwidth! This isn't ideal, both due to the inconvenience and because the files are in "the wrong place" both on my PC and in the Cloud, but I can live with it until either the BT engineers can find out what the problem is from the log files I have sent them, or until "fibre to premises" eventually becomes available on my road...
Overall, I have found that the new version of the cloud software is INFINITELY better than the old one in terms of reliability and working how it should, with the one exception of those problematic large files!