This topic

Backbone Routing Issue

KernowImmigrant — Mon, 31 Oct 2022 13:11:07 GMT

Been trying to work from home via my BT fibre connection and been having this problem since about 12:00. Looks like the BT backbone has failed and didn't fail over properly to it's redundancy. Long shot, but is there any chance that someone knows an ETA to fix? This is affecting multiple sites I need for work, but not all.

Re: Backbone Routing Issue

Eambo — Mon, 31 Oct 2022 13:24:24 GMT

Reporting the same issue here - I can't even get to peeringdb to get the NOC contact details for BT.

This is impacting a huge amount of traffic going from UK to US - which is why downdetector is showing an outage for almost every product out there.

Re: Backbone Routing Issue

mrmarky — Mon, 31 Oct 2022 13:32:33 GMT

Also confirm there is an issue, i'm using a vpn to get around the Backbone Routing Issue for the moment.

Re: Backbone Routing Issue

Eambo — Mon, 31 Oct 2022 14:10:46 GMT

Just got through to BT who confirmed the issue but said it's estimated to take 4-24 hours to resolve. Hopefully they can find an engineer who knows how BGP works before that time and fix their Manchester interconnect, as blackholing this much traffic destined to the US is not a good look for any ISP, nevermind the UK's largest! 🙂

Re: Backbone Routing Issue

ptan — Mon, 31 Oct 2022 14:22:25 GMT

Ah, I wondered why I lost connectivity to Fuze ... I thought it was just me!

traceroute to fuze.com (54.204.252.11), 64 hops max, 52 byte packets
1 192.168.1.254 (192.168.1.254) 0.862 ms 0.658 ms 0.491 ms
2 * * *
3 * * *
4 31.55.186.184 (31.55.186.184) 3.877 ms
31.55.186.180 (31.55.186.180) 3.766 ms 4.352 ms
5 * * *
6 * interconnect2.manchester.fixed.bt.net (194.72.2.5) 10.650 ms 10.187 ms
7 * * *
8 * * *
9 * * *
10 * * *
11 * * *
12 * * *
13 * * *
14 * * *
15 * * *
16 * * *
17 *^C

Re: Backbone Routing Issue

Eambo — Mon, 31 Oct 2022 14:26:09 GMT

We can at least confirm not just you thankfully. I'm not sure if BT sends *all* US bound traffic through that manchester interconnect, but certainly a large chunk of it seems to go through there - and whoever they're trying to handover to is having a bad time.

As such, traffic seems to be going missing in Manchester when destined for US Services 😞

Re: Backbone Routing Issue

Will168 — Mon, 31 Oct 2022 14:49:54 GMT

A few of my services are down too including Amazon (Ring etc.) and Switchbot, down here in Bristol. It's not affecting services connected to the house in Manchester (also BT), but I can only access those on my mobile if I'm not on wifi routing via BT or EE; e.g. I can access my online Ring devices via my mobile on Vodafone.

Re: Backbone Routing Issue

smartie — Mon, 31 Oct 2022 14:56:28 GMT

just looked on here about this as all my ring cameras were offline and even the ring site won't load.

Re: Backbone Routing Issue

Weerab1983 — Mon, 31 Oct 2022 14:58:37 GMT

Same for me with ring devices in Scotland.

In USA at the moment but ring devices are showing as offline on the app.

Re: Backbone Routing Issue

Will168 — Mon, 31 Oct 2022 15:25:09 GMT

My services seem to be back online now.

Re: Backbone Routing Issue

JMo — Mon, 31 Oct 2022 15:29:30 GMT

Same here

Re: Backbone Routing Issue

NeilJonesOnline — Mon, 31 Oct 2022 15:35:28 GMT

Back on for me too, though it's only now everything's working OK again that I've discovered it wasn't an issue with my own home network, I've been tweaking, powering off/on and googling for the last few hours.

It's surprising how much I've learned about network management while trying to fix this!

Re: Backbone Routing Issue

KernowImmigrant — Mon, 31 Oct 2022 15:37:03 GMT

Same for me now as well.

Re: Backbone Routing Issue

Weerab1983 — Mon, 31 Oct 2022 15:38:51 GMT

Thunderbirds are go..

Re: Backbone Routing Issue

DavidM — Mon, 31 Oct 2022 16:15:04 GMT

Thanks all for posting.

@KernowImmigrant can you let me know if this is sorted for you?

Thanks everyone else for the updates.

Cheers

David

Re: Backbone Routing Issue

ptan — Tue, 01 Nov 2022 00:07:18 GMT

All ok later this afternoon - thanks to whoever fixed the issues

Re: Backbone Routing Issue

KernowImmigrant — Wed, 02 Nov 2022 08:48:36 GMT

After 4 hours, it was resolved. Dissapointingly not 1 person from BT acknowledged that there was a fault despite the clear evidence saying there was.