Hello,
Could just be a bad route. There are many times where the route traffic takes goes through a router that is having issues. First thing I would do is run an MTR trace route test from a Linux machine at your location to the server IP Address and see if there is a router along the route losing packets - happens all the time. The reason a VPN you are using won't have the same issue is that the VPN may well take a different route as it is going to the VPN's server, and only then releasing your traffic into the wild, which would then take a route from VPN Server to your hosted Server.
I suppose you could see it like you taking your car to work and always going the same route that has pot holes in the road. Instead you take the VPN bus that goes a different route and drops you off, and you then can walk a new route to work from the drop off point.
So worth checking as it might not be BT themselves, but might just be a point between the two points you are trying to go between (you and hosted server) which is not necessarily under BT control, and nothing you or they could do anything about anyway. In my line of work I see this all the time as I work for a bespoke VPN company.
