You need to connect it to a VDSL modem, like the white Openreach one, not the home hub 5, as that is a modem/router combined.
The ASUS RT-AC68U is just a wireless router and needs to be connected to a WAN not a LAN.
it's not the modem you need to be concerned about it is the router which you connect to it
If the Hub 5 is a" Type A" you can put in a sort of modem only mode.
The best option as others have highlighted is to use an openreach modem ( the unlocked ones have all the sats) or return the asus to currys where they should of recommended the DSL version of the Asus (DSL-AC68U)
Your Asus RT-AC68U hooked up an unlocked hg612 3b openreach modem does work with bt tv and its a very popular set up on here.
You need to connect it to a VDSL modem The ASUS RT-AC68U is a Wi-Fi router. What you need is The ASUS DSL-AC68U witch is AN ADSL2 & DSL modem Router, so you don’t need the O/R Modem plug in your cable into the DSL RJ45 colour Blue for both ADSL2 or DSL when you connection DSL light stops blinking your connected to BT log in to the router user name is admin no P/W on the top left side when page opens use Asus quick setup, it defaults to PPPoE. & follow on screen info.