If you download the installation media for Windows 10, you should be able to activate it using the licence code for your existing Windows 8.1 installation, in fact if you upgrade in place, then this happens automatically. You can always do a clean install later, and it will still activate.
You should be given the option to upgrade your existing installation and keep your files.
As far as I am aware, you can still upgrade for free, but another forum member may be able to confirm this.
Unless you computer is less than 2/3 years old, the chances are it will not be suitable for Windows 11 and you will always be limited to Windows 10 and never get Windows 11.
Have you had a look at windows update? The Windows 10 upgrade may be waiting for you to accept the upgrade.
Your computer will not automatically upgrade from Windows 8.1 to Windows 10 or 11.
You can still upgrade for free to W10 but I doubt your computer will be able to run W11.
If you want to upgrade to W10 see this link. You have to follow the instructions carefully to ensure that the update will take place. If your computer has a valid W8.1 licence it should validate the W10 install without you have to manually enter the licence key.