For the most part, it comes down to control of proprietary binaries of the hardware itself.
Also, they are slowly working towards that end goal:
You can get a Generic System Image, allowing you to install just about any variant of Android. What keeps you from installing other operating systems is actually the kernel.
However, they are now working on a GKI, Generic Kernel Image.
Even under the current way of Android development, you can install other operating systems, it just requires that you A. build the kernel with needed features, and B. have all the appropriate binaries and libraries for the running the hardware.
Most phone manufacturers hold little power over the binaries that they have to use for their hardware, as they didn’t invent the hardware. Usually, they are just putting together other peoples hardware into a package. With that, they either cannot release the source code for the binaries, because they are bound by the hardware manufacturers not to, or they will not, to keep an “edge” over the competition.
I once took a motorola flipside phone, and turned it into a debian computer. It worked great as a computer, but all the hardware had driver support in the kernel, and the kernel could support the old version of debian that I was running on it back in 2016.
If you are board, you can read about that here: Using your i927 as a Debian “Chatter” Phone – AlaskaLinuxUser's Scratchpad
Since then, the kernels have gotten more specific for Android’s needs, and don’t have what you need in them for running other OS’s, like GNU Linux, etc… That is why, if you want to install Ubuntu Touch, you have to rebuild the kernel to include support for several things that you use for running a GNU Linux operating system like Ubuntu.
So, recap, and TL;DR
One of the main limiting factors is kernel driver support for hardware.
To make it truly effective, they need to convince hardware manufacturers to build hardware that has a minimum compatibility. Like your desktop computer, if you install an OS that doesn’t have graphics driver support, it still displays, just in some ridiculously low resolution with low color support. All of these phone ones are super specific and don’t work that way. If you don’t have the drivers, they just don’t work at all.
All that said, just having the ability to install/flash any OS you want doesn’t really help, because most desktop OS’s are not very handy as a cell phone.