Unless you have F-Droid baked into your ROM, you must have "Install from Unknown Sources" checked in settings. This is a (very minor) security vulnerability (if at all). If VLC was in F-Droid, they would build it from the same sources that VLC builds from. If you don't trust them, don't install VLC.
However, if you install VLC from the site, you know that VLC is distributing them (assuming that you install via HTTPS).
The only thing to think about, is if VLC pushes an update, you won't be notified. But you will already be more up-to-date than F-Droid.
Personally, I use VLC from VLC's site.