I didn't see "Show video within the main window" as an option, but after checking "Window decorations" and then relaunching the player, it's back to normal.
I'm not seeing that in the View menu: https://lh3.googleusercontent.com/NQHkWmERK-V8qxU1Ij2TJ5magLSidgid-vwrzDJhuEK54Mt__ScDPJU63ZZHjGEXj_gv4neCc3cjjx_YkJDTYnzwCREcXDT8GUeh1ZOyxECNQHoLVtgrWMsBdSVlrq0aFgrwBqnw_dGpcRE-cVTaQtpjaffa_SekZzgCU-JBNp1h9xbEQBCm28_6YJWPvnHMmamCqjPULrIHNYcuS5wNoxS3PoW6YlMzp7tz...
Not sure with what build the change happened but now, whenever I open any media, the playlist always opens with it and the player controls are attached to the playlist. It's quite annoying because if I try to scrub through a video by dragging the playhead, the playlist window moves to the foreground...