Do I need to leave the USB stick connected for Wi-Fi?

No. The USB CONFIG stick is only used once, to import a network profile into /etc/NetworkManager/system-connections/. After that, you can safely remove it. Home Assistant remembers your Wi-Fi settings without the stick.