A couple of things to try before you take it back for a replacement (which may be the easiest thing to do)...
Make sure you're using the supplied power adapter and not your TV's USB port. If you're already doing that or it doesn't help, find the pinhole reset button and with a paperclip or something similar press and hold the reset button for no less than 30 seconds with the power connected to the Express.
You may have better luck getting through the initial setup using your phone's hotspot as the Wi-Fi connection. Once you get through the setup try your Wi-Fi again. If you still can't set it up, the last thing to try is a different HDMI cable even though it would seem to be completely unrelated.
Roku Community Streaming Expert
Help others find this answer and click "Accept as Solution."
If you appreciate my answer, maybe give me a Kudo.
I am not a Roku employee.