That’s a little confusing as you say you never had a remote and then “the remote does not work”.
I’m going to guess that at one time, you set your roku up (with the remote that came with it), then lost it, then started using your phone. Then you factory reset your Roku, which removes all information from it, including any knowledge of your Wi-Fi network. So, before your phone will work as a remote, you need to get the Roku connected to the Wi-Fi again. That requires an actual remote, not a phone.
Whether you can use a non-Roku remote is going to depend on the model. In general, most third-party remotes are IR-only. “Box-style” Rokus can work with IR or RF remotes. Roku “stick” models only work with RF remotes. (That is to say: Stick models generally require a Roku-brand voice remote.) In the past, Roku has made a few models with odd-ball remotes but chances are you don’t have one of those.
Banned but back. Because why not?