I tried putting the port in both tabs; that didn't seem to fix it.
By the exe, do you mean selecting the folder for the server under where it says applications in the local tab? Or something with inputting an IP in one of the fields? I tried both of those and neither worked, but I may have misunderstood your advice.
I also tried opening a port in the windows firewall, for both inbound and outbound, allowing as much as possible for the 25565 port and opened one for the router too.