I suspect that the default H.264 profile in Corel VideoStudio is the "baseline profile," for which the max supported resolution is 720x?, hence the error above.
To work around this issue in Corel (the approach would be similar in other software packages) one needs to select a H.264 main profile, which supports higher encoding resolutions. Again, in Corel, choose Create Video File / Custom option. Then, select "Save as Type" = *.mp4, click Options, select Compression tab, and set video type to H.264-Main.
Finally, go back to general and notice new (higher!) resolution options became available. Choose the one you want. I was happy to enjoy the 1280x720 option.
Happy encoding!