The Xbox Series S is compatible with all Xbox One controllers, so you can use an older gamepad with your headset adapter.
the Xbox Series X|S is worth considering as your primary streaming device as well as a games console. But if that’s the case you’ll probably want a proper remote to flick through your entertainment ...