How To Record State Of Xbox/gamepad Controller In Python?

I need to know at a specific time the value of all buttons of an xbox controller. The reason being that I'm building a training set for a neural network, and I'm trying to simultan

Solution 1:

the TensorKart Project has already solved that problem: https://github.com/kevinhughes27/TensorKart/blob/master/utils.py

Solution 2:

I feel like I am making this way harder than it needs to be.

No, this is actually hard. It's hard because you don't need to just know what the gamepad state is at a particular time, you also want to know which gamepad state was used to draw a particular frame. The time that the gamepad state was sampled will always be earlier than the time the frame was drawn, and may be delayed due to latency added by the app itself. The added latency might be constant for the whole app or it might vary between different parts of the app. It's not something you can easily account for.

Your python script is recording gamepad inputs as soon as they are received, so I'd expect it to always run at least a frame or two ahead of the screen captures.

I'm thinking that the issue might be related to threading or subprocesses in one of the packages used to read the controller, but I'm not skilled enough to know how to fix it.

It's probably just latency added by the gamepad input code in the app you're measuring and not something that can be fixed. Most apps don't make any attempt to respond to gamepad inputs as soon as they're received and instead handle them all at once during the per-frame update step. On average, that adds latency equal to half the frame rate.

How to fix this? I think measuring gamepad state from another application is going to be difficult due to the latency issues. If you can, it would be best to instrument the app to record the gamepad state during its main loop, that way you know you are recording what was actually used. On Windows it should be possible to do this by providing your own version of the XInput DLL that can record the current state whenever XInputGetState is called.