Can you temporarily apply a redundant 5V ref source to the sensor where you see it varying. That would rule out an intermittent loss of correct ref V. If that fisses it, you can go looking for the cause.

Happy trails, Login to read more.