To keep your drinking water clean in a hotel room, rinse out your drinking glasses, even if there is a paper cover or a plastic wrapping around it.
Dust and debris from the plastic can settle in the glass, leading to an unpleasant first sip.