Yes, it does need to hold vacuum from the engine or vacuum source side. When it does an evap test it pressurizes the system and if that valve doesnt hold vacuum it will allow the engine vacuum to be applied to the pressurized area and cause the pressure to go ... Login to read more.