Generally speaking you want conditioned cooled air, not just cooled air. A huge part of an AC system (and, a fridge is basically a small AC unit) is dehumidify

Generally speaking you want conditioned cooled air, not just cooled air. A huge part of an AC system (and, a fridge is basically a small AC unit) is dehumidifying as well. Functionally, you'd now need to add a de-humidification circuit to the system if you used outside air, since you don't have any humidity control.