VivaJ73

Some idiot (me) spilled a considerable amount of oil when filling up after changing the drain plug.

Now I get a weird rattling sound which seems to come from the air conditioning pump or the power steering pump. (Under the oil fill cap).
Could this be because of the oil? Will this go away?

I also changed the idler pulley, but I am pretty sure the belt and belt tension are all ok.

the sound seems to get worse when I put the car in D or R.

BlackML550

Take off the serpentine belt and start the car. If the rattling stops, you have isolated the problem to something the belt drives. Check the belt also. Its possible that oil has soaked into the a/c compressor or alternator (not sure which engine you have) and thats causing the noise. With the belt back on, does the rattling change pitch when you start the a/c? Also put some load on the alternator by switching on rear demister and high beams and see if the noise changes. Carefully use a larger screwdriver as a stethoscope and listen to all moving parts and you'll find the issue.

ChrisHimself

is your idler pulley possibly backwards or is the shim plate that is normally there not installed?