Go down the 138 like your going up to Silverwood,turn left on the only highway/ two lane road...towards Hesperia. Go down 2 miles or so and you will see a sign for the road to the dam on your right.

You have to park in a lot above the dam and I'm almost positive you do not need an Adventure pass, might have to put five bucks in an envelope for parking. There used to be a dirt road access to the back side of Silverwwod, you would need an AP to go and park back there, it's a fire road basically. You will see the entrance to it right before the parking lot on the dam.More than it's likely closed off though.

Are you going for Stripers or Trout(striper food)?