Are you going for the views or the fishing?
With some research and google maps you can find some really productive fisheries away from the crowds.