This port is solidly industrial. You can walk around the giant parking lots, out to Harbor Drive. There are a couple of tourist spots within easy walking, and you'll be able to see the interesting harbor activity, but it's all pretty gritty.
One cool thing in the port itself is the Battleship USS Iowa museum. If you're taking a cruise that arrives or departs from San Pedro, you might budget some time for this attraction. (http://www.pacificbattleship.com/)
Ports O' Call Village is 1.1 miles away, to the south. It's an older tourist stop, alas, not the destination it was in the 60's and 70's. There are some souvenir stores and restaurants. The TripAdvisor reviews tend to generally negative, but without a lot of walking destinations here, it could be worth a look.
From the parking lots, walk left until you can take a left on Sampson Way, (which parallels S. Harbor Drive,) and a slight left on Nagoya Dr.