Prepare a charcoal or gas grill for medium heat (350° to 450°; you can hold your hand 5 in. above cooking grate only 5 to 7 seconds). Set a cooling rack over half of cooking grate. In a small bowl, mix potatoes and 1 tbsp. oil. Spread potatoes on rack and grill until browned, turning occasionally, about 10 minutes.
While potatoes cook, make sauce and cook fish. For sauce, in a small bowl, mix capers, lemon juice, zest, and mayonnaise. Set aside.
Rub fish on both sides with remaining 1 tbsp. oil and sprinkle with 1/2 tsp. salt. Using a wad of oiled paper towels and tongs, oil clear portion of cooking grate. Lay fish on grate and cook, turning once, until grill marks appear and fish is opaque, about 5 minutes total.
Transfer potatoes and fish to a platter. Sprinkle potatoes with remaining 1/2 tsp. salt and the parsley. Serve with caper mayonnaise and lemon wedges for fish.
Note: Nutritional analysis is per serving.