McGehee homered with one out in the ninth inning to give Milwaukee an 8-7 victory over the Cardinals, preventing St. Louis from celebrating another late-game comeback.
McGehee hit a 2-1 pitch from Kyle McClellan (0-1) into the left-field bleachers to win the game after the Cardinals had rallied for three runs with two outs in the top of the inning off Milwaukee closer Trevor Hoffman (1-1).
There were seven home runs hit in the game, but McGehee's was the one that gave Milwaukee the victory. McClellan had just entered the game to face the Brewers third baseman.
"In my eyes, one pitch lost it," McClellan said. "I came in a tie game. My job was to get two outs and I didn't do it.
"As far as I'm concerned, one pitch cost our team that fought and clawed to get back in it," he said. "Our bullpen pitched our butts off and I come in and blow it on one pitch."
For the second time in the three-game series, the Cardinals rallied off Hoffman in the ninth. This time, it was consecutive two-out home runs by Pujols and Holliday.
"In the ninth inning, we're facing probably one of the best closers in the game, Hoffman," Pujols said. "It's tough to score one run against the guy. Imagine three. But we didn't give up."
With the Cardinals trailing 7-4, Felipe Lopez hit a one-out double down the right-field line. After pinch-hitter Nick Stavinoha popped out, Pujols slammed his second home run of the game to bring the Cardinals within 7-6. Holliday then hit a 2-1 pitch that hit the batter's eye in center field to tie the score.
"When you throw 85 mph right over the middle of the plate, that's usually what happens," Hoffman added, chuckling. "Not to be funny about it, but it's embarrassing."
Stavinoha hit a two-out, two-run homer off Hoffman in the ninth inning on Friday night to give the Cardinals a 5-4 victory.
The late-inning heroics saved Chris Carpenter from picking up his first loss of the season. Uncharacteristically, the Cardinals ace gave up seven runs, five earned, on seven hits and two walks over five innings. He struck out four, but allowed three home runs.
Like Hoffman, Carpenter said he had trouble putting his fastball in the right spot to the hitters.
"I don't care who you talk to, how hard you throw or what you do, if you don't execute and locate your fastball, you're not going to be successful," he said.
Carpenter did not have his usual velocity in the first inning, but it picked up in the ensuing frames.
"There's no question it wasn't coming out the way it typically does, but in that situation, I'm not going to overthrow the ball and start heaving," he said. "I was trying to execute. When I executed, I got outs. When I didn't, they hit 'em a long way."
Last season, the right-hander allowed only seven homers in 192 2/3 innings. So far this year, he has yielded five in two starts.
"This year is a new year," he said. "You have to go out and execute pitches and get outs. I have to get the ball down in the strike zone. I'm not doing it right now."
Carpenter threw 88 pitches over five innings, giving up seven runs, five earned, on seven hits and two walks. He struck out four.
The last time he allowed three home runs in a game was Aug. 9, 2006, an 8-7 loss at Cincinnati, a span of 44 appearances.
"They got to Carp and that's the way it goes," Pujols said. "Next time, it might be Carp's turn.
"It's just one game. I don't think he's going to look down on himself. I'm pretty sure he's disappointed, but you know what? It's going to happen. He's not an automatic pitcher."
There were some other bright spots for the Cardinals, who finished an opening road trip with a 4-2 record.
Joe Mather made a spectacular diving catch at the warning track of a Corey Hart fly in the third, saving two runs from scoring. Allen Craig singled in the eighth for his first Major League hit.
Meanwhile, backup catcher Jason LaRue suffered a strained right hamstring running to first on a groundout. His status is day to day.
Joe DiGiovanni is a contributor to MLB.com. This story was not subject to the approval of Major League Baseball or its clubs.