The decisive blow came from the bat of Jason Michaels, whose two-run walk-off home run sent 29,387 fans at PNC Park into a frenzy.
"It got away, but I give the Pirates credit," Cardinals manager Tony La Russa said. "They've done that to us a couple of times and we know that they are going to play nine innings."
Cards right-hander Chris Perez relieved Kyle McClellan in the bottom of the 10th after a single by pinch-hitter Raul Chavez, and gave up the two-run shot to Michaels. It was Michaels' fourth home run of the season.
"He hit a fastball," Perez said. "I looked at it as just another inning and tried to get three outs and hopefully try to get a ground ball and get out of it quicker, but it just didn't happen. It was a fastball over the plate, and he put a pretty good swing on it."
Troy Glaus appeared to have stolen the game for St. Louis in the top of the frame, when he connected for a solo home run off Bucs reliever Denny Bautista to lead off the inning. It was Glaus' 14th roundtripper of the season and fourth hit of the game.
Cardinals hitters had a feast at the plate all night, knocking out a season-high 22 hits. Wellemeyer, who had dropped three of his last four starts, pitched 6 1/3 innings and allowed four runs on four hits. Perez took the loss, falling to 2-1.
"It's unfortunate we got beat in a game where we scored 11 runs -- that, to me, is the most unfortunate part," La Russa said. "We scored 11 runs and  hits and we got beat, but they scored 12, and 12 is the magic number."
The Pirates pecked away at 10-4 Cardinals lead in the eighth, getting two runs off right-hander Ryan Franklin on a two-run homer by Jason Bay, his second blast of the game.
Cardinals reliever Jason Isringhausen was protecting a 10-6 lead in the ninth inning when he surrendered a three-run home run to Nate McLouth, making it 10-9. The Pirates pushed across the tying run after Luis Rivas, who had singled, went to third on a Ryan Doumit single and then scored on Bay's ground-ball force out to tie the score 10. The Cards almost ended the game at that point, but couldn't turn the double play.
"Isringhausen walked a guy with a four-run lead," La Russa said. "He made a great pitch to Bautista and walks Michaels, which is not good baseball, and then he got a tough break on an infield hit and then he got the ball up -- a three-run homer."
"You don't want to walk anybody," Isringhausen said. "They got a dink base hit and a home run -- so that's three runs on the board and we're still up by one run and just couldn't get it done."
Pirates right-hander Yoslan Herrera, making his Major League debut, gave up six runs and 11 hits in 4 1/3 innings.
Bautista was credited with the win, upping his record to 3-1.
Pittsburgh gave Herrera some early run support in the bottom of the first inning. After a leadoff walk to McLouth, Bay cracked a two-out, two-run homer -- his 18th of the year -- giving the Pirates a 2-0 lead.
The Cardinals got to Herrera in the top of the third inning. Aaron Miles singled and scored on Ryan Ludwick's RBI triple. Albert Pujols then lofted a deep sacrifice fly to left scoring Ludwick, and Yadier Molina singled, plating Rick Ankiel to stake the Cards to a 3-2 lead.
Ankiel, who finished with a 2-for-5 night with an RBI, had to leave the game in the seventh inning after spraining his left ankle. The injury isn't expected to be serious.
Ludwick struck again in the fourth inning, as he tagged Herrera for a two-run homer to left, upping the Cards lead to 5-2. It was Ludwick's 20th home run of the season.
Pirates third baseman Doug Mietkiewicz was ejected by second-base umpire Eric Cooper after arguing a close play at second on a double by Pujols.
St. Louis tacked on four more runs against the Pirates bullpen with two in the sixth, one in the seventh and one in the eighth, but it wasn't enough to offset the Pirates comeback.
"It's just a reminder that this game is never done till it's over," Miles said. "Don't take it for granted, and there's no time limit and you can't run out the clock. It's never over till it's over, and that's what makes the game so great."
George Von Benko is a contributor to MLB.com. This story was not subject to the approval of Major League Baseball or its clubs.