Frazier was 3-for-4 with two singles and a double. John Jaso added a solo homer for Pittsburgh.
The Pirates (18-23) pounded out 13 hits against four Nationals pitchers. They have won four of their past five games and took two of the three in the series with Washington (25-15), the top team in the National League entering the day.
Pittsburgh starter Tyler Glasnow (2-3) allowed four runs -- two of them earned -- on three hits over five innings, with three walks and six strikeouts.
Nationals starter Tanner Roark (3-2) gave up seven runs on eight hits, including both homers, over five-plus innings, with four walks and three strikeouts.
Bell hit his team-high ninth homer with two outs in the second. The blast to right drove in Frazier, who led off the inning with a single, for a 2-0 Pirates lead.
Daniel Murphy followed in the second with a one-out solo homer to right to bring Washington within 2-1.
Pittsburgh tacked on two more in the third for a 4-1 lead on Roark's bases-loaded walk to Jaso and a fielders' choice grounder by Jordy Mercer.
The Nationals tied it in the fourth on Adam Lind's RBI double and, with two outs, Glasnow's throwing error that allowed two more runs to score.
Jaso's two-out homer into the bullpen in center put the Pirates back in the lead, 5-4, in the fifth.
The Pirates chased Roark in the sixth after Chris Stewart singled and Gift Ngoepe walked. Frazier greeted reliever Blake Treinen with a two-RBI double to center, and Josh Harrison's sacrifice fly to left brought home Frazier for an 8-4 lead.
In the seventh, Pittsburgh's Danny Ortiz hit a one-out single to right for his first major league hit. He and Stewart, who singled, later scored on Frazier's base hit to make it 10-4.
NOTES: In South Korea, a court upheld Pittsburgh 3B Jung Ho Kang's suspended sentence for a DUI conviction. That casts into doubt when he might be available to the Pirates. ... Pittsburgh gave C Francisco Cervelli a day off from starting. ... Washington gave LF Jayson Werth and 1B Ryan Zimmerman the day off from starting, although Zimmerman entered in the sixth. ... Pittsburgh and Washington don't meet again until the final series of the regular season.