The Brethren escaped from Llewellyn Motors Field with a 34-16 win but Swifts rattled them in the second half, coming back from 24-0 to trail 24-16.
Brothers were much more clinical in the first half and the early exchanges highlighted the difference between the two teams this season.
While Swifts were profligate, Brothers were clinical in their execution, taking the opportunities available.
Swifts twice got to the last tackle close to the Brothers line in the opening minutes but failed to fire a shot and died with the football in their hands.
Brothers, on the other hand, twice exploited Swifts' defensive frailties on the left-hand side to score tries through winger Ben Seve and halfback Jason Connors, who was backing up a Seve break.
On both occasions Seve got outside his opposite winger Rendy Mam and into the clear.
So it was perhaps no surprise when Mam found himself defending a two-on-one, he fell for Brook Alofipo's dummy to Seve, which put Brothers ahead 18-0 after 22 minutes.
Despite the scoreline, Swifts were working hard and their defence was impressive, with second rower John Maila in particular pulling off some big hits.
But Brothers continued doing what has worked so well for them and it was 24-0 when Liu Faamate scored from close range.
Swifts found a glimmer of hope when Shane Williams dived over from dummy half to make it 24-6 at halftime.
The Bluebirds started the second half like Brothers did the first with Zeb Nuuola scoring and it was soon clear the momentum had shifted to Swifts.
Brothers tried all they could to slow down the Bluebirds but conceded penalties in doing so and from one close to their line, Swifts prop Tele Salesa charged over to score.
Brothers' frustrations were symbolised when winger Lachlan Tuite reacted to a tackle by throwing a punch at opposite winger Kieran Saltner.
Saltner's response earned them both 10 minutes in the sin-bin.
The break in play seemed to take the wind form Swifts' sails and allow Brothers to regroup, ending the Bluebirds' comeback.
"They gave it to us in the middle bit of that second half," Alofipo said.
"I think all the injuries we've had have taken their toll.
"We stuck to our normal structures in the first half.
"In the second half we fell away, until we regained our composure."
Alofipo didn't believe Swifts had exposed any inherent weaknesses in the Brothers make-up.
If anything he thought the fright might serve them well in the long term.
"There's not too much to worry about," he said.
"It is good to play some hard games.
"Every team is a test but we got showed up a bit today."
Swifts fullback Robbie Fletcher thought his team would take confidence from the game, if not the result.
That is despite the loss making Swifts vulnerable to a late charge from Fassifern for the final semi-finals spot.
"We stood up and got under their skin," Fletcher said.
"But you can't give them a sniff really."
In yesterday's other A grade match, Norths beat West End 64-6 at Keith Sternberg Oval.