Eighteen points from the boot of Dan Biggar proved critical as the Ospreys beat Ulster 23-22 at Liberty Stadium.
But the Welsh region were forced to come back from 13-0 down in the first half and 19-14 down at the start of the second to secure the spoils.
Ulster started well opening up a 6-0 lead after 11 minutes through two penalties from outside-half Ian Humphreys.
It was not a great start from the Ospreys who then conceded a penalty try on 15 minutes.
Scottish referee Neil Paterson awarded the score when wing Tom Prydie tackled Humphreys without the ball as he chased a ball across the line.
But after a difficult first quarter the Ospreys launched a comeback. Biggar put over a 20th-minute penalty and that was followed six minutes later when flanker Marty Holah was sent over in the left corner from a long pass from centre Ashley Beck.
With the Ulster pack getting penalised for a second time at a set scrum Biggar kicked a penalty to close the gap to just two points.
And Ulster's cause was not helped when Humphreys was forced off with a hand injury on 32 minutes to be replaced by Paul Marshall.
There was worse to follow for the Ulstermen just before half-time when Tim Barker was sin-binned for throwing a punch on the floor with Biggar converting the subsequent penalty from five metres inside his own half.
It was the last kick of the first period as the Ospreys led 14-13.
Despite being down a man Ulster were kicked back into the lead with a Ruan Pienaar penalty three minutes after the break following a line-out offence, and a further six minutes later the South African was on target again, this time from just outside the 22.
But back to their full complement Ulster gave away three penalties with Biggar kicking penalties on 57, 64 and 67 minutes to give the Ospreys a 23-19 lead.
With seven minutes left on the clock Ulster earned a penalty goal in front of the Ospreys posts, which Pienaar kicked to make the score 23-22 going into the business end of the contest.
Biggar missed a penalty goal deep in to injury time but it mattered little as the Ospreys held on for victory.
Pen: Biggar 6
Pen: Humphreys 2, Pienaar 3
Ospreys: 15 Richard Fussell, 14 Tom Prydie, 13 Sonny Parker, 12 Ashley Beck, 11 Hanno Dirksen, 10 Dan Biggar, 9 Rhys Webb, 8 Jerry Collins (capt), 7 Marty Holah, 6 Tom Smith, 5 Ian Evans, 4 Ian Gough, 3 Cai Griffiths, 2 Huw Bennett, 1 Duncan Jones.
Replacements: 16 Mefin Davies, 17 Ryan Bevington, 18 Craig Cross, 19 James King, 20 Justin Tipuric, 21 Jamie Nutbrown, 22 Gareth Owen, 23 Tom Isaacs.
Ulster: 15 Adam Darcy, 14 Craig Gilroy, 13 Nevin Spence, 12 Ian Whitten, 11 Simon Danielli, 10 Ian Humphreys, 9 Ruan Pienaar, 8 Pedrie Wannenburg, 7 Willie Faloon, 6 Chris Henry, 5 Tim Barker, 4 Johann Muller (capt), 3 Declan Fitzpatrick, 2 Nigel Brady, 1 Bryan Young.
Replacements: 16 Andi Kyriacou, 17 Jerry Cronin, 18 Alan Whitten, 19 Ryan Caldwell, 20 TJ Anderson, 21 Paul Marshall, 22 Luke Marshall, 23 David McIlwaine.
Referee: Neil Paterson (Scotland)
Assistant referees: David Jones (Wales), Greg Morgan (Wales)
Television match official: Huw Lewis (Wales)