He didn't give up >30 mins. He gave up 30 mins minus however long it took Nelson to finish the rope climb. The 30 minutes was the max time, not additional time added once you decide to give up.
He didn't try the rope again because he said he was getting fatigued and the chance of pulling it off was getting slimmer so he didn't see the point in getting more worn out for what was becoming an increasingly negligible benefit. A rare bit of clarity from someone on the CHALLENGE.
But yeah, the penalty situation confused me. I'm not sure it was well explained.
Not explaining penalties is one of the many tools they use to keep us from figuring out the winner. Same way they strengthened Cohutta's accent to prevent us from understanding how he pulled so much tail.