How to remove the end cap of a spinnaker pole

I need help removing the end cap and u-bolt/pin of a spinnaker pole, the end of the pole that attaches to the trampoline's front crossbar. I have already removed the rivets from both sides. The end cap does not seem to be affected in any way, as if it were glued to the pole. Would I need to remove the bolt first, and if so, how?

Thank you very much for all your help.