You question on this was timely for me as I had already placed the order for the rivets.
1st. I drilled the heads off of the rivets and used a center punch to punch them into the tiller arm:
2nd. I slid the connector out of the arm and disassembled them. Note that the nylon washer is severely deformed and I believe this is the reason for the slop.
A view of the bolt inside the connector:
Note: I have a stud sticking out of my connector. I dont keep the boat setup and I found that the rudder assembly was the most dangerous thing I have ever handled (not really, but it was awkward), so I came up with a solution. I replaced the bolt with a stud and I made a ball with a nut glued into it to secure the tiller assy. It really has made handling the unit much safer.