Wear a tether. If you come off the boat in high winds, it can blow away faster than you can swim. Keep a fixed blade knife on your PFD that you can easily access if you need to cut the tether.
Keep a waterproof VHF in your PFD pocket and know how to use it.
Buy a PLB, attach it to your PFD.
Put a strobe light on your PFD.
Carry day/night flares, the kind that make lots of red smoke during the day and bright red light at night.
Most importantly: file a written float plan with someone who cares about you, your shore contact. A good float plan will have the following information:
When/where you plan to start.
When/where you plan to end.
Planned route.
Description of you and your boat.
Your cell phone number.
If you are carrying a VHF or not.
If have a PLB or not.
Contact info for the authorities: USCG, Sheriff, Dept of Natural Resources rangers, Water police, etc.
Communication plan with shore contact: let them know when they can expect to hear from you and if they don't hear from you by some cut-off time that you agree to they will contact the authorities. At that time they can relay the float plan info to the authorities and explain you are overdue.