The spurting of your spray is due to an undersized compressor which is causing the spray gun to starve for air and causing the fluid to not break up well. You will want 20 CFM to be safe for most decent HVLP spray guns. For gelcoat as it is thick usually a pressure pot and a conventional spray gun can be easier to use. Usually you will spray about 4 coats each being about 10 mils per application and let each coat cure prior to the next coat if your recoating an existing gel coat. Sand between each coat for adhesion.
https://www.pittsburghsprayequip.com/2017/04/21/spraying-gel-coat-complete-guide/