Stir together rums, sugar, and ginger in a saucepan. Bring to a boil, then reduce heat and simmer 5 minutes to concentrate the mixture and allow some of the alcohol to cook off. (You’ll have a generous 1/3 cup.) Remove from heat and let cool to room temperature, about 20 minutes.
Transfer rum and ginger mixture to a blender with pineapple and coconut milk and blend until smooth. Pour into 12 (3-ounce) ice pop molds, dividing evenly. Cover molds, insert sticks, and freeze until solid, at least 4 hours and up to 2 weeks.