Lightly grease an 8x8" baking pan and line with parchment paper with overhangs on opposite sides. Set aside.
Melt the butter in a medium saucepan. Remove from the heat. Add the brown sugar, golden syrup, condensed milk and stir until combined with the butter. Return the saucepan to a low heat and cook for 10-12 minutes constantly stirring, until the sugar has dissolved (DO NOT BOIL). Stir in the salt and ginger.
Remove from the heat and add the white chocolate. Stir until combined and the chocolate has melted. The fudge will start to set quite quickly, so immediately pour into your prepared pan, then spread evenly with the back of a spoon. Allow to cool briefly, then place in the refrigerator to cool completely and become firm (at least 2 hours).
Cut into small pieces to serve.
Keep the fudge in an airtight container in the refrigerator for up to 1 week.