In a medium saucepan, whisk the sugar, salt, starch, vanilla seeds (don't add the vanilla yet if using extract), and egg yolks together until a smooth paste forms. Add the milk and whisk until completely smooth. Place the saucepan on the stove and turn the heat to medium-low. Heat the mixture while whisking constantly, until it just starts to thicken to a paste-y consistency (takes about 6 minutes). Immediately remove from the heat and pour it through a sieve placed on top of a heat proof bowl. Push all the pastry cream through the sieve. Add the butter and whisk until melted and smooth. Place plastic wrap on top so that it is touching the pastry cream. Set the cream in the fridge until completely cooled. Meanwhile make the ganache.