Preheat oven to 350 degrees. Line cupcake pans with paper liners.
In a medium bowl, stir together flour, baking soda, baking powder, salt, cinnamon, ginger, and nutmeg.
In a separate large bowl, whisk together, brown sugar, sugar, butter, and eggs. Gradually add in the dry ingredients and whisk until smooth. Stir in the pumpkin puree.
Divide batter evenly among 20 cupcake liners, filling approximately halfway. Bake 20 to 25 minutes, until a toothpick comes out clean. Cool completely on a wire rack before frosting.
In a large mixing bowl, beat butter until fluffy. Gradually add in confectioners' sugar, and continue beating until fully combined. Beat in salt, vanilla, and 3 Tbsp of milk. Add additional Tbsp of milk at a time until you reach desired consistency. Beat on high until smooth and fluffy.
Ice the cooled cupcakes as desired.