Preheat the oven to 350 and line 24 muffin tins with cupcake liners
Empty the cake mix into a large mixing bowl. Make the cake batter according to the directions on the box, but with a few substitutions -
Use the same amount of eggs and oil
Use milk instead of water and add 3 tbsp. more than the box suggests
Add a tsp. of vanilla extract
Add 3 packets of hot cocoa mix
Mix well with a stand or hand mixer until batter is smooth. Because of the hot cocoa mix, this batter will be thicker than usual.
Use an ice cream scoop or measuring cup to fill cupcake wrappers about 2/3 full.
Bake for 17 minutes, then remove from the oven and let cool completely.
Meanwhile, make the frosting by creaming together the butter and vanilla.
Add the sugar, starting with about 2 1/2 cups, and mix well. This will be very thick.
In a small separate bowl, mix 1/2 cup of cold milk with 3 tbsp. vanilla instant pudding mix, and whisk until smooth and thickened.
Add the thickened pudding mixture into the butter and sugar and mix until well combined.
With the mixer on low speed, slowly pour in the whipping cream.
Turn mixer to medium speed and mix until all ingredients are combined and smooth. The frosting should form stiff peaks, but be pretty light. If it is too thin, add a little more powdered sugar and if it is too thick, add a tbsp. or so of milk and mix again.
Spoon into a piping bag or the corner of a gallon size freezer bag.
Pipe the frosting on top of the cooled Hot Cocoa Cupcakes.
Sprinkle the top of the frosting with some mini marshmallows and mini chocolate chips.
Push your straw right into the top of the cupcakes, and you are done!