Preheat oven to 350°F. Grease and line a 8 x 8 cake pan.
In a large bowl, sift together the flour, cinnamon, nutmeg, cloves and all spice. Set aside.
In another bowl, beat the egg whites until soft peaks form. Set aside.
Cream the brown sugar and butter until light and fluffy.
Stir in the egg yolks until well blended.
Add the baking soda to the buttermilk.
Pour the buttermilk and soda mixture, plus the vanilla, into the sugar and egg mixture. Blend thoroughly. The batter will look curdled. Don't panic!
Using a spatula, gently fold the egg whites into the batter, until no more white streaks are visible.
Pour the batter into the prepared pan, leveling the top with a palette knife.
Bake for 35 - 40 minutes, or until the top is golden and a toothpick insert in the center of the cake comes out clean.
Remove from the oven and place on a wire rack to cool. Allow the cake to cool fully in the pan before frosting.