Use a table knife to cut the butter into 1-inch pieces and let stand in a medium bowl at room temperature until softened, about 1 hour. Heat oven to 350° (or 325° if using a glass pan) and adjust the oven rack to the center position. Lightly grease the bottom and corners only of a standard-size loaf pan.
Peel the bananas and place them in a medium bowl. Mash with a fork until mostly smooth, then drizzle the buttermilk directly over the bananas and continue mixing with the fork or a whisk until fully blended. Set aside.
Crumble the brown sugar into the bowl with the softened butter. Using a handheld electric mixer on high speed, or a whisk or wooden spoon, beat the butter and sugar together until light and fluffy, scraping down the sides of the bowl as needed. Add the eggs one at a time, beating well after each addition. Add the vanilla and continue beating for 1 to 2 minutes until well combined.
In another medium bowl, combine the flour, salt, baking powder and baking soda and whisk to blend. Add about half of the dry mixture to the butter mixture and stir with a wooden spoon. Stir in about half of the banana-buttermilk mixture. Repeat with the remaining dry ingredients, followed by the remaining banana mixture, stirring gently from the bottom of the bowl after each addition just until combined, being careful not to overmix.
Transfer the batter to the prepared pan, scraping the bowl with a rubber spatula, and spread evenly. Bake for 50 to 70 minutes, or until a sharp knife inserted into the center comes out clean. Let the bread cool in the pan for at least 15 minutes before removing. To release, tap the pan firmly on the counter and slide the loaf onto a cooling rack. For best results and to prevent crumbling, allow the bread to cool at least 20 minutes longer before slicing and serving.