Preheat oven to 350°F. Grease and line a 9x5-inch loaf pan with parchment.
In a medium bowl, whisk flour, baking powder, baking soda, salt, cinnamon, and nutmeg.
In a large bowl, beat butter, granulated sugar, and brown sugar until creamy.
Add eggs one at a time, then mix in vanilla.
Add sour cream and milk, mix until smooth.
Add dry ingredients to wet, stirring just until combined.
Fold in apples and nuts.
Pour half the batter into the pan. Add half the topping. Repeat with remaining batter and topping.
Bake 55–65 minutes, or until a toothpick inserted comes out clean.
Cool 10 minutes in pan, then transfer to a wire rack to cool completely.