Preheat the oven to 350°F. Lightly grease a 9×13-inch baking dish.
In a large bowl, stir crushed crackers and ½ cup milk. Let sit 2 minutes.
Add ground beef, egg, garlic powder, onion powder, salt, pepper, and paprika. Mix gently until just combined.
Shape into 6 oval patties (about ¾-inch thick).
Lightly dredge each patty in flour, shaking off excess.
Heat olive oil in a large skillet over medium-high heat. Sear patties in batches for 2 minutes per side. Transfer to baking dish.
Reduce heat to medium. Add butter to skillet. Cook onions (and mushrooms if using) 6 to 8 minutes until softened. Add garlic for the last 30 seconds.
Sprinkle flour over onions and stir constantly for 1 minute.
Slowly whisk in beef broth. Add milk, Worcestershire, salt, and pepper. Simmer 2 to 3 minutes until it starts to thicken.
Pour gravy over patties. Cover with foil and bake 40 minutes. Uncover and bake 10 to 15 minutes more, until thickened and patties are cooked through. Rest 5 minutes, garnish if desired, and serve.