Yeah. Axes have the swing:stab factoy (forgot the ratio) where a swing with the axes does 120% dmg, and a stab does 80% (?), which makes them stronger but less stable (as in average damage output). When you see the animations for Brandish, you'll notice that it's 2 swings and a stab. The stab will hurt your damage if you're using an axe. But for swords, swings and stabs do the same damage. Thus Brandish will stronger and more stable with swords (assuming same equips and average clean weapons of that level).
Also another factor is that axes are waaayyy cheaper compared to swords. Hence it's easier to get a much stronger axe, or mass scroll your own, than to get an average sword.
Something like that...