You might as well go low DEX. Brawlers/TBs don't need a lot of DEX to equip their knuckles and it's pretty easy to get at least 20-30 points of DEX from equipment even in 2nd job. 10 DEX bathrobes, Zhelms, agent equips...
Also, I'm keeping my TB DEXless through most of 1st job as I don't have a reason to add DEX. I'll need to use Straight eventually, but other than that? I get more than enough accuracy from the 20 advancement DEX + Blessing + Quick Motion, and I used the beginner knuckle until level 20 (didn't even bother to get a higher attack weapon for SSK use) because the quests bring you up to level 20 so fast there's no point, especially with 1 in Sprite to help you kill. Then, you can use a Fish Spear to train with SSK.