Now, this could be completely wrong and only so applicable to sins but I think the same basic ideas apply. I'm not sure how dex affects the dmg formula for dits but this:
Basically, 100 is the highest you should cap your dex and not have completely fail dmg. To decide what to cap at look at what you want your end-game wep to be. How much dex does it take to use? Now take how much dex you get from equips from that and you'll have your cap.
Also, get in the market to either a) make a good pair of dex boots or b) buy a good pair of dex boots. Of course dex BSS will run several tens of millions as soon as you get up to 5+ dex. A cheaper alternative is to look for regular thief boots that start with dex (most also have luk, which is a nice bonus) and start scrolling them.
Oh, and when you say cap are you talking about base or total? I'm talking about base dex when I say cap.
.......