In those very popular onsen towns, the only kashikiri baths available are for people staying in expensive ryokan overnight.
If this is what you are looking for, and are ready to pay probably around 16000-20000 yen per person, you might want to check this web site, in Japanese only :
http://www.kashikiri-onsen.com which has a comprehensive list of private baths.
Alternatively, there is the ryokan Fujiyoshi (check on the recent thread of this forum), which is located south of Atami and has a great onsen with several baths you can reserve for youself. And you can just go there for a bath, without staying overnight, so this is rather a cheap option...