Hello Emma,
It is winter season and Ryokan with Onsen are pretty much full, I will really not advice to try to go there without any reservation if you want something with nice bath.
To reserve have a look at this post :
How to use Jalan and try to use it.
You can also use it and you will see the availability of the different hotel/ryokan.
To help you choose, you can always check our website for onsen :
Secret-Japan / Onsen.
Bye
PS I edit your title and change Ryoken to Ryokan, this will be better when using the search engine later with more appropriate writting (旅館 りょかん ryo ka n)