Knowing the language is going to help you out tremendously while abroad. As you said… even if you do have basic knowledge of the language before hand; that will all change once you get there. That is because speaking the language in the country it is spoken, especially with locals, is completely different from learning it back home.
It is not impossible, it just takes practice, patience, and the ability to laugh at yourself a little while you make mistakes! Language learning is actually really fun! While learning Spanish in Spain I realized how much I could pick up, how quickly I could understand what others were saying, and how my conversation skills grew immensely! It does take some time to be able to be completely “fluent” but after being completely immersed in Spanish, all day long, for just a few days, I realized that I was well on my way! It was such a great feeling of accomplishment to be able to converse with the locals- and dreaming in Spanish proved to me that I was really understanding the language!