The official USAFA website has some great information regarding visiting the Academy during the academic year. There are public areas that are open year round, and there are secure areas that may or may not be open, and you may or may not need to be escorted by your Cadet. Click this link to go the USAFA Website.
It is important to remember that Cadets can not come and go as they wish. In order to leave the Academy, they must get Leave approved and sign out. There is always the risk that you may plan a trip to the Academy only to find out that your Cadet is restricted that weekend. Often they will know ahead of time. Sometimes they will not. You just need to accept this.
If your Cadet is restricted, they may still be able to meet you at Arnold Hall. Arnold Hall is interesting because it is part of the Public Area for visitors, but it is also still in the Cadet Area. However, it is not uncommon for Cadets to be restricted from Arnold Hall as well.
Unless your Cadet is an IC (Intercollegiate Athlete) and they have a game during the week where they will be off the Hill, it will be very difficult if not impossible to spend time anywhere with your Cadet during the academic week. The best times are on unrestricted leave weekends. Your Cadet will quickly learn the best times.
Finally, if you do visit and have the opportunity to go off base with your cadet and they spend the night with you, don’t feel put out if they want to rest or sleep a bit more than you expected!
Here is a link to a document put together by a Parent. It has some great information, but please do not rely solely on it as things do change. USAFA Info for Parents and Cadets