For starters, the ideal cave would be formed from rock, such as granite or a similarly “hard” stone. This is mentioned because many “caves” are in fact just deep fissures and are made out of ice or comprised partially of ice. This poses a problem of both cave-ins and also floods. Many “caves” on the slopes of mountains are just fissures with ice surrounding the walls…and not a true cave with a floor, ceiling, and walls that will not dissolve with higher temperatures.
First suggestion: find your state’s BLM (Bureau of Land Management) to learn of resources that will locate and map the known caves in your area/vicinity of your home. I stressed the word “known” because for every cave that has been mapped there are at least 10 that have not been. The ones made of rock/stone are usually more stable. Chances are there is nothing being done in the caves that are on federal or state property. Perhaps there will even be “no trespassing” signs, or they’ll try to keep you from going in the cave.
Your job is to find them and learn about them as much as possible, even if you can’t enter into them. After the SHTF, I seriously doubt the “friendly” park ranger will be concerned with keeping you out of the government-owned cave. The advantages to a cave (besides the obvious of keeping the elements off of you) are worth considering. Caves stay cooler in the summertime, and if you manage to occupy one in the winter, can maintain an even temperature that may be less than what is outside by a substantial degree.
Caves can be readily defensible depending on their layout. Many caves have a large, anterior chamber with smaller chambers to the rear or sides. These smaller chambers can be used as storage rooms, and also for a retreat or a hide if for some reason it becomes necessary to vacate the larger chamber. The disadvantage is that few caves have an exit, so if it becomes necessary to defend yourselves, you do not have a recourse but to win.
You also need to find out what “roommates” may be in the cave, such as bats or bears, the latter of which tend to be very defensive of the territory. Also, if you live in grizzly country, the fall months are the time the bears feed before bedding down at the end of the year. You don’t want them coming in while they’re still awake, or moving into the cave when they’re in there asleep.
Keep your plans to yourself. Plenty of others will have the same idea. Ensure there is a water supply nearby the cave, as well as adequate game and also wood for fires. Most of all, learn as much as possible about the cave(s) you are considering, not just geographically but from a geological perspective. A cave may be something to use as a fallback retreat that may provide you and your family just what you need from either a temporary or a permanent perspective after the SHTF.