You don’t need to improve ‘everything’. Selected mortars, towers, AT guns, stables for defenses; economic or research buildings at your discretion, and you should be ready to move up. Defenses at 70-80% max is enough to look after your base (imo), the rest can be done when you’re, presumably, getting more loot with your improved troops.
But essentially, move up when it ‘feels’ right for you.