General Hardy inequalities with improvements and applications