How serious is latex allergy?

Do I really need to worry about vials with latex tops or BP cuffs with latex bladders? Can't rash be treated symptomatically, or is escalation to anaphylaxis a major concern?