An English peer and son of a baronet, Edward Henry Lee married Lady Charlotte Fitzroy, an illegitimate daughter of King Charles II.