In 1851, Charles Ridgely married his first cousin, Margaretta Sophia Howard Ridgely (1824-1904), who was the granddaughter of two Maryland governors, Charles Carnan Ridgely and John Eager Howard. (Howard was also a Baltimore's Revolutionary War hero of the Battle of Cowpens.) Having been raised next door to Hampton on the Cowpens Farm, she was a close friend of Charles’s sister “Didy.”
She was widowed in early middle age when Charles died in Rome of typhoid fever in 1872, leaving her to raise their seven children (including Margie, Eliza, and John) and manage a large estate. She continued to use Hampton principally as a summer residence, moving to her townhouse on Park Avenue in Baltimore’s fashionable Mount Vernon neighborhood in the fall. A firm mother and strong-minded woman, Margaretta established the Ridgelys’ Jersey cattle herd and, under the terms of her husband’s will, continued to control the family finances until her death in 1904.