Who did Harriett live with?

The answer to your question would depend upon Harriet's age and the chapter your question pertains to. At the start of the narrative, Harriet is living with her parents. After her mother's death, she lives with her mistress until she also dies. In her will, Harriet's mistress bequeathed her to her sister's daughter, with whom she was then sent to live.

In complexion my parents were a light shade of brownish yellow, and were termed mulattoes. They lived together in a comfortable home; and, though we were all slaves, I was so fondly shielded that I never dreamed I was a piece of merchandise, trusted to them for safe keeping, and liable to be demanded of them at any moment.

When I was nearly twelve years old, my kind mistress sickened and died. As I saw the cheek grow paler, and the eye more glassy, how earnestly I prayed in my heart that she might live! I loved her; for she had been almost like a mother to me. My prayers were not answered. She died, and they buried her in the little churchyard, where, day after day, my tears fell upon her grave.

Dr. Flint, a physician in the neighborhood, had married the sister of my mistress, and I was now the property of their little daughter.