I am lactose intolerant just like my father and usually avoid milk, but traditionally my family did not avoid it. I can increase my tolerance to milk gradually

I am lactose intolerant just like my father and usually avoid milk, but traditionally my family did not avoid it. I can increase my tolerance to milk gradually, but I find no benefit in doing so, because I can drink up to 200 ml of milk without major issues, I can eat cookies, icecream and so on without major problems. The only time I had major problems was when I used milk protein powder, but I switched to whey protein at that time and learnt that I was lactose intolerant. My point is lactose intolerance doesn't have to be all or nothing.