Rabbits are territorial creatures, some more than others. It’s very normal that he objects to someone reaching into his cage, especially when he’s in it. Make sure he’s not in his cage when you have to clean it or do anything else. Every time you reach in and it causes him to react negatively, it’s a step back and the behavior gets reinforced. My rabbit used to be very aggressive in his cage, as well as when I was giving him food. It took a long time and a lot of patience to change that, and even now he often objects to me messing around in his cage, although it’s usually just a verbal complaint. Try to avoid situations that you know will set him off, let him tell you what works for him.
As far as people getting bit when trying to touch him: Rabbits are prey animals, and as such, they hate being picked up. In nature that means they’re about to become dinner. For the same reason, many rabbits are uncomfortable with being touched, although they may start to like being pet (on their terms) over time. You said there are people (kids?) in the household who will reach for him but don’t know how to approach him correctly. Don’t force physical interactions on your rabbit, let him come to you. Everyone needs to be on the same page about that. Especially with children, there have to be strict rules about how to handle the rabbit. Young kids should never be allowed to pick them up, as they can break their own spine trying to get away if not held properly. Don’t let the kids chase him around/corner him and try to grab him. Unfortunately, rabbits are always portrait as sweet and cuddly fluffballs, a perfect starter pet for young children, which could not be further from the truth. Kids like to touch, pick up and carry their pets around, all of which are things rabbits hate. This is the number one reason why rabbits end up in shelters. The parents get them for their kid, the kid tries to touch, cuddle and love on the rabbit, which eventually makes him become defensive. The kid ends up getting bit, and the bun is deemed aggressive and gets dumped. I’m not saying that’s what happened here, but since there are kids in the household, I wanted to mention it. People often have the wrong expectations of pet rabbits (due to them being marketed as easy pets), which isn’t the owners fault, but it does lead to bad situations for both human and bunny.
If the children are too young to understand and comply with a more “hands off the bunny” arrangement, then you’ll have to keep them away from him, for now.
It’s great that he seeks you out for cuddles and even sleeps with you, he trusts you.