I've never heard of a bunny 'technically' being able to be weaned at 4 weeks, so I would be extremely weary of that vet, for starters.
I also wouldn't get a rabbit just to save it from a bad breeder
I have that rule specifically to keep myself from having 6000 bunnies.
Try to find a exotic vet, check rabbit.org for a list of reputable vets. Call and talk about the issues that come along with an abandoned bunny, it honestly might cost you a LOT of money in vet bills as well as heartache. It could be fine, btw, some buns are resilient, but 4 weeks is just so.stinkin.young