Dont know who should walk me down the aisle?

so my dad has never really been in my life at least not for more then a couple months every few years my mom has basically raised me my whole life... im getting married in sep and my whole family (dads side) is expecting him to walk me down the aisle... but i mean he really doesn't have the right to i mean you cant give away something that you never had to begin with right... i have tried to let the family down easy by saying we were think of having me walk alone cause its easier then just saying i don't want him to walk me because hes been a dead beat dad my whole life... but they just don't seem to understand i would like to have them involved and there... i just don't know what to do