Towards the end of episode 17 (of the second season of SAO), after extricating Excaliber from the dais in the lowest level of Thrymheim, Kirito makes a comment about the sword "still being too heavy" for him. He then throws Excaliber away before jumping onto the flying tentacle-elephant.
Why did he do that? He doesn't seem averse to accepting the sword from Sinon a few minutes later (after she grabs the sword out of the air with one of her arrows), so it didn't seem like a "I am unworthy to wield this sword" type of thing.
It could be that he was still bearing the fact that he first met the sword during his first days in ALO when to him it was still not just a game but a reality so when he had the sword in hand during the 17th episode he began to remember certain unpleasant memories and thought to himself the weight of it all is still to heavy for him to bear its quite possible Sinon saw this (from the sideline) and retrieved the sword for him as a form of gratitude for helping to deal with her traumatic past once again possible kirito saw this in her eyes and accepted the sword as a gift from Sinon.
It is a very simple answer. He has no ability to use the sword unless he carries it forever because he cannot claim it. He cannot claim until Jötunheimr is restored, hence the end of the quest. Also he cannot jump with it because he would fall like a rock. He blames the Cardinal system because it will not let him have it because he has not completed the quest at hand. He has no ability to claim the sword therefore he throws it away because he cannot jump with it. Once on Tonky, sinon retrieves the sword with an arrow, but since Kirito is on Tonky and his handed the sword he can find out what he has to do to claim it without having to jump and fall to his death.
The reason why kirito through the holy sword was because if he were to take it he would have to throw away the other items in his iventory. Though it had seemed that he was able to get it in season 2 mother's rosario he was eventually able to get the holy sword because in the first episode of mother's rosario he had a chest if you were paying attention. He was rewareded with the sword as well as a hammer at the of the episode excalibur.