Devil Fruit users' strength is sucked away when they are in sea or under kairosaki chains. In the Marineford arc, Luffy was standing on the Giant Ice land formed when Aokiji froze the sea. My question is: Ice is basically frozen water, shouldn't that also sap away Devil Fruit users' strength?
Yes, in practice, frozen water also would sap the power of a devil fruit user.
In the SBS of the One Piece Manga — Vol. 41 (p. 206), Oda mentions all types of Water cause weakness in Devil Fruit users. However, in this same SBS it is also mentioned that not touching the water itself will not affect the user.
Hence walking on ice with shoes or the likes, would in most cases prevent any form of power sap/weakening.