Island Challenge Amulet lowers your maximum HP by 100; while not as clear as I would like, the wording is consistent with how cards that increase maximum HP are worded e.g. "...it gets +50 HP." for Buff padding. If something does damage to a Pokémon, it will say to do that amount of damage, or that the Pokémon in question takes that much damage. Actually, the most likely thing with effects which reduce your Pokémon's current HP instead of their maximum HP is for them to place damage counters, so we don't have to worry about damage reducing effects. For example, Rainbow Energy originally did do 10 damage, but was revised many, many years ago to place a damage counter instead.
So, if that wasn't clear... when you equip Keldeo-GX with Island Challenge Amulet, maximum HP becomes 70 (170 - 100). The amount of damage counters already on it remains the same, so if it had no damage counters on it, it still has none, if it had five damage counters on it, there would still be five, etc. If the number of damage counters times 10 is equal to or more than its new HP score, Keldeo-GX is KO'd. You can still heal any damage present, but that won't change what Island Challenge Amulet is doing (reducing the maximum HP).