Suppose we add a value to a map using a pointer as the key. Could the memory address of this pointer ever change? If so, will looking it up in the map fail because when it was inserted, it had a different memory address?
Could the memory address change, in the pointers-as-uints sense? Yes, but outside of the unsafe package go doesn't expose pointers as uints. So, will looking it up in the map fail? No.
See more at the spec. The linked section even includes a pointer-keyed map as an example.