Properties for Distances / Divergences

Bellemare et al 2017 introduce several properties of distances that might be desirable.

• Scale Sensitive: A distance $d(\cdot, \cdot)$ is scale sensitive if $\exists \beta >0$ such that $\forall X, Y, c>0$

$$d(cX, cY) \leq \lvert c \lvert^{\beta} d(X, Y)$$

Intuitively, this just means that scaling the arguments by $c$ scales the distance by $c$, possibly to some power.

• Sum Invariant: A distance $d(\cdot, \cdot)$ is sum invariant if for $A$ independent of $X, Y$, we have

$$d(X+A, Y+A) \leq d(X, Y)$$

Intuitively, this means a constant shift of both $X, Y$ doesn’t change the distance between them.