The changes won't happen until the interface is re-configured, just like for the other settings.
(cherry picked from commit a18f6df7)