When adding custom css, it would be great if the colorpicker allowed you to pick colors from the color vars in the theme AND for those picked colors to automatically get replaced when you change the theme.
Not sure what you mean by "change the theme." If you've designed a site with custom colors in a CSS file, just copy that custom CSS to your new site.