This may not be the 'correct' way of doing something, but you can place your xfce4 color scheme in the following folder:
/usr/share/xfce4/terminal/colorschemes
Look at the existing files for inspiration and add a [Scheme] and Name= line
to the top.
I successfully renamed terminalrc.jellybeans to jellybeans.theme with the
above modifications and... Presto!