You almost have it. Just set the colour in the series you're interested in.
Chart1.Series.Color := clBlue;
Colors are just hex constants in Blue, Green, Red order. The predefined list is in Graphics.pas, but you can use any hex value you like. This line also sets the color of the first series to blue:
Chart1.Series.Color := $FF0000;
If you have more than one series defined, you can do something like this:
Chart1.Series.Color := clGreen;
Chart1.Series.Color := clYellow;