Swift Charts use custom font for legend

We have a custom font we use through-out our app we've recently starting playing with Swift Charts but can't work out how to use our font in the chart legend easily.

Currently we have a simple chart using the default .chartLegend(...). I've managed to use our font in the X-axis and Y-axis by creating our own labels using AxisValueLabel() {} However without making our own completely custom legend, is there a way to specify the font to use in the legend?