The relevance score currently only measures whether the ads that you ran were relevant to the audience that you reached. But from the 30th of April Facebook changes their ad Relevance scores.
Facebook changes the relevance scores with three new metrics:
Quality ranking: How the quality of an ad is perceived, in comparison with ads targeted on the same audience.
Engagement rate ranking: the engagement on an ad, in comparison with ads targeted on the same audience.
Conversion rate ranking: the conversion rate on an ad, in comparison with ads targeted on the same audience and ads that have the same goal.
These new matrices provide you with more information and help you understand where to aim changes.