This depends on the type and size of organisation you're working with.
We usually say that, for a Needs Matrix, the stakeholder group should be composed of people from a wide variety of aspects of the organisation who, critically, can speak on behalf of the organisation. In our experience, this is typically senior management.
The same group is usually involved in the Service Scorecard.
For the Initiative Scorecard, we recommend that people who are more hands-on, or frontline, form this group. This group focuses more on the detail of specific Initiatives so subject matter experts are usually best, whether these are from internal teams or external partners.