Frequently Asked Questions

Is there a recording of a demo we can view?

Yes, here is a link to the recording from the demo on 31st August 2022. All our recordings are available on the SmartyGrants Help Hub under the video tutorials section.

Is there a feature that is new or different about the Outcomes Engine? Could the same be achieved from current form building in SmartyGrants?

kets

The Outcomes Engine comprises several moving parts and features, all of them built into SmartyGrants to allow for seamless incorporation of outcomes tracking into your existing forms and processes. The SmartyGrants team has invested heavily over many years in researching and co-designing (with grantmakers, grantseekers and evaluation experts) a best-practice system for tracking grantmaker outcomes. The system helps solve a big problem that’s particular to grantmakers – how to roll up results from multiple rounds and programs and multiple grantseekers to provide a consolidated view of progress across your entire portfolio, or according to filters you apply to the data (e.g., per round; per program; per year; per domain; per outcome; per metric).

The Outcomes Engine includes:

a) the ability to upload outcome frameworks (template provided) from settings and apply specified aspects of your framework(s) at both program and round levels. (You can upload your own framework or choose from our bank of best-practice frameworks.)

b) a series of default Outcomes Engine standard sections, that have been reviewed and refined by evaluation experts and tested by grantmakers and grantseekers, which allow you to connect data fields to one another (e.g., connect a grantseeker’s outcome to your outcome) and pipe that data through to subsequent forms to enable streamlined reporting of progress over time (e.g., ongoing reporting of metrics). The frameworks and standard sections are linked, allowing you to connect specific questions in your forms to specific domains, outcomes and metrics.

c) access to a suite of template outcomes reports, allowing you to track, aggregate and report on results over time. Word and Excel templates are available to download from the Help Hub and upload into your own SmartyGrants account. These templates are designed to allow you to pull data from the Outcomes Engine standard sections into pre-configured reports. We also provide examples of how the templates look when populated with data, as well as information on how to connect your results to third-party reporting tools such as Power BI. Visit the Help Hub Outcomes Engine reporting page here.

Will the standard sections functionality be extended across the platform or is this unique to outcomes only?

Standard sections is a new feature of SmartyGrants that will be available across all accounts in the future. Users of this functionality will have the ability to create their own custom standard sections (like you do now with standard fields) and there will be some default standard sections also provided (e.g., budgets, and the Outcomes Engine standard sections).
The default Outcomes Engine standard sections are enabled immediately for Outcomes Engine users; this does also provide you with the ability to create your own standard sections.

Can you have multiple frameworks loaded into a round?

Yes, you can upload as many frameworks as you like. You then have the option to select which elements (domain, outcome, metric) from any of those frameworks you wish to track for each program (for example, you may select all of one framework plus part of another; or all of two; or just parts of one framework). Items from your framework(s) that are selected for a program are available for selection for any round within that program. You can choose to use all of these on your round, or just some or none of them. If you want to add additional outcomes to a round, you need to add them to the parent program.

Are you looking to include any New Zealand specific frameworks?

Yes, recommendations welcomed! Please email your suggestions to the SmartyGrants Support Team with “Outcomes Engine frameworks” in the subject line.

Will you be creating more standard outcomes frameworks, or can we suggest new standard ones?

Yes, please feel free to send through your suggestions for frameworks that we should add to the library (see link above). Email the SmartyGrants Support Team with “Outcomes Engine frameworks” in the subject line. The existing bank of standard frameworks can be found in the Help Hub here.

Does the Outcome Engine support Google Documents?

At this stage the Outcome Engine only supports Microsoft Excel, so the Outcome Framework template needs to be downloaded and uploaded as a Microsoft Excel file. If you do not have access to Microsoft Excel please contact us and we can organise to have your Outcome Framework uploaded to your instance in an Excel format.

Do you have to use the Outcomes Engine at the time of application?

Ideally you will include some of the relevant Outcomes Engine default standard sections in your application form. This allows you to assess the applicant’s alignment with the overriding outcome goals of your program and allocate your funding accordingly. Indeed, most users employ aspects of the Outcomes Engine throughout the delivery of their round: in the application phase; during assessments; and during progress reporting and final acquittals. Data entered during the application phase (e.g. proposed outcomes, metrics and associated targets) are piped through from the application into assessment forms and on to your grantseekers’ progress/acquittal forms so the applicant can see what they committed to delivering on.

That said, you can start using the Outcomes Engine at any phase of your round rollout. For example, you could use the outcomes standard sections in acquittals to collect retrospective data and information about your grantseekers’ results. The sections, by default, contain language suitable to applications; however, the text can be altered so they make sense for acquittals instead. 8. Can you have more than one outcome column (in the grantmaker’s outcomes framework) – e.g., one for short-term outcomes and another for long-term outcomes?

In the Excel outcome framework template, there is currently only one column for all grantmaker outcomes, and it is not possible for a grantmaker to alter the template format (doing so will generate a system error). This could be considered for a future release if demand is high.

To identify short-term, medium-term or long-term outcomes the grantmaker could identify whether the outcomes are short, medium or long term in the name of the outcome i.e. “Increase knowledge of health and wellbeing (short term)” or make a note in the description column of the outcome framework.
In the Outcome default Standard Section in the form builder there is a default question that allows grantseekers to classify their outcome as ‘immediate’, ‘medium-term’ or ‘long-term’ but this does not currently exist for grantmakers.

Does everyone in the account have access to the outcome engine or is it just admin?

An Admin is the one who has access to upload a framework and needs to select on the program's settings which outcomes will be tracked for the program. Then Admins or Program Managers can select when they create a round, and which outcomes they are tracking for that particular round.

Separate to that is form building (this is a customisable permission in SG, so it depends on who has access to build a form to add the standard sections, etc) - but by default that is Admins and Program Managers also. But if an Admin for example has said only Admins can create forms, then it is only them. Or they may have allowed Standard Users to edit forms (more unusual).

Is there a way to allow people filling in forms to replicate their outcome so that they can align it with multiple of our outcome goals?

No, but they can add more rows (if you have enabled that option on the form) and then simply copy and paste their outcome into a new row.

If we decide on mandated outcomes, do we need the outcome table or can we just include the metrics table in the application form?
You could just use the metrics table if that suits your purpose. We can work with you during your first round (you get eight hours of free expert support in your first year) to ensure you understand the best way to set up your form for your purposes.

Can you collect expected outcomes separately to actual outcomes?

The Outcomes Engine default outcomes standard section is designed for use on application forms, so these are ‘expected outcomes’. However, when you add that section to an acquittal form, you can allow users to comment on whether those outcomes have been achieved; plus, they can add new rows to document additional positive outcomes that have arisen.

There is an additional default standard section that can also be used on acquittal forms to allow grantseekers to document “unintended negative outcomes”.

Can the Outcomes Engine templates be made conditional within one form? We often have a question asking what Research or Strategic Theme they are delivering to. Each Theme may have slightly different output metrics. If they choose Theme A, can they be directed to the Outcome Engine metrics for Theme A etc? Not yet, but this is something we can looking into if we note demand for this functionality. Currently, people use the ‘domain’ headers within the framework template to help applicants sort those outcomes that may be relevant to them.

Can outcomes be completed within an administration form (instead of the application form) as part of an internal process when developing the agreement and a monitoring & evaluation plan, then have this 'pull through' to the acquittal form?

Yes. The Outcomes Engine default standard sections can be added to admin forms (indeed they can be added to any forms). The responses entered in the admin form would then populate into the acquittal form if the standard section was reused on that form.

Do we have to set up a defined list of metrics within the framework or can we allow applicants to set metrics relevant to their specific initiative?
You can do both or neither! That is, you can add metrics to your framework so that applicants can select one or some of these to track against, or you can leave this section of your framework blank.
You can also (or alternatively) use the grantseeker metric default standard section to collect information on what metrics grantseekers use, and their progress against these metrics (some grantmakers do this in the first round so they can get an idea of what metrics make sense to their grantseekers, adding the common ones to their own framework the following year).

Grantmaker metrics can be more easily aggregated than grantseeker metrics, but it’s still very useful to see what grantseekers use to assess their progress.

Can the maximise button be available for use in other grids use in forms?

This will be available for all grids when standard sections is released later in the year.

Where is the ‘all fields’ template (the Excel template that aids multi-program and round reporting) located?
You can download the ‘all fields’ template from the Help Hub.

Was the template report that you demonstrated in the Public Webinar derived from one program? Can it be used across all programs for a financial year?

The ‘all fields’ report will allow you to pull together data from across multiple programs and rounds across multiple years.

Is there any way to connect Microsoft Power BI or have it query the database without having to download reports into SharePoint?

There are two approaches to connect SmartyGrants and Power BI:

a) Using scheduled reports (Excel) and Microsoft 365 to build a data pipeline to Power BI

b) Connecting Power BI directly to the SmartyGrants APIs.

Guides are available for each of these approaches. Please contact the SmartyGrants Support Team to request a copy of the guides.

Note: The SmartyGrants APIs were designed for systems integration rather than reporting, and there are some limitations with regards to data coverage and usability. We are currently working on an interface that is designed specifically for reporting (using OData). In the meantime, the recommended approach is to use scheduled reports.

You showed an impact report (pdf) during the demonstration. Is this a template from SmartyGrants or is it a custom report that we (grantmaker) would create that includes data visualisations that need to be done by us?

We provide Excel and Word templates that you can use (you can see examples, and download the templates, in the Help Hub). The “glossy” impact report shown during the demo was developed using a mixture of the Excel charts and Word template reports drawn into a third-party design tool.

A more fulsome explanation of Outcomes Engine reporting can be found in the Help Hub here.

Is it possible to have a copy of that dummy report to show management the sort of reports and outcomes we could produce with the Outcomes Engine?

You can download copies of all the sample reports from this Help Hub page.

Wondering about the chronology of the reporting feature and how an individual interacts with it. Often towards the end, or after, or sometime after a project, participants come together in an evaluation exercise to agree on outcomes and so forth.

Increasingly grantmakers and grantseekers are agreeing on outcome goals at the beginning of a project so they can baseline their ‘current state’ and set up reporting to monitor outcomes over time. Also, setting up intended outcomes at the beginning of a grant round can be useful during assessment to ensure you’re funding is being directed to the applications most likely to help you achieve your program outcome goals.
Often new outcomes do emerge over time that may not have been foreseen at the beginning of a project and we recommend that they be collected using the ‘unintended outcomes’ standard section.
All of that said, the system is very flexible, and you could use the outcomes standard section for the first time on an administration or acquittal form. You would just need to change the default text and amend the fields to reflect the fact that the events are being recorded in retrospect, rather than as anticipated outcomes, activities and metrics.

What about information privacy for grantseekers and grant orgs?

Data collected through the Outcomes Engine is handled in line with our customer Agreements and Privacy Policy.

You mentioned there was a per-instance fee. Does ‘per instance’ mean per person, or per organisational account?

‘Per instance’ means per organisational account. If your organisation has one SmartyGrants applicant site, you will pay $2,990 (including GST per annum). If you have 2 applicant sites, you will pay $5,980.

For technical support our support team is available as per our Support and Maintenance Policy, and our Managed Services team can provide further assistance such as grant program reviews, form- and report-building services, or further outcomes measurement and support services.