Kate Fitzgerald Consulting Limited

Arts Council England (ACE) recently shared an update on Illuminate. In this blog post I…

• summarise what organisations are required to report on in 2024/2025;
• explore the recent changes to Illuminate and the positives they bring;
• summarise changes to the survey collection;
• delve into some of the challenges and issues users have been facing; and
• share learnings and available resources.

For the 2024/2025 Financial Year

✅ NPOs, IPSOs and CPPs must use Illuminate to report on funded activity and audience/participant numbers every quarter (referred to here as activity data).

❌ NPOs and IPSOs are not required to survey audiences (though are encouraged to) with a set of mandatory questions (referred to here as survey data).

✅ CPPs are required to survey audiences for Illuminate as well as use the Impact & Insight Toolkit* for at least four events per year.

* Illuminate helps organisations to understand which audiences they are reaching, and which they aren’t. The Impact & Insight Toolkit helps organisations to evaluate specific pieces of work and measure progress against their ambitions.

Illuminate Survey News

The new survey template for 2024/2025 is available in the Illuminate platform as well as on ACE’s website.

Guidance is also available in the Illuminate platform for both creating and sharing surveys as well as uploading survey response data collected outside Illuminate. You can find How to Create and Issue Surveysand/or How to Upload Offline Survey Responses in the Training section in the platform.

The survey has been re-ordered so that demographic questions now come at the end.

The 2023/2024 template will remain available to allow the transition to using the new template. Any data you have collected during this time will remain visible to you going forward.

Finally, you DO NOT have to undertake Illuminate audience surveys if your organisation is solely:

• Touring
• Business to business i.e., you provide a service rather than directly engaged with audiences/participants
• Focused on artist/talent development
• A publisher

Or if your audiences/participants are solely:

• Under 16
• Vulnerable adults

New Enhancements to Illuminate

• Event collaboration – allows you to tag partners e.g., other funded organisations in your activity/event, creating one entry with data shared by all partners.
• Unattended mode – allows surveys to be set up on a device without remaining logged in to Illuminate i.e., when a survey needs to be left unattended, or you have multiple field workers collecting responses on different devices.
• Rest of world countries – when creating a location outside of the UK organisations can now list the country rather than just ‘Rest of the World’, providing more detailed evidence of international work.
• Refreshed language – across the platform and guidance, ‘NPO’ will be removed as a catch all term and be replace with ‘Organisation’ where possible.
• General API – allowing ticketing providers to connect to Illuminate and directly share data.

Summary of Change to the Survey Data

There are also some changes to the demographic questions themselves.

• The question sets have been reordered so they now run Event Information, Optional Questions, Type of Audience, and Demographics.
• “Prefer not to say” has been moved to top of each answer list
• The ethnicity question has been streamlined so it now matches the national standard as set out by the Office for National Statistics (ONS), giving ACE a comparable dataset that they can map against other sources and population data, allowing them to run national and area level analysis.
• The wording of the disability question has been updated so that the second part of the question is more aligned with ONS data (although it is still not directly comparable).
• “Not applicable” and “Not known” have been removed from the occupation question and replaced with an all-encompassing “None of the above / not applicable / not known” option.
• The sex question has been updated with “Intersex” being removed and replaced with “None of the above”
• The question asking respondents to describe their gender identity has been removed completely.
• The wording to the sexual orientation question has been updated.

The Good News

ACE has listened to feedback and made some positive changes, and those I know who have used the PwC Helpdesk have reported that they are always understanding, thorough, helpful, and swift with their customer service.

Survey Data

Whilst surveying audiences was not compulsory in 2023/2024 (for CPPs it was compulsory), organisations were still encouraged to do so – a recommendation I passed onto clients. Some organisations chose to use the Illuminate platform, others worked the mandatory questions into their own surveys in other platforms and uploaded their results to Illuminate. But the key is, organisations did engage, with ACE reporting over 220,000 survey responses so far. It will be interesting to see if ACE go on to publish the number of organisations the survey data comes from, but this level of response should allow them to weigh the data regionally against the population.

Unattended mode is also a valuable new feature and one that a lot of organisations have been waiting for. It would be better still if this could be developed further to allow for offline collections.

When the Illuminate survey template was first released, one of my main gripes was the survey began with the demographic questions first – they now sit at the end. Again, this is a really positive change.

Activity Data

Event collaboration is a welcome addition as this feature should be a pathway for touring or non-venue-based organisations to have easier and better access to audience data via their host venues on the platform.

The Not So Good News

People may be using the Illuminate platform and survey, but they are struggling. I put the feelers out to see what kind of challenges users were experiencing…

1. No Data Processing Agreement

If you’re sharing personal data with a third party, you should have a Data Processing Agreement in place before any data changes hands. If you are ever involved in a reportable data breach, either directly or via a third party, the Information Commissioner’s Office (ICO) is likely to ask to see any contracts in place around the data sharing, so they are important. Instances of data sharing should also be reflected in your Privacy Policy.

For my money, ACE should have a Data Processing Agreement in place for organisations using Illuminate. Independently, the survey data is not personally identifiable data. However, the requirement to link the data to a performance and postcode, means that it would be possible to identify a customer with a particular postcode who had been to a particular event. By process of elimination, it would be feasible to join the dots and see the profile and sensitive characteristics of that person. In that way, it becomes personally identifiable information and as such, I firmly believe a Data Processing Agreement with all users should be in place.

When this was raised with ACE by one NPO, they were informed that “ACE and their advisors concluded that DPAs were not required, as they were previously, as the data captured cannot be attributed to a single individual. As such, we have relevant privacy statements in place for users of the platform, but it has been agreed there is no longer a requirement for DPAs.”

In the Illuminate FAQs, ACE have confirmed that as a supplier, PwC are acting as data processors and are not able to use the data collected for any purpose other than the requirements stipulated by the Arts Council. PwC are also not able to commercially exploit the data at any point during the contract or beyond, nor will they keep the data at the end of the contract period. But what happens after that? Remember, this is your organisation’s data, you are responsible for it.

2. Lack of Communication

If organisations don’t know what data to collect, how can they collect it?

Because the monitoring guidance is not shared well in advance of the new financial year, a lot of organisations and consultants had already redone their evaluation frameworks, updated their survey tools, and started collecting data. Yes, ACE shared an Illuminate update on the 8 April 2024, but a lot of this work took place prior to that – it’s the normal way of working, you prepare for the year ahead. Consequently, everyone ends up having to do everything twice – this takes up valuable time.

3. Administrative Burden

ACE reporting requirements in general are an onerous process, with organisations having to report in multiple formats, in multiple ways, at multiple times. In January, a group of anonymous NPO Chief Executive Officers explained their frustrations and struggles with the reporting requirements to ArtsProfessional. ACE responded to these concerns and committed to continuing to make improvements to their processes.

For Illuminate, organisations must report activity data quarterly. If you have Spektrix or Ticksolve which are fully integrated, that process is automatic. If you don’t, it’s manual. Hopefully the general API will go a long way to solving this current issue. However, with the time and development costs falling to the suppliers, we will have to wait and see how and if they will engage with it.

Survey Data

If your organisation uses any other survey tool, they also aren’t integrated so you have the choice between a manual upload or running two surveys across two platforms.

Everything is event/activity level. Organisations cannot use one survey link for all their events – each event requires a unique link. Similarly, organisations cannot bulk upload all their survey data – it must be done at event/activity level. This involves downloading a template for each survey, completing it, and uploading it. You cannot change the name of the file as the system won’t accept it back.

It is also worth noting that Illuminate will not accept any exports that have blank responses (where respondents have skipped a question rather than answer “Prefer not to say”) to demographic questions. In short, if you don’t make every demographic question mandatory, you might need to do some manual data formatting. One organisation, who chose not to make the questions mandatory, reported that this means that they now have to manually change any blank responses to “Prefer not to say” just so their upload will be accepted.

Add to this that there is no easy way to download ALL data or ALL postcodes. Instead, you must download results spreadsheets for each survey and then combine them, making analysis more manual and time consuming. Again, all this takes up valuable time.

Some consultancies are offering support in doing this for you. E.g., Indigo Share offers this to their subscribers as part of their service. Similarly, Audience Answers provides users with raw data downloads for surveys, and, for an additional charge, can format the data so that it is ready for Illuminate.

But that means it’s not just organisational staff who are affected by this– an evaluation consultant (who wishes to remain anonymous) said

“I’ve spent an additional £3,500 worth of work on this that I haven’t billed the client for/wasn’t in my contract but felt I needed to help with… Replicate this by all the evaluation consultants and the extra time people are putting in – blogs, sharing their views in their networks, sharing their expertise in forums when we’re allowed to go to them. That stuff isn’t paid for (or baked in) – most of the time we’re doing it because we really, really want to get this working!”

On a personal note, I’ve also spent a lot of additional time making updates to on-going survey collections for clients as changes to Illuminate have occurred. So far, I haven’t charged anyone for this extra time, as it really isn’t their fault and often their budgets are limited. Whether that is sustainable going forward, I’m not yet sure. I will not always be able to work for free.

Some organisations are also experiencing technical difficulties – as part of our call out for any challenges currently being experienced, one NPO shared that they cannot export any data from the Insights panel – with the option showing as unavailable.

4. Demographics: Survey Data

There’s no getting away from the fact that there are still a lot of them. In fact, they make up 58% of the survey. Some audiences also find them intrusive. Aside from clients reporting customer complaints, I regularly review survey data and often see feedback such as “How is that question relevant?” and “Don’t agree with these questions” (comments taken from recent client survey results).

So, what can you do to reduce complaints? Well, if you’re an organisation that is collecting data outside of the Illuminate platform, and your survey tool allows you to add explanatory text, then include some! It’s important that individuals understand why you’re asking the questions that you’re asking, and what that data will be used for. For all client surveys, I include an introduction to the demographics section. This functionality is limited in Illuminate but if you are collecting outside the platform, I have shared the text I use on the surveys I produce in an earlier blog post – please feel free to cut and paste!

5. Activity Data Doesn’t Add Up

One thing that remains an issue for some organisations is reporting on activities that are ongoing or happening regularly. As I’ve already covered, reporting is quarterly. The issue some organisations are facing is that when they input their monthly attendance to these events/activities, the “total attendance” doesn’t add up gradually (e.g., Q1 = 10, Q2 = 15, TOTAL = 25). Instead, it saves the last number that’s been updated (in this example the “TOTAL” would be saved as 15 because this was the last number logged). To have factual audience numbers in quarterly reports, the data must be added up manually, again adding to the administrative burden.

6. Survey Data Benchmarks

Illuminate offers you the option to benchmark against other organisations in your area as well as against other organisations that share your discipline(s). However, this functionality is currently limited. For example, one organisation I spoke to reported that the functionality currently only works on the “occupation” question. Another organisation shared that they can only access occupation and ethnicity, but the results are only available as percentages for occupation, and as numbers for ethnicity. Organisations can also use the Understanding Your Areas tab to explore available Census data.

In fairness, benchmarks cannot be created until there is enough available data. But if the survey collection was not mandatory last year and won’t be in 2024/2025 (except for CPPs), it may be a while before they do become available. In Audience Finder days, organisations could compare their results to the regional and national picture – some could also compare them to regional organisational clusters. This simply isn’t possible at the moment with Illuminate. However, sector benchmarks are still available with Audience Answers and Indigo Share both offering these insights.

In Summary

Arts Council England have listened to feedback and taken some really positive steps. However, with survey data still not mandatory (except for CPPs) for 2024/2025, there is the risk that some organisations will not engage with it if they don’t have to. I would encourage all organisations to collect the data regardless – whether that be in the Illuminate platform or via another survey tool or service. There are still some challenges and issues that need to be overcome, and as Illuminate beds in, I hope that ACE continues their dialogue with the sector and listens to the experiences and challenges organisations are facing and continue to make improvements.

Finally

I would like to extend my thanks to anyone who reached out and shared their experiences. I understand that that people are a bit nervous about saying anything publicly, especially if they work at an NPO, IPSO or CPP, but if you are experiencing any challenges or issues and would like to share them, please feel free to contact me – anything you say will be treated in confidence.

Also, a huge thanks also to my sector colleagues who were kind enough to give up their time to peer-review this article before publishing – you know who you are, and I am truly grateful.

For general updates, please follow Kate Fitzgerald Consulting Limited on LinkedIn.