Sometimes things don’t work out the way you think. While a current legislative procedure edged closer to the finish line in the European parliament, one of our client’s policy teams realized that they did not have any insights into the over 5.000 amendments put forward. And apparently neither did their sector associations. While they had their engagements activities lined up, they actually lacked the content to engage about. Enter Team Bernstein.
We were asked to analyze the thousands of amendments in a couple days and to identify those, that are the most critical positively as well as negatively for their business case.
How AI rescued us…not
It would be a testament to the awesome power of AI if we could tell you a story how we used a large language model (think ChatGPT) to put all the amendments in and got a nicely formatted analysis which amendments are actually important and why. Alas, while we use AI in many ways, we did not feel comfortable using it here. The matter at hand was complex, technical and there were second- and even third-order effects to consider. The tight deadline also left little time for experimentation.
So what did we do? Gather the subject matter experts, the process experts and data experts together and develop a framework that would allow us to identify and prioritize those amendments that are important to our client.
Taking the easy (data) route
While the data team was already itching to scrape all amendments from parliament docs, we remembered the fine folks at Lobium actually created an amendment analyzer. After a quick call we were certain, that their data product would help us get a head start. Not only could we get all amendments and the duplicates within them in a nice machine readable format, but we could use the platform to drill down into the mass of text. That was invaluable to get an understanding of the amendments and guide our analytics framework and understand. To do our in-depth analysis however, we decided to export the necessary data.
Keeping it simple – almost
When looking at the articles we need to understand two things: content and context. Content refers to the actual amendment text and its relevance to the client. Data and subject matter experts defined keywords and patterns that correspond to different categories of relevance to us. Using natural language processing we analyzed the amendments and categorized them on a simple ordinal scale.
While content is obviously supreme, context matters. Again, we use simple ordinal scales to categorize context along several dimensions:
- Location: Some articles are more important than others. We use a reduced scale here, since this is more about reading prioritization than filtering.
- Stakeholders: While all MEPs in the relevant committees can submit amendments, rapporteurs or the lead committee carry more weight. The scales reflect that importance, giving more weight to consolidated amendments or amendments introduced by rapporteurs.
- Politics: Not all parties are equally important in the client’s engagement plan. Again, using ordinal scales we can reflect that prioritization.
After categorization, we use a weighted mean to gauge the relative relevancy. While this is not “data magic” it is effective. For final filtering and prioritization we use a combined weight of content and context. Due to the fact how the amendment process works (process expertise!) we decide to give content more weight than context – it is not unheard of that sensible amendments are re-introduced if for political reasons the original amendment is not “votable”.
Analyzing what matters
We end up with a list of 1.039 amendments, about 20% of the original sum. We do decide to filter those down to the highest relevance category, retaining 2% or 101 amendments. While still quite a bit to read, it is a manageable count.
Not for its own sake: Creating Impact
We understand that a single word can often have a significant impact on legislation. Thanks to the data analysis conducted by our experts, we are able to thoroughly analyze the remaining amendments even on a tight schedule.
We carefully evaluate the effects of these amendments on our client's business model, with a focus on identifying the most critical ones. Of the 101 remaining amendments, we identified 35 as crucial for our client, including 3 as the most critical. With the remaining 35 amendments, we helped our client to meaningfully engage with key stakeholders and achieve positive business outcomes while working under tight deadlines and limited resources.
If you have similar challenges or need a data-driven approach to your political and regulatory challenges, feel free to reach out!
