VerbaLab
In winter 2022, one of our largest clients at Kantar expressed interest in a qualitative report focused on brand perception as expressed by the language in open-ended survey responses. Citing my linguistics background, a vice president recommended me for the job. I delivered Kantar's first linguistic deep-dive report, a product clients have continued to request and for which I've trained team members.
​
I recognized an opportunity to drive workstream efficiencies by developing software that could use AI to parse thousands of verbatims and equip analysts with an arsenal of linguistic tools, allowing them to extract more meaningful insights in a fraction of the time.
My Role
Kantar is the world’s leading data and consulting company, with branches in 90 markets. I was working in the quantitative insights division when the opportunity presented to do a linguistic analysis for a client.
Following its positive reception, I was introduced to the company's qualitative division, with whom I partnered on subsequent reports; their focus was on extracting insights from what respondents said, while I extracted insights from how they said it.
As scale increased, I independently coded a prototype for a piece of software I called VerbaLab, with the intention of facilitating future linguistic analyses.
​The Challenge
Kantar clients have several options for how to collect open-ended responses to questions, which we leverage as brand consultants. While written questionnaires are an attractive choice, a client also may elect to conduct video surveys, which garner longer, often more candid responses. A transcript may look something like this:
​
PROMPT: What services do you look for in healthcare providers?
​
VERBATIM: Hello, so regarding my role in our health care system and how I think about health care services. My role as it relates to procuring new health care services for a company is the controller responsible for not only most, if not all, of the procurement or whether its materials or services provided to the organization than our employees. What we want to make sure that we're getting the best and most beneficial benefits for the employees at the most reasonable cost. Not always the cheapest cost. How are we getting this work done today? It has become more, there has been more competition in the area. There's been a lot more flexibility in terms of the benefits, but there's also been a lot of newcomers to the industry. It makes it a little bit more difficult to, for us to choose the best, best provider for our employees.
​
Brush away the messiness, and you have an insights goldmine. When a project collects only a hundred or so of this open-ended responses, or verbatims, it's easy to give each one this requisite thoughtful attention. Less so with sample sizes in the thousands.
The bottom line: analysts are spread too thin sifting through verbatims for relevant content to give attention to the language.
Discovery
I had been quick to board the artificial intelligence bandwagon, particularly interested in exploring the linguistic capacity and limitations of generative AI (it was good at solving problems, but how well could it comprehend why someone phrased their remark the way they did?).
Once I connected the VerbaLab prototype to a pre-trained generative AI tool, it didn't take long to get distilled summaries and key points from dummy verbatims. But those points just as easily could have come from a written survey. The value in hearing someone speak is the ability to read between the lines. I wanted to automate the synthesis of mass quantities of verbatims while maintaining linguistic integrity.
​
At the same time, keeping verbatims, well, verbatim, posed a data privacy risk. Even if the verbatims were synthesized anonymously, uploading word-for-word responses to a public AI backend still challenged the confidentiality expected by our respondents.
​
VerbaLab's minimum viable product needed to facilitate:
​
-
Extracting key content, figurative language, and paralanguage from verbatims
-
Enabling an analyst to categorize verbatims more efficiently
-
Not breaching respondent confidentiality
​
Development
I decided to build VerbaLab 1.0 with Swift for MacOS to be leveraged by our Mac-using teams. While I was comfortable working with Swift on iOS products like Phissy, the shift from writing for mobile to desktop meant relearning core workflows and familiarizing myself with new human interface guidelines.
​
A familiar environment, reimagined
The more sophisticated interface allowed me to create more complex molecular elements. For instance, when sifting through a large spreadsheet of verbatims, analysts often had difficulty filtering by multiple attributes. Often, multiple sheets would be used within a workbook—one per question, one per demographic, etc. And while this worked, it wasted time and prohibited finding trends across the sheets. (Yes, Excel has a robust advanced filtering feature, but UI is lacking—users have to click into the header to remind themselves what their filtering criteria are, resulting in missing out on certain responses).
​
VerbaLab's solution was to preserve the "spreadsheet environment" familiar to our analysts but to introduce a full-length sidebar that would be populated automatically when importing the CSV of verbatims, programmed to determine age ranges, genders, questions asked, and other variables that could be valuable filters. This enhanced visibility and eliminated the dual-purpose header—now users could filter smoothly with the sidebar and sort columns by clicking the header of each.
​
Language needs → AI calls
During this CSV import was when I wanted to run each verbatim through our AI engine, effectively adding additional columns (which could be toggled from another sidebar on the opposite side of the window) for each linguistic lens applied. These would help the analyst group the verbatims meaningfully.
​
I decided to keep the summary, as this proved helpful in grounding the analyst's first pass through the data. Fine-tuning the AI calls with the MVP in mind, I also added columns that would pull figures of speech and sensory language from the verbatims, as well as higher-order calls that would synthesize constructive ("this respondent envisions a future where...") and emotional ("this respondent feels...") interpretations. Turns out summarizing was easy for the AI and constructive and emotional interpretations were doable with well-crafted prompts, while extracting figurative language (metaphors, similes, hyperboles, sarcasm) and sensory language was hardest to teach. Ultimately, we reached 90% success.
But I wasn't satisfied. As an applied linguist, I wanted VerbaLab to offer our team as many tools as possible to draw conclusions from these verbatims. I decided to add another set of toggles under the AI-generated ones, which would highlight (in corresponding colors) relevant details within the verbatim text. Users now could easily find convenience/efficiency modals ("want to"/"need to"), desire modals ("want to"/"would like to"), fulfilled modals ("can"/"get to"), and mentions of obviousness—check out the demo for why these six textual elements are uniquely valuable.
​
Now the analyst would have a full digital suite to do the grunt work automatically and the tools to draw more educated, but uniquely human, inferences. Without VerbaLab, this process might have taken an analyst several days and netted less insightful conclusions.
​
​
New ways to group
At this point, most analysts would decide to group the verbatims into buckets. Before, this might have looked like creating an additional Excel column to assign each verbatim to a theme (which the analyst may have been brainstorming on a napkin while reading through the verbatims). I created a key interface in VerbaLab to add, edit, delete, and keep track of existing themes at any point in the process—a sort of built-in, dynamic sticky note. When the analyst were ready to start grouping, VerbaLab then would funnel these themes into a drop-down on each row. Decide to change a theme name later? Just edit it in the key and watch all rows update accordingly.
​
I wondered—perhaps AI is not yet insightful enough to identify themes as well as a human with extensive linguistic training, but if given a list of themes, could it match each verbatim to the closest fit? Sure enough, it can, and with remarkable accuracy. I programmed this beta feature as optional—it shaves time, though some manual fine-tuning still is warranted.
​
Visualizing results
Kantar utilizes interactive topline dashboards across solutions, and I wanted VerbaLab to enable similar data visualization with:
​
-
​a succinct AI-generated composite gist of all verbatims
-
a donut chart of truncated infinitives attached to convenience modals, indicating processes that respondents wish could be eliminated
-
a donut chart of truncated infinitives attached to efficiency modals, indicating processes that respondents wish could be made more streamlined
-
a word cloud of most common terms across verbatims, filtered for a customized list of stopwords
-
a bar chart to illustrate the distribution of themes
​
Avoiding a data breach
The efficacy of VerbaLab hinged on the power of its AI backend.
My initial pitch suggested using the latest model of OpenAI—at the time considered the most revolutionary available model—each input word would cost about 1/500 of a cent. Since VerbaLab performed between 6-20 calls per input depending on the depth of analysis needed, a pull of 100 verbatims at 100 words each would cost approximately USD $4, whereas a team member may rack up $500 in billable hours performing the same operations manually over multiple days. Cleanest results, lowest cost. Also lowest degree of privacy.
To proceed with OpenAI, we would need either to request further consent from our respondents or to anonymize any identifiable keywords (such as names, business names, etc.) before making API calls. I discussed the latter with Kantar's global data team of AI specialists, who concurred but had not yet found a systematic way of anonymizing all identifiable information within a block of text (even using NLP to flag any proper noun and blanket replace all instances across verbatims with a nonce word was not secure enough). The global team was more eager to pour resources into training an internal Kantar AI system, than implementing VerbaLab or a refined equivalent when safe to move forward.
​
I set these up all to refresh upon changing the demographic or question filters in the left sidebar, so inferences and comparisons could be performed more quickly than ever before.
​
Results
-
Started cross-functional conversation around qualitative brand/media use cases for NLP and generative AI among global data science teams at Kantar
-
Built prototype for proprietary software that will lay groundwork for linguistic solutions that could generate $10K+ in annual revenue while saving thousands in labor hours
What I learned
I joined Kantar because I am a creative problem solver. I get to leverage that proficiency when consulting with clients on how to optimize their brand image, but sometimes optimization opportunities are internal, as well. We only serve our clients as well as we are equipped to!
​
Given confidentiality, I look forward to sharing updates in the future. Contact me to learn more.