CRAP Talks 15 – Social Game Experiments

On October 8th, we ascended the elevator to the fifth floor for our third (and possibly final!) time at the Busuu venue in Moorgate for CRAP Talks 15 with a fully female, fantastic line up. Once again, we took up the colourful stools in the cosy event space, pizza and pint in hand, to be entertained, educated and enthralled by the fascinating developments in the world of conversion rate, analysis and product.

Speaker 1 – Jess Percival

Our first speaker of the evening was Jess Percival, a Research Analyst at Twitter. She introduced the platform as a ‘hivemind’ for opinions, and spoke briefly about the impressive statistics surrounding the individual communities existing on there. Her talk focused on what Twitter was doing for customers, and how her and her team give them insight to boost their popularity, create more impact for marketing campaigns and allow them to engage directly with their customers.

The different ways this is monitored is through four key groups or resources. These were insiders, an exclusive panel which will give customers insight before campaigns go live; thought leadership in a variety of industries such as travel, beauty, gambling and automotive; audience insights to match up conversations around cultural moments and the ‘creative council’, who give qualitative research from different industries.

In addition to these, Jess spoke about the value of the main message which appears in the comment box before you type on Twitter – ‘what’s happening?’. This gives brands direction, exposure and power by reaching out to their audiences and providing a new outlet for updates to be shared as a nice addition to the approved content plan.

The Research Analysis team at Twitter have revolutionized the way brands communicate personally and connect with their customer base, making it a crucial social media tool for any brand who want to stay at the modern forefront of trends and gain a more active voice in comparison to their competitors.

Speaker 2 – Holly Emblem

Holly Emblem, our next speaker, is a Data Scientist at Rare, a game-as-a-service provided through Microsoft, which takes online gaming to new levels by always being live. Her topic revolved around the significant effect that design thinking had in the data team during their release of one of their most popular games, ‘Sea of Thieves’, and the evolution of choosing to utilise human centred design in the process.

Bringing data science into a process which is mainly driven through art and design had to be done with a little sympathy toward the teams around them, such as customer service and research. This helped not only to build rapport and open doors, but allowed them all to see any opportunity to collaborate as a chance for success.

The data scientists were able to provide insight into understanding the user and emphasizing audience demand, which challenged the assumptions from the design teams and helped to redefine the problems they were facing. This comparison in approach was highlighted by Holly outlining the before and after situations when adopting this design thinking process, and what the overall learning was.

She explained the design thinking process being comprised of five key stages – empathise, design, ideate, prototype and test. This new way of reasoning made her realize an ideology known as the ‘numbers complex’ to which I’m sure any data scientist can relate at one time in their career. It is the idea that having data doesn’t mean you have the answers, and the sacrifice of being right in isolation is never worth it.

This humble, honest and heartwarming confession spurred a litany of learning wholly unrelated to data, but around collaboration, such as checking biases, really listening to your teams, creating pathways of open conversations and focusing on achieving an end result rather than perfection.

Workshop With Ellie

Up next, Ellie Hughes, the Optimisation Manager at Photobox, treated us to a never-before-seen-at-CRAP interactive talk around some new optimization methodologies she had developed, which she urged the community to take into their teams and experiment with.

After a short five-minute introduction, she left the stage and took those interested attendees (almost everyone!) around to the kitchen area, where she had the ideas tacked onto a glass wall, and talked us through them during the interval.

The three core ideas included an Experiment Definition canvas, which aimed to help teams to get their experimentation design right and learn about sample size. Then came an Experimentation Retrospective, with the purpose of diagnosing issues teams were having and strive to make it better. Lastly, we learnt about the Enablement Model, which focused on supporting teams in their learning goals and correcting their mindsets, addressing the failures by asking how, what and why.

For such a highly recognized, respected and regular attendee in the CRAP community as well as powerful figure in the optimization world, this innovative change of scenery and fresh ‘call-to-arms’ approach to speaking was a brilliant and inspiring decision. It prompted the community (some of which who were first timers) to engage with the frameworks, consider improved upgrades in the methods they were used to, and be amongst the select few to volunteer in using it in their work.

The sheer energy and excitement was a pleasure to witness, and even after the final speaker of the night, it was hard to get many product managers and CRO specialists to leave the venue as they stood, engrossed, their minds ticking away, engaged in conversation and eager to use the ideas as soon as they could! Looking forward to hearing updates about people’s trials!

Speaker 3 – Bhavik Patel

Our very own host, Bhav Patel took the stage for the slot of final speaker, a man who needs no introduction, but for those who may not know of his recent change, I shall do anyway – Head of Conversion at Teletext Holidays. He performed a talk he had delivered not even a week before at another convention, which gave advice for building experiments on a shoestring, explaining how he had successfully achieved this since switching from his previous post at Moo.com.

He talked about having to educate the misconception of stakeholders who thought conversion rates of the company had to rival the Big Four and how experimenting with volume, length and quantity of tests would be an indicator of clarity on traffic to the website. He shared a few pearls of wisdom from his own experience and revealed some truths to which I’m sure most of the audience could relate.

The most visual of these stories was one of a chess analogy, where he had to arm himself with the knowledge against other stakeholders and teams in order to aid in decision-making. He admitted being a ‘jack of many trades’ had given him the opportunity to fill gaps by ‘filling in’, which was a powerful message on which to end. The main takeaways from this talk overall were firstly being mindful of conscious cross-overs during multiple tests across the webpages, not wasting time on low value tests and forcing yourself into test groups to challenge your intuitions.

Something different – Bhav & Aimi

Finally, for the ‘something different’ section of the night, we got to hear Bhav’s superb slam poem about the CRO world, ‘A Love Affair with Experiments’, the title of which was inspired by a poem I had performed just over a year ago at CRAP 8. Both the creativity and bravery of performing this was a wonderful surprise, and well received by the audience. I even had the opportunity to take up the stage again to recite some spoken word, sharing two short poems around the theme of loss. It was a great feeling to be there alongside Bhav while we shared pieces born of our mutual passion, despite our different career paths.

With a brand new CRAPPY Hour podcast already at its second episode in, and supportive new sponsors in iTech Media and Content Square (and of course the ongoing support from Pivotal London), the CRAP Talks event feels like it’s in full force. Consistently stripping away the noise, keeping it simple and creating a meetup community like no other in the tech/ecommerce space across London and other cities, this bar-raising hub for sharing ideas remains strong by living to its values. Looking forward to seeing you all in December!