Product Case Study
A New Data Strategy Begets a New Product Strategy
In 2015, MikMak was a new NYC start-up reimagining infomercials (like those on QVC or HSN) for a new generation of shoppers. The business was founded on the idea that humorous video advertising could garner brand affinity and mobile purchase conversion. At the time, the team was comprised of the founder/CEO, a merchandiser, a videographer, two engineers and myself. I was responsible for all aspects of product management and design for our mobile app and bespoke video CMS.
Here I share the process of designing and building a new analytics program to measure the performance of MikMak’s video ads and watching experience, and how this data helped reorient our product strategy.
MVP & Beta Launch
Initially, we built a simple iPhone app to bootstrap our content-production operations, learn from early user behavior, and establish brand partnerships. The app opened directly into a feed of shoppable videos (under one minute each). These short informercials hosted by social media influencers featured a mix of products. Users could navigate through merchandise categories and purchase products, but not much else.
MikMak released the app discretely without media or PR; we recruited a small cohort of beta users through a partner so that we might observe their behavior. Page-level eventing via Google Analytics showed me where users spent their time in our app, and which products interested them. Surveys and user interviews provided qualitative feedback on our video creative and merchandise assortment. My analysis of this data and user feedback indicated anecdotally that one type of content was producing a more impactful shopping experience: “how-to” videos. Intriguingly, the demo format performed across a number of categories — how to style beauty products, how to use kitchen gadgets, how to serve specialty foods.
Measuring “Product Intent”
Early on the MikMak team recognized that we wouldn’t effectively compete against mature platforms for actual purchases. Instead we preferred to focus on developing awareness, desire and intent to buy. MikMak’s value would be in producing affinity-driving ad creative for mobile and delivering it at the right time and place.
We hypothesized that interactions demonstrating affinity for a product should be looked at in aggregate — some people express interest by adding an item to their cart, while others might fave or share a video with friends. Inspecting and adjusting SKU options showed us that a shopper was interested enough to interrupt their viewing to learn more about the merchandise or brand. I began a a plan to capture these “product intent” actions and analyze them collectively.
A New Analytics Program
Determining how MikMak content led to product intent would require a new, more robust approach to collecting and analyzing behavior data from the app. I began designing a new analytics program (the data strategy and tooling solution) that would track interaction-level user events and content dimensions. I engaged a data-science consultant to help me specify taxonomies for interactions, ad content and users. (For example: which user-attributes accompany content-sharing? which of those originate locally on the device vs. user-data on our servers?) I delivered these as engineering requirements for our platform and client developers.
MikMak’s Analytics Architecture
New Interactions in the App
In parallel with implementing our new events tooling, I worked with engineering to iterate on the user experience to better capture shoppers’ interest: we fine-tuned video favoriting and cart interactions, added sharing capabilities, and disambiguated product detail elements both in our CMS and in the user interface. Through these changes we gained more a granular view of in-app behavior. But with so much detail available, I wanted to make sure the content and merchandising team wouldn’t lose sight of the overall success or failure of each video ad to drive product intent. MikMak would need new metrics to evaluate content performance across a variety of interactions that could be looked at in aggregate or by type.
New Sharing Flow
Refined Reporting
I set out to design product intent metrics by putting myself in the mindset of the future customers of my analytics program: MikMak’s CEO, merchandiser, and video producer. What questions would each ask about our content or shoppers? What data would help them improve ad creative and targeting? For instance, I considered that our content producers would want to understand whether the same ad format drove both app loyalty and product intent, and whether two videos for the same product would produce different engagement patterns. I further refined these questions with paper prototype dashboards the team might use to report and dig deeper on content performance.
I defined which types of interactions to track to measure intent: watching the videos (especially watching all the way through and replaying), favoriting an item or inspecting product details, and sharing the video or adding the item to cart were all actions that indicated some degree of consumer interest. Eventually, I hoped to be able to assess each of these actions to produce a holistic score for each video. But without significant historical data to draw from, that level of abstraction wasn’t yet attainable.
Product Intent Dashboard
To improve my composite metric for product intent, I carried forward an observation from the first version of the MikMak app and our beta cohort: brand-new users tended to tap around a lot, seemingly investigating the interface, and then often bounced out of the app altogether. Repeat users settled into longer sessions, watching numerous videos but only occasionally viewing details or favoriting products. So, when measuring which content drove engagement, I sought to dampen the impact of new users exploring the app. I defined parameters for recurring users in terms of total visits and time-lapsed since their last visit, and then limited our measure of product intent to actions taken by these users. (This approach also helped normalize our content performance metric during user acquisition campaigns.)
I designed and built dashboards in MixPanel to rank videos by aggregated product intent actions; the stacked bar chart visualization also displayed the breakdown of all interaction types. It allowed the team to analyze our ads along a set of content dimensions such as SKU, merchandise category, host, and video format. We could see, for example, that some of our hosts performed significantly better than others across merchandise categories. Using this metric, I affirmed the early observation that how-to videos captured shoppers’ attention and accrued interactions. In response MikMak gently shifted our assortment towards merchandise categories that lent themselves to entertaining demonstrations.
Iterating on Product Strategy
Analysis of content performance and product intent interactions also helped the business establish media partnerships by demonstrating that MikMak’s ad format could deliver product awareness and brand affinity. In shifting our focus almost solely to content performance we had to let go of our original desire for MikMak’s app to be a destination in itself. We adapted our product strategy by developing a second product, an embeddable player to stream ads on partner sites. My design for the new web player enabled lightweight guest checkout in context and routing to MikMak for further shopping; it still looked to capture shopper interest through all-up measurement of product intent even though some of the constituent interactions had changed.
A few years and many iterations later, MikMak continues to refine its offering, creating tools where consumers learn about and purchase products through shoppable videos on media properties and social channels. They now focus entirely on routing shoppers to established retailers, and show success in lowering bounce rates and increasing product consideration.
