Optimizing video for UX: What a developer needs to know

More than 80% of the content on the Internet is video. But despite its popularity, the mechanics of incorporating video into app experiences (along with related metrics) remain a challenge for developers.

Whether your degree of familiarity with video is uploading to YouTube or a service provider like Vimeo, your first experience trying to bring video natively to your website, web app, or mobile app will present layers of bespoke UX considerations. What do you think about viewing the video? experience. As with any other front-end experience, people have short attention spans, and the video user experience is critical to your video assets.

A newbie’s journey to the video user experience

There are all sorts of nuances to how video is rendered, how it looks on different devices, how the app and video infrastructure handle peak usage, native apps vs. mobile web, and many other considerations. The deeper you get into it, the more you’ll want some solid data to understand what’s really going on with your video at scale, in the wild. If we can turn to tools like Fathom or Google Analytics for site traffic and visitor behavior, and Lighthouse performance score for web page performance metrics like first paint with content and time to engage So where can we look for the equivalent metrics that we should care about? optimize the video user experience for our sites and applications?

When you start building a new app, you dream of people finding it and getting value from it, and who knows, maybe the usage will grow and you will have a huge success on your hands. At each step of the way, you as a developer have different needs when it comes to the type of infrastructure you need to scale your application, and different data points you need to understand user engagement and how to improve your offering. Video has the same kind of trajectory, where those simple user engagement questions you have early on evolve as you get real traction and scale with your video assets.

So, let’s consider the “hierarchy of video data needs” (with due credit to Dr. Maslow), outlining the type of data you need when you launch an application, all the way to the pinnacle, when you can invest in fine-tuning your video operation. video to optimize the user experience.

multiplexor

Reproduction

Before you invest in scaling and optimizing your video platform, you’ll first want to know if your videos are playing correctly. This is critical data to understand. If viewers encounter a bad experience in the first few moments of finding a video and pressing play, there’s no hope of keeping them around.

The main thing you’ll want to pay attention to is simple: Are your videos playing? You have your player and video aligned, but if there’s a problem in the video pipeline for live video, your users hit play and there’s a long delay to start, or worse, nothing happens, that’s bad. And there are a multitude of things that could go wrong with playback:

• A problem streaming your live stream from your source device
• An updated player that your video does not support
• New JavaScript on a page that doesn’t work correctly
• An up-to-date browser that is not supported
• Expired SSL Certificate Issues
• A bad URL redirect
• Different experiences across browsers and devices

Asking these key questions will help you understand video playback:

• Did your video start for viewers?
• How long did it take to start?
• Did the video start without an error?
• In the case of live video, did your video go through the encoding pipeline to delivery?
• Did the user actually get to the playback or did they exit before the playback started?
• How many views are your videos racking up?

viewer engagement

The next step is to understand basic viewer engagement metrics. If you’re posting videos to the world, you need to understand if people are watching and start to understand what types of videos are the most popular.

Some things you’ll want to track include:

• start the video
• unique viewers
• Time to play

These metrics are valuable to businesses and teams of all sizes, from developers monitoring their services to product owners trying to understand the success of their product, to marketing teams who need detailed usage data. There are a variety of ways these metrics can be useful. For example:

• KPI Metrics – Track how many users you have this month compared to previous months and measure your success over time.
• Capacity planning: Measure how many views you’re actually serving for your events to ensure you have the server resources to meet demand.
• Maintenance: Provide a quantitative measure of low and high traffic periods to operations teams so they can determine when platforms can be taken offline for maintenance.

observability

For companies where the stakes are higher with video, let’s say, where video is the product, or is a critical touch point for a large volume of customers; now you’re starting to experience demand for the kind of team collaboration and sharing requirements you see in mature products and features (like DevOps, for example). Here’s what those considerations begin to look like:

• Information sharing starts small and informal: taking screenshots, pasting data into PowerPoint or email, and exporting CSV. But this is rapidly evolving from the domain of a single business or technical stakeholder, and beginning to inform other parts of the business, such as operations, marketing, customer service, and finance teams.
• On the cost side, stakeholders like financial and product organizations will want more information tying video viewing to specific costs like encoding and CDN. This will help with cost management, but requires additional reporting.
• Eventually, you will need to put data collection, sharing, and analysis into practice. The need for higher level insights into the data will push you towards observability tools like Grafana, New Relic or Datadog, systems for operational reporting and even ingestion of data into data lakes for further processing and analysis.

This integration layer will be driven by higher-level questions in the data, as video becomes a bigger part of your business and the product experiences you deliver to your customers.

content information

As we move up the hierarchy, we enter the realm of companies that invest heavily in video and use content to drive business results. Video is expensive to create and deliver, and companies that invest significantly in video will want better data on what people are watching and what is working well, so they can adjust their video production priorities accordingly. This step implies the following, for example:

• Deeper analysis of display behavior within a piece of content. Seeing which videos are being played is one thing, but more granular data about what viewers are doing during a viewing session returns a video completion percentage, lets you build engagement heatmaps within the video, and more.
• Conversion rate tracking. If you’re a retail or e-commerce business bringing video to the point of sale, correlating video playback data with your sales conversion KPIs matters most to you.
• Expand beyond views to unique viewers and new users vs. returning users.

Quality of experience (QoE)

At the top of the hierarchy of needs are the truly thorny concerns about the video experience. This is where streaming providers with millions of viewers have video engineering teams that dig into the most granular optimizations to make improvements that can have massive impact at scale and drive business forward.

“Quality of experience” is a coded standard that describes the video experience from the point of view of viewers. It includes things like:

• Video stops: How widely viewers experience video stops across their videos, platforms, geographies, or networks.
• Responsiveness – How responsive is video playback on the different devices, different locations, different CDNs, different ISPs, and different video formats (HLS, Dash, etc.) that it supports.
• Failure Handling: How to address buffering, playback failures, and other strategies so that a user doesn’t experience any issues from a two-second internet interruption.

Those who have video UX vs. those who don’t

In industries like fitness, e-commerce, and retail, more and more companies are embedding live video-on-demand and even streaming video directly into their products, websites, and mobile apps. We’re all familiar with the more obvious examples like Peloton, but we can expect to see more videos pop up in online product catalogs (“live shopping” experiences) and sales interactions (speaking directly with customer service through of a live video instead of launching a separate Zoom meeting). Expect more companies to use user-uploaded videos as well (to increase app and community engagement).

For some companies, this push into video is so central to the evolution of their products that one percent QoE optimizations are a worthwhile pursuit. For smaller businesses that are just getting to grips with native video, understanding the basics of engagement and playback performance may be all they need.

But I think one thing is for sure: video will become increasingly familiar to website and app developers, and it’s only a matter of time before you find yourself trying to figure out which optimizations in the hierarchy you should spend more time on. .

Steve Lyons is a director of product management at Mux, a video infrastructure platform for developers.