Disney / Lenovo’s Star Wars Jedi Challenges AR device invites players to engage in lightsaber duels with life-size Sith lords (Disney)
Hollywood and Silicon Valley, Bound Together
Technology and entertainment have always bolstered one other.
Innovation in technology enables new platforms for content creation and the widespread distribution of meaningful stories and experiences. Consumers continue to be drawn to new technology primarily by the content and experiences it enables. Content, as the saying goes, is king.
Audiences’ primary emotional connection is to the imagined world, character, feeling, or memory associated with content - not the device, file, or site they used to experience it. A usable, accessible technology platform can serve to further enhance and strengthen emotional connection to a story - but only when designed in an integrated way that is central to and honors creative intent and vision.
Over the last century, we have witnessed the entertainment industry lagging in advances in technology and being slow to adopt emerging platforms. Examples span from the early resistance by some studios of motion picture sound, to fear of peer-to-peer file sharing, to the more recent slow development by studios to catch up with streaming media services like Netflix and user generated content behemoths such as YouTube, Twitch, and TikTok.
And regrettably within the last decade - with the exception of some digital effects and games studios - advancements in the tech industry have developed largely separately from the world of entertainment production and the content it depends upon to succeed with users.
BUT WHY? TECHNOLOGY AND ENTERTAINMENT NEED EACH OTHER NOW MORE THAN EVER
Audiences have historically adopted the methods necessary to engage in their favorite entertainment experiences, such as going to a public theater or a movie theater, recording a television program on a VCR or DVR, signing up for a Netflix or HBO NOW subscription to see a particular show, contributing fanfic ideas or feedback on social media, or searching for a user generated video only available on YouTube. All it requires, of course, is a motivated interest in connecting with that story or experience, often times over and over again.
Currently Apple, Facebook’s Oculus, Amazon, Netflix, Sony, Samsung, Snap, and more are all in a race to deliver increasingly personalized content (media streaming services) and more compelling and immersive platforms (such as AR and VR) to increase engagement with users and attract new customers.
And the battle for securing cutting-edge platforms and devices in consumer homes (with the development of smart speakers & displays, connected TVs, cameras, and head-mounted displays, just to name a few) is becoming more and more dependent on the content necessary to gain user interest and drive engagement. And content such as television, film, and music will need to be designed to leverage and showcase the new wave tech-forward capabilities such as mixed reality, AI, 4k cameras, biometric sensors, facial recognition, and more.
Clockwise, from top left: Netflix announces development of ‘choose your own adventure’ Black Mirror episode for 2019, @Lilmiquela is an AI Celebrity on Instagram, Spectator e-sports on the popular Twitch streaming platform, owned by Amazon.
Thanks to the increasing impact of social media and data-curated streaming services, audiences have started to expect a more personal, direct interaction with their entertainment. Increasingly, people don’t just want to view a story and characters from a distance, they wish to experience it - even influence it - themselves. The evidence is rampant, from increased user-generated (or ‘prosumer’) content expanding story properties and marketing, the exploding popularity of cosplay and location-based entertainment such as immersive theater and escape rooms, continuing development and integration of virtual and augmented reality experiences where viewers are ‘part of’ a world or acting directly with a character, and more. Audiences are enthusiastically responding to new experiences which offer more personal interaction and a deeper connection to stories and characters, which of course data and technology can and have enabled on a large scale.
THE FUTURE IS IMMERSIVE, PERSONAL INTERACTIONS
National Theater in London Offers Glasses With Live Subtitles (Courtesy New York Times)
Oculus Touch controllers precisely recreate users’ hands and gestures in VR media (Oculus Quest)
Meanwhile, the entertainment and tech industries still struggle to develop products together with shared goals, motivations, and timelines. We’re still a long way off from creating truly seamless, cohesive technology experiences which prioritize and enhance content and emotional connection over layers of paywalls and complexity.
A few inspiring examples of product experiences demonstrating seamless integration of technology with the magic of story and character include:
AR head mounted displays for theater audiences which integrate subtitles in various languages (without obscuring the show itself)
Disney’s Star Wars Jedi Challenges AR kit for the home which enables a first-person lightsaber dual with a Sith lord
Oculus Touch controllers with haptic feedback recreating human hands and gestures within VR worlds - enabling users to feel the experience of petting a character directly
Amazon Echo smart speaker acting as gateway through which audiences can quickly and easily access a single story from over 400,000 audiobooks
These product experiences integrate technology and content in such a way that the devices act as the means, not the ends, to innovative entertainment while playing a critical role in access and immersion.
So how do we work better together to make truly compelling experiences for audiences that integrate the power of new technologies and interactions?
Developing and shipping meaningful entertainment experiences like those listed above will require a tighter than ever connection among entertainment and technology industries and practices. We need to ensure more fluent sharing of knowledge, processes expertise, and tools so that we can collaborate from the ground up and deliver to audiences (and users) the personalized, realistic, and overall seamless and delightful experiences they crave.
RECOGNIZING SIMILARITIES AND DIFFERENCES
Entertainment and tech inarguably share a goal to deliver audiences (and users) meaningful, compelling experiences that endure and build connection to brand.
And from one point of view, production practices across entertainment and tech can be viewed as quite similar, if given different titles:
Tech /Digital Production
Investor
Product Managers
Development & Content Authors
User Experience Design
Visual UI Design, Industrial Design
Cross-Platform Development
Elevator-pitch ideas to illustrate impact i.e “Uber for air travel”
Interaction Design
User Experience Testing & Research
Data Analytics & Qualitative Research
Entertainment
Executive Producer
Show-runners / Producers
Storytelling & Writing
Direction
Artists & Art Department
360 Marketing and Brand Strategy
High concept pitches to predict audience appeal. i.e. “Jaws in space” (Alien)
Storyboarding
Pitching & Feedback
Box Office Numbers and Sales
…and more
Pixar Story Artist Valerie Lapoint pitches story ideas for feedback and notes (Kahn Academy / Pixar in a Box)
The author user testing the Microsoft Surface mixed reality device to determine its intuitiveness and usability (Microsoft)
However a few key differences should also be considered which heavily impact production goals and approaches.
Overall tolerance for risk. Hollywood is the undisputed master of storytelling and emotional impact. Yet film, television, and music production yields such thin profit margins that Hollywood is equally notorious for strategies of risk avoidance and investment in established, predictable properties and experiences. Conversely, one can look at tech’s foundation of venture capital wealth for unbounded optimism and risk taking in technological innovation, demonstrated through the creation of computer animation & digital effects tools, VR, and more. Few tech ventures succeed, but when they do they are often widely impactful to users.
Anticipated length of engagement. In entertainment, audiences fully submitting submit themselves to an experience for a limited time (such as 2 hours). This context allows creators to deliver a pre-scripted user experience that audiences abide by and can provide full attention to. In tech, the intent of building consumer technology platforms is for the user to be able to use, adopt, and live with the technology on an ongoing basis. This requires product experiences to be designed, developed, and tested to be primarily intuitive and easy to use - if they are to be successfully adopted. Technology teams constantly test with users, their version of audiences, to make sure the product is centered around their needs and expectations, not the developers’.
The introduction of access and usability obstacles. Within the nature of developing and releasing new, innovative technology is the introduction of entirely new devices, uses, functions, and interactions. Unlike a typical song format or story spine, the goal is actually to break out of predictable formula. New technology requires that users willingly try something entirely new, learn and understand it, successfully interact with it, and adopt it. Think for example of the first time a user encounters a virtual reality head-mounted display or new game controllers; users are frequently not entirely successful with the experiences at first use, and the products are often unintuitive to use, requiring considerable guidance. And when we consider a wide range of audience or users with varied experience with technologies, the visible usability problems become more acute. The integration of approachability, usability, and easy access for all remain major challenges for technology development (and the tech industry as a whole) and can critically hinder access to entrainment content.
How to Move Forward Together
In practice, we’ll need to
Value - not fear - each other. Tech is not the enemy of creative! When done well, it can and should act primarily in service of great story and emotional impact. Tech on the other hand absolutely must prioritize creative input if it is to be compelling and successful, baking in decision-making authority and development time dedicated to creative inspiration, exploration, and iteration.
Share pitches, ideas, and explorations more frequently - especially in areas where tech and creative ideas converge - so that we can drive experiences that leverage each others’ developments.
Testing sketched content with an early cardboard prototype of a hardware device. This practice enables cheap, rapid iteration to ensure an intuitive ‘total experience’ across content and tech design. (Stanford)
Develop and test concepts together. Combine early content ideas and technology prototypes to test the overall experience earlier and more frequently with audiences and users. For example, artist sketches and storyboards can be integrated into paper prototype testing and ‘Wizard of Oz’ practices (common in tech) to ensure that the combination of content and technology is intuitive, fluid, and delightful.
Trust each other's expertise. Technologists would benefit from giving artists and storytellers’ more of a platform to contribute early and often when developing interactive experiences, such as AR and VR. And artists must find ways to work more closely in partnership with technology design teams to test and iterate stories and characters quickly, so they can match the speed of Internet-based feedback and ensure a story interaction makes sense within the limitations of the technology.
Supplement each other’s risk tolerance. Tech should invest in more creative story talent that can help stretch the boundaries and impact of immersive experiences. Storytellers can imagine their world in the context of various platforms and levels of viewer interaction and influence.
Tech in particular will need to step up the human-centered design processes and work harder to remove barriers of usability, lack of familiarity, and user discomfort that still plagues most advanced technology products. This includes addressing critical barriers of trust with use of user data, and including inclusive design processes and accessibility. More than ever, storytellers will depend on platforms which enable them to reach as many audiences as possible.
Consider Audiences, Viewers, Consumers, Players, and Users as the same person at various moments of engagement. Once we adopt a shared perspective to consider all of these roles as various points of a single journey throughout a product experience, we have made major headway into carting a map for a more collaborative development process across entertainment and tech.
Look to the Legacy of Video Games For Tools
We must carve a path together sharing expertise, process, and techniques, and ideally build new tools together to create truly engaging, cohesive experiences.
Fortunately, one example we can look to is the development of video games over the past 50 years, which can show us some valuable clues how to achieve a productive synergy. In my next blog post, I’ll outline tools that the video game industry has developed to create cohesive and compelling experience across advances in content and technology.