15.3 Augmented and virtual reality for data storytelling

Augmented and virtual reality are transforming data storytelling. These immersive technologies allow journalists to create interactive 3D visualizations and simulations, bringing complex data to life in ways that engage and captivate audiences.

AR and VR offer unique opportunities for data journalism, from virtual tours of data-rich environments to interactive games that make exploring data fun. While technical challenges exist, these tools have the potential to revolutionize how we understand and interact with data-driven stories.

Immersive data stories with AR/VR

Augmented Reality (AR) and Virtual Reality (VR) technologies

• Augmented Reality (AR) overlays digital information onto the real world, allowing users to interact with data in a more engaging and contextual way (Google Glass, Pokemon Go)
• Virtual Reality (VR) creates a fully immersive digital environment that users can explore and interact with, providing a more immersive experience for data-driven stories (Oculus Rift, HTC Vive)
• AR and VR can be used to create interactive data visualizations, allowing users to explore and manipulate data in a more intuitive and engaging way
  • Users can interact with 3D charts, graphs, and maps
  • Data can be presented in a more immersive and contextual way (virtual tour of a city with embedded data points)

Creating immersive data stories with AR/VR

• Immersive data stories can be created using 360-degree videos, 3D models, and interactive elements, providing a more engaging and memorable experience for audiences
  • 360-degree videos can transport users to a specific location or event related to the data story (virtual tour of a refugee camp)
  • 3D models can be used to represent complex data sets or concepts in a more tangible and intuitive way (interactive model of a human brain)
  • Interactive elements can allow users to explore and manipulate data in real-time (adjusting parameters in a simulation)
• AR and VR can be used to create data-driven simulations and games, allowing users to explore complex data sets in a more interactive and engaging way
  • Simulations can allow users to experience the impact of different scenarios or decisions (climate change simulation)
  • Games can incentivize users to explore and engage with data in a more playful and competitive way (data-driven escape room)

Technical requirements for AR/VR journalism

Hardware and software requirements

• AR and VR require specialized hardware and software, such as head-mounted displays, motion tracking systems, and 3D modeling tools
  • Head-mounted displays (HMDs) provide a stereoscopic display and motion tracking for immersive VR experiences (Oculus Rift, HTC Vive)
  • Motion tracking systems allow users to interact with virtual objects using hand gestures or body movements (Leap Motion, Microsoft Kinect)
  • 3D modeling tools are used to create the virtual environments and objects for AR/VR experiences (Autodesk Maya, Blender)
• AR and VR experiences can be created using game engines such as Unity or Unreal Engine, which provide a framework for building interactive 3D environments
  • Game engines handle rendering, physics, and user input for AR/VR experiences
  • They provide a visual scripting language for creating interactive elements and behaviors (Unity's Visual Scripting, Unreal's Blueprints)

Delivery platforms and technical limitations

• AR experiences can be delivered through mobile apps or web-based platforms, while VR experiences typically require a dedicated VR headset
  • Mobile AR apps can be downloaded from app stores and run on smartphones or tablets (Apple's ARKit, Google's ARCore)
  • Web-based AR experiences can be accessed through a web browser using WebXR standards (Google's WebXR Experiments)
  • VR experiences require a dedicated VR headset connected to a computer or gaming console (Oculus Rift, PlayStation VR)
• Journalists need to consider the technical limitations of AR and VR, such as the processing power of devices and the need for high-quality 3D assets
  • Mobile devices have limited processing power compared to desktop computers, which can impact the complexity of AR/VR experiences
  • High-quality 3D assets require significant storage space and bandwidth to download and render in real-time
• Creating high-quality AR and VR experiences requires a team with diverse skill sets, including 3D modeling, programming, and user experience design
  • 3D modelers create the virtual objects and environments for AR/VR experiences
  • Programmers develop the interactive elements and behaviors using game engines or web frameworks
  • User experience designers ensure that the AR/VR experience is intuitive and engaging for users

Effectiveness of AR/VR for data narratives

Increased engagement and retention

• AR and VR can increase user engagement and retention by providing a more immersive and interactive experience compared to traditional media
  • Users are actively engaged in exploring and interacting with the data, rather than passively consuming it
  • The immersive nature of AR/VR can create a sense of presence and emotional connection to the data story
• The novelty and "wow factor" of AR and VR can attract new audiences and generate buzz around data-driven stories
  • AR/VR experiences can be promoted as cutting-edge and innovative, attracting tech-savvy audiences
  • The immersive nature of AR/VR can create a memorable and shareable experience that generates word-of-mouth buzz

Communicating complex data and abstract concepts

• AR and VR can be particularly effective for explaining complex data sets or abstract concepts, as they allow users to visualize and interact with data in a more intuitive way
  • 3D visualizations can make complex data sets more accessible and understandable (interactive model of a neural network)
  • Abstract concepts can be represented in a more tangible and relatable way (VR simulation of a black hole)
• AR and VR experiences can be designed to encourage user exploration and discovery, allowing audiences to uncover insights and draw their own conclusions from the data
  • Users can navigate through a virtual environment and discover data points at their own pace
  • Interactive elements can allow users to test hypotheses and see the impact of different scenarios (adjusting parameters in a simulation)
• User testing and analytics can be used to measure the effectiveness of AR and VR experiences in engaging audiences and communicating data-driven narratives
  • User testing can provide qualitative feedback on the usability and engagement of the AR/VR experience
  • Analytics can track user behavior and engagement metrics, such as time spent in the experience and interactions with data points

Designing AR/VR experiences for data journalism

Designing for purpose and narrative

• AR and VR experiences should be designed with a clear purpose and narrative in mind, using the unique capabilities of the medium to enhance the story
  • The purpose of the AR/VR experience should align with the goals of the data-driven story (raising awareness, encouraging action, etc.)
  • The narrative should be structured to guide users through the experience and highlight key data points and insights
• The data should be presented in a clear and accessible way, using visual cues and interactive elements to highlight key insights and trends
  • Data visualizations should be designed for legibility and clarity in a 3D environment
  • Interactive elements should provide context and guidance for interpreting the data (annotations, tooltips, etc.)

User experience and interaction design

• The user interface and interaction design should be intuitive and easy to use, with clear instructions and feedback to guide users through the experience
  • The user interface should be minimalist and unobtrusive, allowing users to focus on the data and narrative
  • Interaction design should be consistent and predictable, using familiar gestures and controls (gaze-based selection, hand tracking, etc.)
• The visual design should be engaging and immersive, using high-quality 3D models, textures, and lighting to create a realistic and compelling environment
  • 3D models should be optimized for performance and visual quality, using techniques such as low-poly modeling and texture baking
  • Lighting and shading should be used to create a sense of depth and atmosphere, guiding users' attention to key data points and elements
• The experience should be optimized for the target platform and device, taking into account factors such as screen size, processing power, and user input methods
  • Mobile AR experiences should be designed for smaller screens and touch-based input
  • VR experiences should be optimized for the specific capabilities and limitations of the target VR headset (field of view, refresh rate, etc.)
• User testing and iteration should be an integral part of the design process, with feedback from users used to refine and improve the experience over time
  • User testing should be conducted at various stages of the design process, from early prototypes to final implementations
  • Feedback from users should be used to identify usability issues, refine the narrative and data presentation, and optimize the overall experience