7.4 Photogrammetry and 3D scanning for realistic assets
4 min read•august 7, 2024
and 3D scanning are game-changers for creating realistic assets in AR/VR. These techniques capture real-world objects and environments, turning them into detailed 3D models. They're essential for bringing authenticity to virtual worlds.
From photogrammetry software to LIDAR and , there are various ways to digitize the physical world. Each method has its strengths, helping developers choose the right tool for their project's needs and budget.
Photogrammetry Techniques
Fundamentals of Photogrammetry
Top images from around the web for Fundamentals of Photogrammetry
MLab in the Humanities » University of Victoria » Testing Photogrammetry for the Humanities View original
Is this image relevant?
Frontiers | A Review of Photogrammetry and Photorealistic 3D Models in Education From a ... View original
Is this image relevant?
Frontiers | A Review of Photogrammetry and Photorealistic 3D Models in Education From a ... View original
Is this image relevant?
MLab in the Humanities » University of Victoria » Testing Photogrammetry for the Humanities View original
Is this image relevant?
Frontiers | A Review of Photogrammetry and Photorealistic 3D Models in Education From a ... View original
Is this image relevant?
1 of 3
Top images from around the web for Fundamentals of Photogrammetry
MLab in the Humanities » University of Victoria » Testing Photogrammetry for the Humanities View original
Is this image relevant?
Frontiers | A Review of Photogrammetry and Photorealistic 3D Models in Education From a ... View original
Is this image relevant?
Frontiers | A Review of Photogrammetry and Photorealistic 3D Models in Education From a ... View original
Is this image relevant?
MLab in the Humanities » University of Victoria » Testing Photogrammetry for the Humanities View original
Is this image relevant?
Frontiers | A Review of Photogrammetry and Photorealistic 3D Models in Education From a ... View original
Is this image relevant?
1 of 3
- Photogrammetry involves capturing multiple overlapping photographs of an object or environment from different angles and positions
- Uses computer vision algorithms to analyze the photographs and extract 3D information
- Relies on the principle of triangulation to determine the 3D coordinates of points in the photographs
- Requires a high degree of overlap between photographs (typically 60-80%) to ensure accurate 3D reconstruction
Structure from Motion (SfM) Process
- is a specific photogrammetry technique that automatically extracts 3D structure from a series of 2D images
- SfM algorithms identify and match feature points across multiple images to estimate camera positions and orientations
- Creates a sparse representing the 3D structure of the scene or object
- Dense point cloud is generated by interpolating additional points between the sparse points to capture more detailed geometry
Point Cloud to Mesh Conversion
- Point cloud is a set of 3D points in space representing the surface of an object or environment
- algorithms connect the points in the point cloud to create a polygonal mesh surface
- is a common method for mesh reconstruction, creating a network of triangles that closely approximates the object's surface
- techniques, such as mesh decimation and smoothing, can be applied to reduce the mesh complexity and improve its quality
Texture Mapping and Baking
- is the process of projecting and combining the color information from the photographs onto the reconstructed 3D mesh
- is used to define how the 2D texture coordinates correspond to the 3D mesh surface
- is created by unwrapping the 3D mesh and packing the texture information into a single 2D image
- High-resolution textures can be baked to capture fine details and realistic appearance of the object or environment (4K or 8K textures)
3D Scanning Methods
LIDAR Scanning
- LIDAR (Light Detection and Ranging) is an active 3D scanning technology that uses laser light to measure distances
- Emits laser pulses and measures the time it takes for the light to bounce back from the object's surface
- Creates a dense point cloud by scanning the object or environment from multiple viewpoints
- Provides high accuracy and can capture fine details, making it suitable for industrial and engineering applications (reverse engineering, quality control)
Structured Light Scanning
- Structured light scanning projects a pattern of light (stripes, dots, or grids) onto the object's surface
- Cameras capture the deformation of the projected pattern caused by the object's geometry
- Triangulation is used to calculate the 3D coordinates of points on the object's surface based on the deformation of the pattern
- Provides high-resolution and accurate 3D scans, commonly used for small to medium-sized objects (dental impressions, artifacts)
Comparison of 3D Scanning Methods
- 3D scanning methods differ in terms of accuracy, resolution, speed, and cost
- offers high accuracy and long-range capabilities but can be expensive and requires specialized equipment
- Structured light scanning provides high-resolution scans but is limited to smaller objects and requires controlled lighting conditions
- Photogrammetry is more accessible and cost-effective but may require more manual processing and have lower accuracy compared to active scanning methods
Photogrammetry Tools
Photogrammetry Software Workflow
- Photogrammetry software automates the process of generating 3D models from photographs
- Typical workflow includes importing photographs, aligning cameras, generating point clouds, creating meshes, and texturing
- Popular photogrammetry software includes , , and
- Cloud-based photogrammetry services, such as and , offer web-based processing and storage solutions
Key Features of Photogrammetry Software
- and to estimate camera positions and orientations
- using multi-view stereo algorithms
- Mesh reconstruction and optimization tools to create a polygonal mesh from the point cloud
- and baking functionality to project photographic details onto the mesh
- Editing tools for cleaning up and refining the generated 3D models (hole filling, noise reduction)
- Export options to common 3D file formats (OBJ, FBX, PLY) for use in other 3D software
Considerations for Choosing Photogrammetry Software
- Ease of use and learning curve, especially for beginners
- Compatibility with different camera types and file formats (DSLR, drone, smartphone)
- Processing speed and hardware requirements for handling large datasets
- Quality and accuracy of the generated 3D models
- Integration with other 3D software and pipelines (CAD, game engines, VFX)
- Cost and licensing options (one-time purchase, subscription, educational discounts)
Key Terms to Review (30)
Agisoft Metashape: Agisoft Metashape is a photogrammetry software that enables users to create high-quality 3D models from a series of 2D images. It processes photographs taken from various angles and generates realistic, textured 3D assets suitable for applications in augmented reality, virtual reality, gaming, and more. This software plays a crucial role in digitizing real-world objects and environments for various digital applications.
AliceVision Meshroom: AliceVision Meshroom is an open-source 3D reconstruction software that enables users to create detailed 3D models from a series of photographs through photogrammetry. It uses advanced computer vision algorithms to process images and generate realistic 3D assets, making it a popular choice for developers and artists in the realm of augmented and virtual reality.
ASTM Standards: ASTM Standards are technical standards developed by ASTM International that provide guidelines and specifications for a wide range of materials, products, systems, and services. These standards are crucial in ensuring quality, safety, and efficiency across various industries, including construction, manufacturing, and technology, facilitating consistent practices in processes like photogrammetry and 3D scanning for creating realistic assets.
Autodesk Recap Photo: Autodesk Recap Photo is a software tool designed for processing and managing photogrammetry data to create 3D models and point clouds from 2D images. It leverages advanced algorithms to stitch images together, allowing users to generate highly detailed and accurate 3D representations of real-world assets, which can be crucial in fields like architecture, construction, and visual effects.
Bundle adjustment: Bundle adjustment is a mathematical optimization technique used in computer vision and photogrammetry to refine 3D reconstructions by minimizing the error between observed image points and projected 3D points. This process improves spatial mapping accuracy and enhances the quality of 3D models by adjusting the camera parameters and the structure of the scene simultaneously, resulting in a more accurate representation of the environment.
Camera alignment: Camera alignment is the process of positioning and orienting a camera in relation to the objects being captured, ensuring accurate representation of geometry, scale, and perspective. This is crucial for creating realistic 3D assets through photogrammetry and 3D scanning, as misalignment can lead to distortions and inaccuracies in the final output. Proper camera alignment helps achieve consistent lighting, shadowing, and depth perception in the generated models.
Cultural heritage preservation: Cultural heritage preservation refers to the process of safeguarding, maintaining, and protecting tangible and intangible cultural heritage for future generations. This includes historical sites, artifacts, traditions, and languages that are integral to a community's identity. Utilizing techniques such as photogrammetry and 3D scanning plays a crucial role in documenting these assets accurately, ensuring they can be studied, appreciated, and even replicated digitally.
Delaunay Triangulation: Delaunay triangulation is a mathematical technique used to create a mesh of triangles from a set of points in a plane, ensuring that no point is inside the circumcircle of any triangle in the mesh. This method is significant for its ability to optimize spatial representation and maintain a balance between triangle size and shape, making it ideal for applications in areas like spatial mapping and creating realistic 3D assets through photogrammetry and scanning techniques.
Dense point cloud generation: Dense point cloud generation refers to the process of creating a highly detailed and accurate representation of a physical environment or object in the form of numerous spatial data points. This technique is commonly employed in photogrammetry and 3D scanning, where multiple images or sensor data are used to capture the intricate details of surfaces, allowing for realistic asset creation. The resulting dense point cloud can be further processed to create 3D models, providing a foundation for virtual and augmented reality applications.
Digital Elevation Model (DEM): A Digital Elevation Model (DEM) is a 3D representation of terrain's surface created from terrain elevation data. It is crucial for modeling and analyzing landscapes in various fields, including augmented and virtual reality, as it provides a foundation for creating realistic environments. DEMs enable the visualization of topographic features, supporting the generation of realistic assets by allowing users to understand how terrain interacts with light and shadows.
Game engine integration: Game engine integration refers to the process of incorporating various technologies, tools, and assets into a game engine to create a cohesive environment for developing interactive experiences. This integration allows developers to utilize realistic assets, including those generated through techniques such as photogrammetry and 3D scanning, to enhance the visual fidelity and realism of their projects. It connects the underlying code of the game engine with the art assets and functionalities needed to deliver engaging user experiences.
ISPRS Guidelines: The ISPRS Guidelines refer to a set of standards and best practices established by the International Society for Photogrammetry and Remote Sensing (ISPRS) for the use of photogrammetry and remote sensing technologies. These guidelines are crucial for ensuring the accurate capture, processing, and representation of 3D data, making them essential for creating realistic assets in various fields, such as architecture, gaming, and virtual reality.
Kraus: Kraus refers to a mathematical model used in quantum mechanics that describes the effects of decoherence and the interaction of quantum systems with their environment. This model is crucial for understanding how classical information emerges from quantum systems and is especially relevant in applications like photogrammetry and 3D scanning, where realistic assets require precise data representation and manipulation.
Lidar scanning: Lidar scanning is a remote sensing technology that uses laser pulses to measure distances and create precise, three-dimensional representations of physical environments. This method captures detailed spatial data, which can be transformed into realistic 3D models for various applications, including mapping, surveying, and creating realistic assets for virtual and augmented reality experiences.
Mesh optimization: Mesh optimization is the process of improving the quality and performance of 3D models by reducing their complexity without sacrificing visual fidelity. This technique involves simplifying the geometry of a mesh while maintaining its essential features, which is crucial for creating realistic assets from photogrammetry and 3D scanning, as it ensures efficient rendering and better performance in augmented and virtual reality environments.
Mesh reconstruction: Mesh reconstruction is the process of creating a 3D representation of an object or environment from data points, typically using techniques like photogrammetry or depth sensing. This method allows for the accurate representation of real-world objects in virtual spaces, enabling the integration of anchors and world-locked content in augmented reality applications. As a foundational aspect of creating realistic 3D assets, mesh reconstruction serves as a bridge between the physical and digital realms, enhancing user experiences in immersive environments.
Photogrammetric Society: A photogrammetric society refers to a collective or organization focused on the study and practice of photogrammetry, which is the science of making measurements from photographs, particularly for recovering the exact positions of surface points. This society plays a significant role in advancing the techniques and technologies used in photogrammetry, fostering collaboration among professionals, and promoting education and standards within the field. Through shared knowledge and resources, a photogrammetric society enhances the quality and accuracy of 3D scanning for realistic assets in various applications such as mapping, architecture, and virtual reality.
Photogrammetry: Photogrammetry is the science of making measurements from photographs, particularly for recovering the exact positions of surface points. This technique is essential for creating accurate 3D models of real-world objects and environments, allowing for detailed visualization in augmented and virtual reality applications. By capturing multiple images from different angles, photogrammetry enables the reconstruction of complex shapes and textures, making it a vital tool for generating realistic assets in digital media.
Pix4d: Pix4D is a software suite that specializes in photogrammetry, allowing users to create 3D models and maps from 2D images. It plays a crucial role in transforming visual data captured by drones and cameras into realistic assets for various applications, including architecture, engineering, and surveying. With powerful tools for image processing and analysis, Pix4D enhances the quality of 3D representations and helps in decision-making processes across multiple industries.
Point cloud: A point cloud is a collection of data points defined by coordinates in a three-dimensional space, often generated from 3D scanning or photogrammetry processes. These points represent the external surface of an object or environment, enabling the creation of highly detailed and realistic digital models. Point clouds serve as the foundational data for various applications, including virtual reality, augmented reality, and 3D modeling, allowing for accurate visualizations and analyses of real-world objects.
Point Cloud Generation: Point cloud generation is the process of capturing spatial data from the physical world and converting it into a digital representation, typically consisting of a large number of points defined by their 3D coordinates. This method is essential for accurately modeling environments and objects in augmented and virtual reality applications, enabling systems to understand and interact with the surrounding world effectively.
Real-time rendering: Real-time rendering is the process of generating images on-the-fly, allowing for immediate visual feedback as scenes are created or modified. This technique is essential in applications like video games and virtual reality, where user interaction demands that visuals are produced at a rapid pace, typically at 30 to 60 frames per second or more. This capability has seen significant advancements through improved algorithms and hardware, enhancing realism in immersive experiences.
Realitycapture: Reality capture refers to the process of collecting, digitizing, and converting real-world objects and environments into accurate 3D models using various techniques like photogrammetry and 3D scanning. This technology is crucial for creating realistic assets in virtual and augmented reality experiences, as it enables the precise representation of physical spaces and objects, thereby enhancing immersion and authenticity.
Structure from Motion (SfM): Structure from Motion (SfM) is a computer vision technique that allows the creation of 3D models from a series of 2D images taken from different angles. It identifies key points in the images and tracks their movement across the frames to reconstruct the 3D structure of the scene. This process is crucial for generating realistic assets, as it bridges the gap between 2D photography and detailed 3D modeling, making it widely used in photogrammetry and 3D scanning.
Structured light scanning: Structured light scanning is a 3D scanning technique that uses a series of projected light patterns onto an object to capture its shape and texture. This method involves projecting a known pattern of light onto the surface of an object and analyzing the deformation of the pattern to reconstruct a 3D model. It is widely utilized in applications requiring high accuracy and detail, especially for creating realistic digital assets.
Texture atlas: A texture atlas is a large image that contains multiple smaller textures or sprites packed together in a single file. This technique is used primarily to optimize rendering performance in graphics applications by reducing the number of texture bindings needed during rendering. By grouping various textures into one atlas, developers can enhance efficiency and minimize the overhead associated with switching between different texture files.
Texture baking: Texture baking is a process used in 3D graphics to pre-render and store surface details such as colors, lighting, and shadows onto a texture map. This technique allows artists to create highly realistic assets by capturing the visual characteristics of complex surfaces and applying them efficiently during real-time rendering. It streamlines workflows in creating realistic assets, especially when combined with methods like photogrammetry and 3D scanning, where intricate surface details can be captured and baked into textures for use in virtual environments.
Texture Mapping: Texture mapping is a technique used in computer graphics to apply an image or texture to a 3D surface, enhancing the visual detail and realism of the rendered object. This process involves wrapping a 2D image around a 3D model, which allows for the simulation of complex surface details without increasing the geometric complexity of the model itself. This technique connects closely with various aspects of rendering, including geometry, spatial mapping, and asset creation.
Topographic mapping: Topographic mapping is the process of creating detailed and accurate representations of the Earth's surface, showing its elevation changes and landforms. This mapping technique captures both natural and man-made features, providing essential information for various applications such as planning, engineering, and environmental studies. By using contour lines, symbols, and colors, topographic maps help visualize the terrain's shape and structure.
UV Mapping: UV mapping is the process of projecting a 2D image texture onto a 3D model's surface. It involves creating a coordinate system that maps each point on the 3D model to a corresponding point on the texture, allowing for accurate placement and alignment of images on the model. This technique is crucial for creating realistic visual appearances, as it defines how textures wrap around objects, impacting both aesthetic quality and material properties.