Turning a VIDEO into 3D using LUMA AI and BLENDER!

Bad Decisions Studio
17 Apr 202303:18

TLDRIn this video, the creator explores the innovative Luma AI technology, which enables 3D modeling from video footage. Despite time constraints and challenging conditions, the AI successfully converts various scenes, including a payphone and a car, into detailed 3D models. The results, particularly from the DSLR footage, are impressive, showcasing the potential of this technology for future applications in 3D rendering and animation.

Takeaways

  • 🌐 Luma AI enables 3D modeling from video footage, not just photos.
  • ⏱️ The process needs to be completed quickly before dark, highlighting a time-sensitive challenge.
  • 🎥 The video demonstrates the use of Luma AI on different objects, including a payphone and a car.
  • 📸 The quality of the 3D models depends on the camera used, with DSLR footage providing sharper results.
  • 🖼️ The AI separates the scene from the object automatically, showing impressive accuracy.
  • 🚀 The technology is in its early stages, suggesting potential for significant future improvements.
  • 📹 The video footage used for the 3D model took only 1 minute and 42 seconds to record.
  • 🛠️ Post-processing, such as cleaning up reflective surfaces and adjusting for darkness, is necessary for better model quality.
  • 🔄 The script mentions plans to use the created 3D assets in a short video to test their performance in 3D software.
  • 📦 Luma AI provides a glTF model and an Unreal Engine plugin for further use and development.
  • 🔧 The process of creating 3D models from video is simplified, allowing for quick and easy integration into various applications.

Q & A

  • What is the main topic of the video?

    -The main topic of the video is the demonstration of Luma AI's ability to convert video footage into 3D models using photogrammetry.

  • What is the significance of the Luma AI technology?

    -Luma AI technology allows users to create 3D models from video footage, which is faster and more convenient than traditional photo-based 3D capture methods.

  • How long did it take to process the 3D mesh in the video?

    -Each 3D mesh took about 20 to 30 minutes to be processed.

  • What was the issue with the DSLR footage initially?

    -The initial issue with the DSLR footage was that it was failing to upload. The problem was resolved by running the footage through DaVinci with h.265 encoding and re-uploading them.

  • What did the AI do with the video scene?

    -The AI automatically separated the scene from the object, creating a detailed 3D model.

  • What software was used to refine the 3D model?

    -Blender was used to refine the 3D model by smoothing out sharp edges.

  • How long did it take to record the video footage that was used to create the 3D model?

    -It took only one minute and 42 seconds to record the video footage.

  • What was the quality difference between the iPhone and DSLR results?

    -The DSLR had sharper quality due to better image resolution and closer proximity to the subject, while the iPhone footage followed the website's instructions for three levels of loop from high, mid, and low angles.

  • What was the challenge with the car footage?

    -The car had a long and reflective paint job, and the footage was taken in low light conditions, which made it difficult to capture a clear 3D model.

  • What is the plan for the 3D models created in the video?

    -The plan is to use these 3D models to create a quick and short video to demonstrate how these assets perform when used in 3D software for background purposes.

Outlines

00:00

🎥 Introducing Luma AI Video to 3D Photogrammetry

The script describes the excitement of discovering Luma AI's ability to convert video into 3D models. The narrator rushes to capture a video before sunset and discusses the process of uploading the footage to Luma AI's website. The AI's capability to separate the scene from the object is highlighted, and the user's experience with the software is shared, including the time it took to process the 3D mesh and the quality of the resulting models. The script also mentions the potential for future improvements in the technology.

Mindmap

Keywords

💡3D model

A three-dimensional model is a representation of any object in a three-dimensional space using a mesh to enclose a shape. In the video, the 3D model is created using Luma AI from a video feed, which is a significant advancement in capturing real-world objects in digital form. The video demonstrates the process of turning a real-life object, like a payphone, into a 3D model for various applications.

💡Luma AI

Luma AI refers to an artificial intelligence tool that enables the conversion of video footage into 3D models through a process called photogrammetry. This technology is highlighted in the video as a time-saving and efficient method for creating detailed 3D representations of objects from video clips. The narrator discusses the ease of using Luma AI to generate 3D models from their video footage.

💡Photogrammetry

Photogrammetry is the science of making measurements from photographs, especially for recovering the exact positions of surface points. In the context of the video, it's the process that Luma AI uses to create 3D models from video. The video showcases the capabilities of photogrammetry by capturing various objects, such as a payphone and a car, and turning them into 3D models.

💡Video to photogrammetry

This term refers to the process of converting video content into a series of photos that can then be used to create 3D models. The video script describes this process as a new feature of Luma AI, which allows for the capture of objects in 3D space using video instead of individual photos. This method is demonstrated with a time constraint, emphasizing the practicality and speed of the process.

💡DaVinci

DaVinci is a professional video editing software used in the video for encoding footage in h.265 format, which is necessary for the Luma AI to process the 3D models. The script mentions using DaVinci to re-encode DSLR footage that initially failed to upload to Luma AI's website, showcasing the technical steps involved in the 3D modeling process.

💡GLTF model

GLTF, or GL Transmission Format, is an open standard file format for 3D models and scenes. In the video, the narrator downloads a GLTF model from Luma AI, which is a format that can be used in various 3D applications and game engines. The GLTF model is an output of the photogrammetry process and is used to demonstrate the versatility of the 3D models created by Luma AI.

💡Unreal Engine

Unreal Engine is a powerful game engine used for creating video games and other interactive experiences. The video mentions an Unreal Engine plugin, which suggests that the 3D models generated by Luma AI can be integrated into game development or virtual environments created with Unreal Engine, expanding the potential applications of the technology.

💡Blender

Blender is an open-source 3D creation suite that supports the entire 3D pipeline, including modeling, rigging, animation, simulation, rendering, compositing, and motion tracking. In the video, Blender is used to refine the 3D models by smoothing out sharp edges, which is an important step in preparing the models for use in various digital environments.

💡Payphone

A payphone is a public telephone designed to be used with coins, phone cards, or credit cards. In the video, the payphone serves as one of the objects being converted into a 3D model using Luma AI. The process of capturing the payphone in 3D showcases the practical applications of the technology for creating detailed digital representations of everyday objects.

💡Reflective surfaces

Reflective surfaces are materials that reflect light, making them challenging to photograph or capture in 3D due to the way they distort images. The video discusses the difficulties in capturing a car with reflective paint, illustrating the limitations and challenges of photogrammetry when dealing with reflective objects.

💡Background purposes

In the context of 3D modeling and animation, using assets for background purposes means incorporating them into scenes to create a realistic setting without them being the primary focus. The video script mentions using the 3D models for background purposes in 3D software, indicating how these models can enhance the realism of digital scenes without requiring the same level of detail as foreground objects.

Highlights

Luma AI enabled video to photogrammetry allows 3D capture from videos instead of photos.

The process needs to be completed before dark, indicating a time constraint.

The Luma AI technology was tested with a video of a payphone from different angles.

The video was captured quickly, in just a couple of minutes.

Luma AI's website was used to upload the video clips for processing.

DSLR footages initially failed, so they were re-uploaded after encoding with h.265.

Each 3D mesh took about 20 to 30 minutes to be processed.

The AI automatically separates the scene from the object, resulting in a detailed 3D model.

A glTF model and an Unreal Engine plugin are available for the 3D outputs.

The 3D model was refined in Blender to remove sharp edges.

The video footage was compared with the 3D model output, showing the potential of the technology.

The iPhone and Sony DSLR results were compared, with the DSLR having sharper quality.

The car model, despite challenges like reflective paint and darkness, yielded impressive results.

The assets created will be used to create a short video to test their performance in 3D software.

The technology is expected to improve, offering better quality in the future.

Elevate Your Presentations with Live AI Avatars

Create AI avatars with voiceovers to engage your audience.

Elevate Your Presentations with Live AI AvatarsTry It Now