I must say right away that I am not an expert in this field, rather I want to get advice from knowledgeable users. My attempts to study this issue are still weak. Dear Keet, he correctly noted that the method of replacing 3D models with images on boxes has an important drawback: you can't "walk around" the room, the box only works from one point of view. But I tried using neural networks. Here's what I got and here are my thoughts. The main difficulty in working with artificial intelligence is that it is very difficult to get the desired result. There are probably specialists who can do everything, but I mostly got surprises. AI does everything with unexpected options, even if the promt is written in great detail and clearly (as it seems to a living person). For now, I've settled on the Chinese startup HAILUO AI. It has several very important advantages over its competitors. First of all, incredible generosity. A beginner is given a very large margin for training renders, it was enough for me to work very hard with video generation for two days. There is no such thing anywhere, everywhere developers are very greedy. During this free limit, I was able to figure out the possibilities and make almost 30 interesting videos for myself. That's a lot!! Now I can tell you my conclusions.
The second important advantage is that there is a generation option with the ability to control camera movement. This is very important because it allows you to do something consciously, at least partially. Rather than hoping for a miracle from a neural network without guarantees. This generation method is called 12V-01-Director. Camera movements can be of several types. To work with interior photography, you can take one view or combine several views. For example, moving to the side, turning your head, lifting the camera up, and other combinations. You can make more complex combinations, but the clip time is always limited to 6 seconds. My first experience stunned me. Artificial intelligence was able to recognize all the individual objects in the photo, distribute them in virtual space and even looked behind your back, sometimes very correctly!! Unbelievable. Here's an example of this render.
A 6-second clip with a camera consisting of four types of movement (left shift, right turn, upward tilt) looks like this. Pay attention to the chair!! And the mirror on the ceiling!!! There are reflections that are almost right!! And it's originally just a photo, it's not a group of 3D models. Clip 1 is simple
to set up the camera, it is important not to go much beyond the original frame during these 6 seconds, otherwise the intelligence will come up with something unexpected, most likely unnecessary and incorrect. Therefore, I advise you to combine the opposite movements - a shift to the left, then a turn of the head to the right, so that the center of the photo remains in place.
I'll continue in the next post.
