lens distortion

[ganzou]

Hey guys, we got a clip of an aerial shot of Las Vegas which I am using for a class project. My auto track with a bit of cleanup gives me a pretty good solution, however near the end there are some distant points that slowly move out out of position, this does not affect my foreground objects (hotels), but these distant points in a way represent my ground plane and I want to get them right. I try using one or two manual tracks on that part of the image that is not solving correctly however that worsens my solution.

Upon inspection of my point cloud I noticed that a certain hotel had an odd horizontal curvature...something I know for a fact it does not have so immediately I remembered about lens distortion. I started searching for a good reference frame to lay down a couple of lines to cancel the distortion, however all streets are hidden by the buildings and just small patches of "straight lines" can be seen on every frame. I'm using PFT by the way. I tried laying down a couple of lines...basically what I could get from the buildings, but it doesn't seem to be getting me an adequate undistorted solution. Will laying down lots of small lines help? Is there another approach to this?

I'm really stuck on this and have been trying to get something acceptable these past few days...so any help would really be appreciated.

[milkey]

Please post an eloquent frame of you shot so we can comment on it.

[ganzou]

sorry..my bad.
The points in the yellow area are the ones trailing off near the end of the sequence. By the way...it is a 30 sec sequence. Maybe solving by sections?

[milkey]

These lines should be the most relevant for lens distortion correction. But upon my inspection, your shot dose not suffer from severe distortion.

The curvature of the point cloud is a result of the huge tracker numbers in your shot. I bet you have more than 3000 throughout the shot. Use the error threshold option to decimate the erroneous trackers. Also keep in mind that the yellow area trackers are far away representing minute features thus unreliable. A deviation of more than 1.2-1.5 RMS is not acceptable for that area.

To keep a good track add manual trackers that are active throughout the hole shot.

[ganzou]

Thanks a lot milkey....my solve is running pretty smoothly including the features on the background.

From the beginning I was constantly modifying the threshold to get rid of bad auto tracks. I mean the solution was pretty good except for those far away points which I think I needed to get right because I wanted to place some elements on the far background.

Anyway, I went back and added 6 manual tracks divided through foreground, mid and back on pretty solid areas thinking that would help out the solve. I was still getting one or two large spikes that were raising my average rms at around 1.7; My average RMS when just using the auto tracks was at about .9 The manual tracks put a bump or two on the solve. I ignored the manuals and went back to solve just auto to see if all was still good and it was except for those tracks on the background...so I took a good look at the spikes I was getting, the points causing those spikes and the points I wanted to keep steady from the already fair solve result. And ended up just setting up three manual tracks on specific areas and that did the trick. As a side question: if in the perspective window I see points that are just floating dead in space and not stuck to anything on the sequence...will deleting them and improving the solution really improve the solution or are they still needed for the solution even if they don't end up on any object?

I already tried some test primitives and they are sticking really good...even tried some test objects on the background points and they are holding pretty well. I guess 1040 frames sequence was a bit too long. Anyway...I bumped into the next problem and hope you can give me hand with it.

I set up my ground plane by setting an obvious street level point as the origin and then grabbed some more ground level points and oriented the scene according to them. I turned on the horizon and that gave me the red horizon line....I then just wanted to see if I could optimize by setting up some x,y,z lines and orient using those instead and that gave me the blue horizon line.

The blue line looks like the horizon eye view of the helicopter, but the red line looks like where the ground plane fades into the distance...sooo...which one is the right one? I might be confusing geometric horizon and true horizon. Sorry, I'm still a match moving learner trying to improve the skills

[RobPhoboS]

Placing my bet on 'red'

[ganzou]

boy did I look like an idiot lol.... still...the wrong horizon is giving me better results...weird.

[RobPhoboS]

I'm not saying I'm correct, I'm just guessing that it could be the right one (only 50/50 chance - vegas style ).

Can you post up a render preview?
Not with any typical test objects but with geometry attached to the face of a building or something ?

[ganzou]

I have tried geometry, and it pretty much sticks, from the viewpoint of the camera of course. So my solve is pretty decent in the sense that all my points are lining up and am not getting any major sliding. However, the point cloud shows a flaw that I have not been able to get rid of. It is a hotel that is supposed to line up to a street...however comparing the sat view and point cloud...both tell different stories. I'm new into match moving...so should point clouds represent the actual scenery 100% or all that matters is having the points stick from the view of the camera?...please share any ideas you guys might have on getting an accurate point cloud.

[milkey]

This is one simple way of establishing an approximate horizon line. This would be the true horizon line.

But please keep in mind that there are some elevation differences in the landscape. The city is not flat you know. So by pointing planar constrains to various trackers around the floor would not point out a good horizon line either. The best practice is to establish the true horizon line and match it as good as possible but also keeping the planar constraints.

If your point of interest is not placed far from the buildings, and you are trying to blow up the building for example there is no point on aligning everything to the horizon.

[milkey]

And about the point cloud.... Your focal length is not as wide as the real camera. Size constrains between points in the landscape (Measured with google earth) will provide the correct fov if you don't know it. For long shots, PF track has this tendency of constantly adjusting focal length.

[ganzou]

Hey guys, thanks a lot for taking the time to look at my posts and giving your feedback on it. I really appreciate it.

If your point of interest is not placed far from the buildings, and you are trying to blow up the building for example there is no point on aligning everything to the horizon.

Well as you can see in my last post I managed to get a good horizon I did some rotation and planar constraint, however...for this little project I wanted to basically just comp in some fire elements, debris, rubble, cam proj, etc and show a Vegas which has been badly damaged, in the background where there are not many buildings I wanted to add a crater or something that helped explain why Vegas was damaged. Thats the reason for me wanting to get a good horizon.

As for the field of view, since the beginning I thought it was something with the lens, that's why I was going for a lens distortion problem. I'm new to pftrack and match moving in general...don't really know how to set up size constraints but I will look into it and update with what I get. Thanks for the tip btw....it does make sense...gonna get some measurements outta google earth and set up those constraints..thanks.

Can you post up a render preview?
Not with any typical test objects but with geometry attached to the face of a building or something ?

Since the point could on my geo is pretty messed up, I'm going to fix that first, after I get that sorted out I'll gladly set up some geo and do test renders to show ya. I'll also move this little project to the WIP so you can crit on the stuff I come up with.

Thanks again guys, I'll keep working on this and hope to get it sorted out soon, will post more updates soon.

[ganzou]

As a side note.... I started out with matchmover and then moved to pftrack when it was no longer developed. Should I move to a different match moving app for this type of shot?

[ecto]

I concur that the problem is probably the focal length...usually when you see the features distorted from what you know is right, this is the case). I've found the PFTrack struggles quite a bit with focal length and often it is necessary to help it a bit by specifying at least a suggestion for the focal length. YOu can do this one of two ways: putting a value in as approximate (but in this case i would assume you don't know even an approximate value), or a better option may be to use the Estimate Focal Length Tool. With this tool, you can line up a box to a square object, like one of those buildings in the front, and then resolve. I've found this helps a lot. Another thing to try would be to add more manual markers throughout the environment as though you were doing a manual track and solve together with the auto tracks.

[ganzou]

BWAHAHAHA.....I finally got the track working!

Milkey:
Yup, pftrack definitely does a terrible job when it comes to estimating the focal length on these long shots. I finally got the whole thing working. My solution was pretty close and getting the exact focal length was all that was needed. Thanks for your help dude!

Ecto: Thanks for the tip, I never used the focal length tool before, but that actually made the trick, it was all pftrack needed to finally get right those small details that were left.

Now that I'm done with this I can move on to layout and then its off to nuke! Again thanks for all the advice guys...wish I could do match moves more often.