Rectify function coordinate change?

[zalo]

Hey so I've been working with processing images... I've figured out that both the left and right camera's images are now packed into a single array (vertically, so 2x640x240 = 1x640x480); the issue arises when I try to query the Rectify function to go back from image space to world space.

What are the image coordinates that I should feed into Rectify, and how do I tell it which camera's distortion I'm trying to correct against?

[Joe_Ward]

Both the image data and the distortion data are stacked in single buffers now. Unfortunately, that was a very late change before we released and the Rectify() and Warp() functions have not been updated. They will only provide correct results for the left image. On the plus side, the source code is now available in the C# so you might be able to fix it on your end before the next release. If it looks non-trivial, let me know and I'll see if I can post a fix here.

[zalo]

Hm, the fix didn't make it into the 4.0.1 Unity assets so I took a look.

Fixing the Warp function to sample from a stacked Distortion array was pretty easy, just have to add a "leftimage" bool to the Warp/Rectify arguments (if true, is sampling left image, if false, sampling right image) and adding:

			y1 += leftimage?0:64;

after y1's initial definition.

However, there's an issue I can't fix. And that's that the RayOffset and RayScale variables that the API is outputting now are total garbage numbers.

In 2.3.1, the RayOffset was always 0.5 and the RayScale was .125 (for both axes), so I tried plugging those numbers in, and they're at least in the right order of magnitude now.... But they're still kind of messed up; it seems like the Rayscale on the Y axis is now almost double the Rayscale on the X axis.

If you happen to have the correct Ray values for the new distortion units on hand, that would hopefully at least let me get some stable tracking going for now...

[Joe_Ward]

RayScale should always be .125 and RayOffset should always be .5. The only reason this would change is if the FOV of the cameras changed.

[zalo]

Looks like we'll need to screw down those camera lenses tighter so the FoV doesn't keep changing like this

Jokes aside, plugging those old values in gets me 90% of the way there (I now get rays that converge on a point and look plausible), but the tracking is warped and distorted in a way that it wasn't with 2.3.1; I'm pretty sure something else has changed...

[ashish090494]

Hi,
I am using Rectify() (in C++) to get the slope direction and using it to get the depth using the triangulation example as given in the documentation. However, on doing static tests, the depth is not coming out right. Can it be because of some issue with rectify() ??
Thanks!