VSE Effects: Wipe box: fix compilation errors

This is still broken I cant tell if it is the fact that the in_band
funtion does not work properally or an issue in the box algorithm, or
both.

It seems like the calculation of the size of the box while roatated
needs to be fixed also.