## PI3D - Rotating around local axis

cowminer27
Posts: 10
Joined: Thu May 12, 2016 3:04 pm

### PI3D - Rotating around local axis

I am currently creating a game useing a IMU to control camera rotation. i am trying to have it ratate on its x,y and z axis but the camera seems to be rotating using the world axis.

Is there a way of rotating a camera around it's local axis?
Currently Working On:
- A DIY HTC Vive
- Extra stuff for my AIY project

Posts: 2616
Joined: Sat Jan 28, 2012 11:57 am
Location: UK

### Re: PI3D - Rotating around local axis

Hi @cowminer27, I imagine that you are wanting to define a local coordinate system rotated Za,Xa,Ya (that's the order the rotations are applied) then rotate the camera relative to this new frame of reference by Zb,Xb,Yb? It's quite tricky! With the rotation of Shape objects it's possible to do this by making the Shape a child of another Shape (i.e. an 'empty', normally just a very small Triangle) However this functionality isn't in the Camera and it may be quite tricky to do.

I will have a look at what might be required but it will involve something like the process done within Shape.draw() see https://github.com/tipam/pi3d/blob/mast ... pe.py#L178 where the critical bit would be something like

Code: Select all

``        camera_matrix = numpy.dot(camera_matrix, camera_frame_of_ref_matrix)``
One of the issue is that the Camera matrix multiplication must only be done once per frame then used for drawing all the Shapes so the components of the transformation (Rx,Ry,Rz,Sx,Sy... etc) are not held individually, just the resultant matrix which is modified in situ after the Camera method calls. Anyway, if you confirm this is the kind of thing you're looking to do I will do some rummaging and experiments.

EDIT PS
A bit harder now that I think about it as I it would almost certainly need some hacking inside the Camera class. The order of applying the matrix multiplication is pretty crucial i.e. something like
dot(Txyz1, dot(R2z, dot(R2y, dot(R1z, R1y)))) At the moment the translation is done as part of the normal Camera matrix but this would have to be split out to allow the rotations to be applied together. In the analogy of having an 'empty' with Camera transformation relative to that, the existing Camera rotations would be relative to the empty but the existing Camera translations would have to be used as 'empty' translations. Does that make sense?

Posts: 2616
Joined: Sat Jan 28, 2012 11:57 am
Location: UK

### Re: PI3D - Rotating around local axis

@cowminer27, I'm not sure this is doing what you want, it's rather disorientating to use but you might be able to experiment further and come up with something that works.

Code: Select all

``````# in pi3d/Camera.py
...

def relocate(self, rot=None, tilt=None, point=np.array([0.0, 0.0, 0.0]),
distance=np.array([0.0, 0.0, 0.0]), normal=None,
slope_factor=0.5, crab=False,
frame_rot=None, frame_tilt=None, frame_roll=None): # <<<<<<<<<<<<<<<<
...
self.reset()
if frame_roll is not None: #<<<<<<<<<
self.rotateZ(frame_roll)
if frame_tilt is not None:
self.rotateX(frame_tilt)
if frame_rot is not None:
self.rotateY(frame_rot) #>>>>>>>>>
if tilt is not None:
self.rotateX(tilt)
if rot is not None:
self.rotateY(rot)

################
in pi3d_demos/ForestWalk.py
...
CAMERA = pi3d.Camera()
frame_rot, frame_tilt, frame_roll = 0.0, 0.0, 0.0
# Display scene and rotate cuboid
while DISPLAY.loop_running():
xm, ym, zm = CAMERA.relocate(rot, tilt, point=[xm, ym, zm], distance=step,
normal=norm, crab=crab, slope_factor=1.5,
frame_rot=frame_rot, frame_tilt=frame_tilt, frame_roll=frame_roll) # <<<<<<<<<<<<
...
elif k == 112:  #key p picture
pi3d.screenshot("forestWalk" + str(scshots) + ".jpg")
scshots += 1
elif k == ord('z'):   #key z, inc cam frame roll #<<<<<<<<<<<<
frame_roll += 1.0
elif k == ord('y'):  #key y, inc cam frame yaw
frame_rot += 1.0
elif k == ord('x'):  #key x, inc cam frame tilt
frame_tilt += 1.0                              #>>>>>>>>>>>>
elif k == 10:   #key RETURN
mc = 0
...

``````
If you are using the StereoCam class then there is the extra compliction that the lateral movment between the two images needs to be along the x axis in the rotated frame of reference so this probably can't be done in the start_capture() method with the move_camera() doing the gross Camera movement before hand. i.e. start_capture() might have to regenerate the Camera matrix twice including the x offset part way through (or maybe as the initial process)

EDIT PS.

Thinking about this again, what you probably want is for the Camera rotated frame of reference to inherit the modified position each frame. To do this you might have to keep a copy of the revised rotations, something like this:

Code: Select all

``````# do once per frame the equivalent of
forMatrix = dot(forMatrix, dot(Ry, dot(Rx, Rz))) # Rxyz are rotation matrices relative to rotated frame of reference
# then for each view position in start_capture()
camera_3d.reset()
camera_3d.position([offs, 0.0, 0.0])
camera_3d.mtrx = np.dot(forMatrix, camera_3d.mtrx)
camera_3d.position([Tx, Ty, Tz]) # cam coordinates relative to origin
``````

Posts: 2616
Joined: Sat Jan 28, 2012 11:57 am
Location: UK

### Re: PI3D - Rotating around local axis

Hmm, this seems to work as far as I can tell doing some cross-eyed tests!
pi3d/util/StereoCam.py

Code: Select all

``````import ctypes
import numpy as np
import math

from pi3d.constants import *
from pi3d.Camera import Camera
from pi3d.shape.Sprite import Sprite
from pi3d.util.OffScreenTexture import OffScreenTexture
from pi3d.Display import Display
from pi3d import opengles

class StereoCam(object):
"""For creating an apparatus with two sprites to hold left and right
eye views.

This Class is used to hold the 3D Camera which should be used to draw
the 3D objects. It also holds a 2D Camera for drawing the Sprites"""
def __init__(self, shader="uv_flat", mipmap=False, separation=0.4, interlace=0):
""" calls Texture.__init__ but doesn't need to set file name as
texture generated from the framebuffer. Keyword Arguments:

to use when drawing sprite, defaults to post_base, a simple
3x3 convolution that does basic edge detection. Can be copied to
project directory and modified as required.

*mipmap*
can be set to True with slight cost to speed, or use fxaa shader

*separation*
distance between the two camera positions - how wide apart the
eye views are.

*interlace*
if interlace > 0 then the images are not taken with glScissor and
must be drawn with a special interlacing shader.
"""
if interlace <= 0:
else:
precision mediump float;
attribute vec3 vertex;
attribute vec2 texcoord;
uniform mat4 modelviewmatrix;
varying vec2 texcoordout;
void main(void) {
texcoordout = texcoord;
gl_Position = modelviewmatrix * vec4(vertex,1.0);
}
precision mediump float;
uniform sampler2D tex0;
uniform sampler2D tex1;
varying vec2 texcoordout;
void main(void) {{
vec4 texc0 = texture2D(tex0, texcoordout);
vec4 texc1 = texture2D(tex1, texcoordout);
vec2 coord = vec2(gl_FragCoord);
gl_FragColor = mix(texc0, texc1, step(0.5, fract(coord.x / {:f})));
}}
""".format(interlace * 2.0))
self.camera_3d = Camera()
self.forMtrx = np.identity(4, dtype='float32') # initially not rotated
self.position = [0.0, 0.0, 0.0]
self.camera_2d = Camera(is_3d=False)
self.offs = separation / 2.0
self.interlace = interlace
self.textures = []
self.sprites = []
self.tex_list = []
for i in range(2):
self.textures.append(OffScreenTexture(name="stereo"))
ix, iy = self.textures[i].ix, self.textures[i].iy
#two sprites full width but moved so that they are centred on the
#left and right edges. The offset values then move the uv mapping
#so the image is on the right of the left sprite and left of the
#right sprite
self.sprites.append(Sprite(z=20.0, w=ix, h=iy, flip=True))
if interlace <= 0:
self.sprites[i].positionX(-ix/2.0 + i*ix)
self.sprites[i].set_offset((i * 0.5 - 0.25, 0.0))
else:
self.sprites[i].set_2d_size(w=ix, h=iy)
self.textures[i].blend = True
self.textures[i].mipmap = mipmap
self.tex_list.append(self.textures[i])

def move_camera(self, position, rot, tilt, roll=0.0):
sy, cy = math.sin(rot), math.cos(rot)
sx, cx = math.sin(tilt), math.cos(tilt)
sz, cz = math.sin(roll), math.cos(roll)
self.forMtrx = np.dot(self.forMtrx,
np.dot([[cy, 0, -sy, 0], # rotation
[0, 1, 0, 0],
[sy, 0, cy, 0],
[0, 0, 0, 1]],
np.dot([[1, 0, 0, 0], # after tilt
[0, cx, sx, 0],
[0, -sx, cx, 0],
[0, 0, 0, 1]],
[[cz, sz, 0, 0], # after roll
[-sz, cz, 0, 0],
[0, 0, 1, 0],
[0, 0, 0, 1]])))
self.position = position

def start_capture(self, side):
""" after calling this method all object.draw()s will rendered
to this texture and not appear on the display.

*side*
Either 0 or 1 to determine stereoscopic view
"""
self.camera_3d.reset()
offs = -self.offs if side == 0 else self.offs
self.camera_3d.position([offs, 0.0, 0.0])
self.camera_3d.mtrx = np.dot(self.forMtrx, self.camera_3d.mtrx)
self.camera_3d.position(self.position)
tex = self.textures[side]
tex._start()
if self.interlace <= 0:
xx = tex.ix / 4.0 # draw the middle only - half width
yy = 0
ww = tex.ix / 2.0
hh = tex.iy
opengles.glEnable(GL_SCISSOR_TEST)
opengles.glScissor(ctypes.c_int(int(xx)), ctypes.c_int(int(yy)),
ctypes.c_int(int(ww)), ctypes.c_int(int(hh)))

def end_capture(self, side):
""" stop capturing to texture and resume normal rendering to default
"""
self.textures[side]._end()
if self.interlace <= 0:
opengles.glDisable(GL_SCISSOR_TEST)

def draw(self):
""" draw the shape using the saved texture
"""
if self.interlace <= 0:
for i in range(2):
else:

``````
pi3d_demos/ForestStereo.py

Code: Select all

``````#!/usr/bin/python
from __future__ import absolute_import, division, print_function, unicode_literals

""" ForestWalk but with stereoscopic view - i.e. for google cardboard

NB in this example the cameras have been set with a negative separation i.e.
for viewing cross-eyed as most people find this easier without a viewer!!!
If a viewer is used then the line defining CAMERA would need to be changed
to an appropriate +ve separation.

NB also, no camera has been explicitly assigned to the objects so they all
use the default instance and this will be CAMERA.camera_3d so long as the
StereoCam instance was created before any other Camera instance.
"""

import math,random

import demo
import pi3d

# Setup display and initialise pi3d
DISPLAY = pi3d.Display.create(w=1200, h=600)
DISPLAY.set_background(0.4,0.8,0.8,1)      # r,g,b,alpha
# yellowish directional light blueish ambient light
pi3d.Light(lightpos=(1, -1, -3), lightcol=(1.0, 1.0, 0.8), lightamb=(0.25, 0.2, 0.3))
CAMERA = pi3d.StereoCam(separation=-0.5, interlace=0)
""" If CAMERA is set with interlace <= 0 (default) then CAMERA.draw() will produce
two images side by side (each viewed from `separation` apart) i.e. -ve
requires viewing slightly cross-eyed.

If interlace is set to a positive value then the two images are interlaced
in vertical stripes this number of pixels wide. The resultant image needs
to be viewed through a grid. See https://github.com/pi3d/pi3d_demos/make_grid.py
"""

tree2img = pi3d.Texture("textures/tree2.png")
tree1img = pi3d.Texture("textures/tree1.png")
hb2img = pi3d.Texture("textures/hornbeam2.png")
bumpimg = pi3d.Texture("textures/grasstile_n.jpg")
reflimg = pi3d.Texture("textures/stars.jpg")
rockimg = pi3d.Texture("textures/rock1.jpg")

FOG = ((0.3, 0.3, 0.4, 0.8), 650.0)
TFOG = ((0.2, 0.24, 0.22, 1.0), 150.0)

#myecube = pi3d.EnvironmentCube(900.0,"HALFCROSS")
myecube = pi3d.EnvironmentCube(size=900.0, maptype="FACES", name="cube")
myecube.set_draw_details(flatsh, ectex)

# Create elevation map
mapsize = 1000.0
mapheight = 60.0
mountimg1 = pi3d.Texture("textures/mountains3_512.jpg")
mymap = pi3d.ElevationMap("textures/mountainsHgt.png", name="map",
width=mapsize, depth=mapsize, height=mapheight,
divx=32, divy=32)
mymap.set_draw_details(shader, [mountimg1, bumpimg, reflimg], 128.0, 0.0)
mymap.set_fog(*FOG)

#Create tree models
treeplane = pi3d.Plane(w=4.0, h=5.0)

treemodel1 = pi3d.MergeShape(name="baretree")

treemodel2 = pi3d.MergeShape(name="bushytree")

#Scatter them on map using Merge shape's cluster function
mytrees1 = pi3d.MergeShape(name="trees1")
mytrees1.cluster(treemodel1.buf, mymap, 0.0, 0.0, 400.0, 400.0, 50, "", 8.0, 3.0)
mytrees1.set_draw_details(flatsh, [tree2img], 0.0, 0.0)
mytrees1.set_fog(*TFOG)

mytrees2 = pi3d.MergeShape(name="trees2")
mytrees2.cluster(treemodel2.buf, mymap, 0.0, 0.0, 400.0, 400.0, 80, "", 6.0, 3.0)
mytrees2.set_draw_details(flatsh, [tree1img], 0.0, 0.0)
mytrees2.set_fog(*TFOG)

mytrees3 = pi3d.MergeShape(name="trees3")
mytrees3.cluster(treemodel2, mymap,0.0, 0.0, 300.0, 300.0, 20, "", 4.0, 2.0)
mytrees3.set_draw_details(flatsh, [hb2img], 0.0, 0.0)
mytrees3.set_fog(*TFOG)

#Create monument
monument = pi3d.Model(file_string="models/pi3d.obj", name="monument")
monument.set_normal_shine(bumpimg, 16.0, reflimg, 0.4)
monument.set_fog(*FOG)
monument.translate(100.0, -mymap.calcHeight(100.0, 235) + 12.0, 235.0)
monument.scale(20.0, 20.0, 20.0)
monument.rotateToY(65)

#screenshot number
scshots = 1

#avatar camera
rot = 0.0
tilt = 0.0
roll = 0.0
avhgt = 3.5
xm = 0.0
zm = 0.0
ym = mymap.calcHeight(xm, zm) + avhgt

# Fetch key presses
mykeys = pi3d.Keyboard()
mymouse = pi3d.Mouse(restrict = False)
mymouse.start()

omx, omy = mymouse.position()

# Display scene and rotate cuboid
while DISPLAY.loop_running():
CAMERA.move_camera((xm, ym, zm), rot, tilt, roll)
roll = 0.0
myecube.position(xm, ym, zm)
for i in range(2):
CAMERA.start_capture(i)
monument.draw()
mymap.draw()
if abs(xm) > 300:
mymap.position(math.copysign(1000,xm), 0.0, 0.0)
mymap.draw()
if abs(zm) > 300:
mymap.position(0.0, 0.0, math.copysign(1000,zm))
mymap.draw()
if abs(xm) > 300:
mymap.position(math.copysign(1000,xm), 0.0, math.copysign(1000,zm))
mymap.draw()
mymap.position(0.0, 0.0, 0.0)
myecube.draw()
mytrees3.draw()
mytrees2.draw()
mytrees1.draw()
CAMERA.end_capture(i)
CAMERA.draw()

mx, my = mymouse.position()
buttons = mymouse.button_status()

rot = -(mx-omx)*0.02
tilt = (my-omy)*0.02
omx=mx
omy=my

#Press ESCAPE to terminate
if k >-1 or buttons > mymouse.BUTTON_UP:
if k == 119 or buttons == mymouse.LEFT_BUTTON:  #key W
xm += CAMERA.camera_3d.mtrx[0, 3]
zm += CAMERA.camera_3d.mtrx[2, 3]
ym = mymap.calcHeight(xm, zm) + avhgt
elif k == 115 or buttons == mymouse.RIGHT_BUTTON:  #kry S
xm -= CAMERA.camera_3d.mtrx[0, 3]
zm -= CAMERA.camera_3d.mtrx[2, 3]
ym = mymap.calcHeight(xm, zm) + avhgt
elif k == ord('a'):
elif k == ord('d'):
roll -= 0.02
elif k == 112:  #key P
pi3d.screenshot("forestWalk"+str(scshots)+".jpg")
scshots += 1
elif k == 10:   #key RETURN
mc = 0
elif k == 27:  #Escape key
mykeys.close()
mymouse.stop()
DISPLAY.stop()
break

halfsize = mapsize / 2.0
xm = (xm + halfsize) % mapsize - halfsize # wrap location to stay on map -500 to +500
zm = (zm + halfsize) % mapsize - halfsize
``````
Where essentially I've just added a roll (into screen rotation) using the 'a' and 'd' keys and made rot, tilt and roll incremental values from one frame to the next. I've put a video here https://youtu.be/1guKBfqYMC0

Posts: 2616
Joined: Sat Jan 28, 2012 11:57 am
Location: UK

### Re: PI3D - Rotating around local axis

I've pushed up some modifications to github pi3d develop branch to make this functionality available without hacking the module. There are also two Camera methods for helping to rebase gyro dead reckoning orientations to magnetometer vectors. See revised demo here.

chris-bunting
Posts: 7
Joined: Wed Jul 03, 2019 7:43 pm

### Re: PI3D - Rotating around local axis

Try setting the camera's 'absolute' argument to True, then you can give it the angle in x, y, and z with each iteration, because these then apply to local camera coordinates, and not world coordinates.

The code would look something like this:

Code: Select all

``````camera = pi3d.Camera(absolute=True)

while display.loop_running():
camera.rotateY(angley)
camera.rotateX(anglex)``````

cowminer27
Posts: 10
Joined: Thu May 12, 2016 3:04 pm

### Re: PI3D - Rotating around local axis

Thanks, I've not messed with this project for a while (I realise now that I never replied to paddy). I suspect that parameter didn't exist back then, but if it did, I missed something super obvious, my bad. Either way, I believe I ended up solving it using Paddy's solution above, but I lost the code a while ago so I can't be certain.

Thanks anyways!
Currently Working On:
- A DIY HTC Vive
- Extra stuff for my AIY project