cowminer27
Posts: 10
Joined: Thu May 12, 2016 3:04 pm

PI3D - Rotating around local axis

Fri Jul 01, 2016 11:31 am

I am currently creating a game useing a IMU to control camera rotation. i am trying to have it ratate on its x,y and z axis but the camera seems to be rotating using the world axis.

Is there a way of rotating a camera around it's local axis?
Currently Working On:
- A DIY HTC Vive
- Extra stuff for my AIY project

User avatar
paddyg
Posts: 2616
Joined: Sat Jan 28, 2012 11:57 am
Location: UK

Re: PI3D - Rotating around local axis

Fri Jul 01, 2016 5:07 pm

Hi @cowminer27, I imagine that you are wanting to define a local coordinate system rotated Za,Xa,Ya (that's the order the rotations are applied) then rotate the camera relative to this new frame of reference by Zb,Xb,Yb? It's quite tricky! With the rotation of Shape objects it's possible to do this by making the Shape a child of another Shape (i.e. an 'empty', normally just a very small Triangle) However this functionality isn't in the Camera and it may be quite tricky to do.

I will have a look at what might be required but it will involve something like the process done within Shape.draw() see https://github.com/tipam/pi3d/blob/mast ... pe.py#L178 where the critical bit would be something like

Code: Select all

        camera_matrix = numpy.dot(camera_matrix, camera_frame_of_ref_matrix)
One of the issue is that the Camera matrix multiplication must only be done once per frame then used for drawing all the Shapes so the components of the transformation (Rx,Ry,Rz,Sx,Sy... etc) are not held individually, just the resultant matrix which is modified in situ after the Camera method calls. Anyway, if you confirm this is the kind of thing you're looking to do I will do some rummaging and experiments.

Paddy

EDIT PS
A bit harder now that I think about it as I it would almost certainly need some hacking inside the Camera class. The order of applying the matrix multiplication is pretty crucial i.e. something like
dot(Txyz1, dot(R2z, dot(R2y, dot(R1z, R1y)))) At the moment the translation is done as part of the normal Camera matrix but this would have to be split out to allow the rotations to be applied together. In the analogy of having an 'empty' with Camera transformation relative to that, the existing Camera rotations would be relative to the empty but the existing Camera translations would have to be used as 'empty' translations. Does that make sense?
also https://groups.google.com/forum/?hl=en-GB&fromgroups=#!forum/pi3d

User avatar
paddyg
Posts: 2616
Joined: Sat Jan 28, 2012 11:57 am
Location: UK

Re: PI3D - Rotating around local axis

Fri Jul 01, 2016 10:51 pm

@cowminer27, I'm not sure this is doing what you want, it's rather disorientating to use but you might be able to experiment further and come up with something that works.

Code: Select all

# in pi3d/Camera.py
...

  def relocate(self, rot=None, tilt=None, point=np.array([0.0, 0.0, 0.0]),
                distance=np.array([0.0, 0.0, 0.0]), normal=None,
                slope_factor=0.5, crab=False, 
                frame_rot=None, frame_tilt=None, frame_roll=None): # <<<<<<<<<<<<<<<<
...
    self.reset()
    if frame_roll is not None: #<<<<<<<<<
      self.rotateZ(frame_roll)
    if frame_tilt is not None:
      self.rotateX(frame_tilt)
    if frame_rot is not None:
      self.rotateY(frame_rot) #>>>>>>>>>
    if tilt is not None:
      self.rotateX(tilt)
    if rot is not None:
      self.rotateY(rot)


################
in pi3d_demos/ForestWalk.py
...
CAMERA = pi3d.Camera()
frame_rot, frame_tilt, frame_roll = 0.0, 0.0, 0.0
# Display scene and rotate cuboid
while DISPLAY.loop_running():
  xm, ym, zm = CAMERA.relocate(rot, tilt, point=[xm, ym, zm], distance=step, 
                              normal=norm, crab=crab, slope_factor=1.5,
                              frame_rot=frame_rot, frame_tilt=frame_tilt, frame_roll=frame_roll) # <<<<<<<<<<<<
...
    elif k == 112:  #key p picture
      pi3d.screenshot("forestWalk" + str(scshots) + ".jpg")
      scshots += 1
    elif k == ord('z'):   #key z, inc cam frame roll #<<<<<<<<<<<<
      frame_roll += 1.0
    elif k == ord('y'):  #key y, inc cam frame yaw
      frame_rot += 1.0
    elif k == ord('x'):  #key x, inc cam frame tilt
      frame_tilt += 1.0                              #>>>>>>>>>>>>
    elif k == 10:   #key RETURN
      mc = 0
...

If you are using the StereoCam class then there is the extra compliction that the lateral movment between the two images needs to be along the x axis in the rotated frame of reference so this probably can't be done in the start_capture() method with the move_camera() doing the gross Camera movement before hand. i.e. start_capture() might have to regenerate the Camera matrix twice including the x offset part way through (or maybe as the initial process)

Let me know your thoughts.

Paddy

EDIT PS.

Thinking about this again, what you probably want is for the Camera rotated frame of reference to inherit the modified position each frame. To do this you might have to keep a copy of the revised rotations, something like this:

Code: Select all

# do once per frame the equivalent of
forMatrix = dot(forMatrix, dot(Ry, dot(Rx, Rz))) # Rxyz are rotation matrices relative to rotated frame of reference
# then for each view position in start_capture()
camera_3d.reset()
camera_3d.position([offs, 0.0, 0.0])
camera_3d.mtrx = np.dot(forMatrix, camera_3d.mtrx)
camera_3d.position([Tx, Ty, Tz]) # cam coordinates relative to origin
also https://groups.google.com/forum/?hl=en-GB&fromgroups=#!forum/pi3d

User avatar
paddyg
Posts: 2616
Joined: Sat Jan 28, 2012 11:57 am
Location: UK

Re: PI3D - Rotating around local axis

Sat Jul 02, 2016 11:52 am

Hmm, this seems to work as far as I can tell doing some cross-eyed tests!
pi3d/util/StereoCam.py

Code: Select all

import ctypes
import numpy as np
import math

from pi3d.constants import *
from pi3d.Shader import Shader
from pi3d.Camera import Camera
from pi3d.shape.Sprite import Sprite
from pi3d.util.OffScreenTexture import OffScreenTexture
from pi3d.Display import Display
from pi3d import opengles

class StereoCam(object):
  """For creating an apparatus with two sprites to hold left and right
  eye views.

  This Class is used to hold the 3D Camera which should be used to draw
  the 3D objects. It also holds a 2D Camera for drawing the Sprites"""
  def __init__(self, shader="uv_flat", mipmap=False, separation=0.4, interlace=0):
    """ calls Texture.__init__ but doesn't need to set file name as
    texture generated from the framebuffer. Keyword Arguments:

      *shader*
        to use when drawing sprite, defaults to post_base, a simple
        3x3 convolution that does basic edge detection. Can be copied to
        project directory and modified as required.

      *mipmap*
        can be set to True with slight cost to speed, or use fxaa shader

      *separation*
        distance between the two camera positions - how wide apart the
        eye views are.

      *interlace*
        if interlace > 0 then the images are not taken with glScissor and
        must be drawn with a special interlacing shader.
    """
    # load shader
    if interlace <= 0:
      self.shader = Shader(shader)
    else:
      self.shader = Shader(vshader_source = """
precision mediump float;
attribute vec3 vertex;
attribute vec2 texcoord;
uniform mat4 modelviewmatrix[2];
varying vec2 texcoordout;
void main(void) {
  texcoordout = texcoord;
  gl_Position = modelviewmatrix[1] * vec4(vertex,1.0);
}
    """, fshader_source = """
precision mediump float;
uniform sampler2D tex0;
uniform sampler2D tex1;
varying vec2 texcoordout;
void main(void) {{
  vec4 texc0 = texture2D(tex0, texcoordout);
  vec4 texc1 = texture2D(tex1, texcoordout);
  vec2 coord = vec2(gl_FragCoord);
  gl_FragColor = mix(texc0, texc1, step(0.5, fract(coord.x / {:f})));
}}
    """.format(interlace * 2.0))
      #self.shader = Shader("2d_flat")
    self.camera_3d = Camera()
    self.forMtrx = np.identity(4, dtype='float32') # initially not rotated
    self.position = [0.0, 0.0, 0.0]
    self.camera_2d = Camera(is_3d=False)
    self.offs = separation / 2.0
    self.interlace = interlace
    self.textures = []
    self.sprites = []
    self.tex_list = []
    for i in range(2):
      self.textures.append(OffScreenTexture(name="stereo"))
      ix, iy = self.textures[i].ix, self.textures[i].iy
      #two sprites full width but moved so that they are centred on the
      #left and right edges. The offset values then move the uv mapping
      #so the image is on the right of the left sprite and left of the
      #right sprite
      self.sprites.append(Sprite(z=20.0, w=ix, h=iy, flip=True))
      if interlace <= 0:
        self.sprites[i].positionX(-ix/2.0 + i*ix)
        self.sprites[i].set_offset((i * 0.5 - 0.25, 0.0))
      else:
        self.sprites[i].set_2d_size(w=ix, h=iy)
      self.textures[i].blend = True
      self.textures[i].mipmap = mipmap
      self.tex_list.append(self.textures[i])
    opengles.glColorMask(1, 1, 1, 1)

  def move_camera(self, position, rot, tilt, roll=0.0):
    sy, cy = math.sin(rot), math.cos(rot)
    sx, cx = math.sin(tilt), math.cos(tilt)
    sz, cz = math.sin(roll), math.cos(roll)
    self.forMtrx = np.dot(self.forMtrx, 
                      np.dot([[cy, 0, -sy, 0], # rotation
                              [0, 1, 0, 0],
                              [sy, 0, cy, 0],
                              [0, 0, 0, 1]],
                          np.dot([[1, 0, 0, 0], # after tilt
                                  [0, cx, sx, 0],
                                  [0, -sx, cx, 0],
                                  [0, 0, 0, 1]],
                                      [[cz, sz, 0, 0], # after roll
                                       [-sz, cz, 0, 0],
                                       [0, 0, 1, 0],
                                       [0, 0, 0, 1]])))
    self.position = position

  def start_capture(self, side):
    """ after calling this method all object.draw()s will rendered
    to this texture and not appear on the display.

      *side*
        Either 0 or 1 to determine stereoscopic view
    """
    self.camera_3d.reset()
    offs = -self.offs if side == 0 else self.offs
    self.camera_3d.position([offs, 0.0, 0.0])
    self.camera_3d.mtrx = np.dot(self.forMtrx, self.camera_3d.mtrx)
    self.camera_3d.position(self.position)
    tex = self.textures[side]
    tex._start()
    if self.interlace <= 0:
      xx = tex.ix / 4.0 # draw the middle only - half width
      yy = 0
      ww = tex.ix / 2.0
      hh = tex.iy
      opengles.glEnable(GL_SCISSOR_TEST)
      opengles.glScissor(ctypes.c_int(int(xx)), ctypes.c_int(int(yy)),
                    ctypes.c_int(int(ww)), ctypes.c_int(int(hh)))

  def end_capture(self, side):
    """ stop capturing to texture and resume normal rendering to default
    """
    self.textures[side]._end()
    if self.interlace <= 0:
      opengles.glDisable(GL_SCISSOR_TEST)

  def draw(self):
    """ draw the shape using the saved texture
    """
    if self.interlace <= 0:
      for i in range(2):
        self.sprites[i].draw(self.shader, [self.tex_list[i]], 0.0, 0.0, self.camera_2d)
    else:
      self.sprites[0].draw(self.shader, self.tex_list, 0.0, 0.0, self.camera_2d)

pi3d_demos/ForestStereo.py

Code: Select all

#!/usr/bin/python
from __future__ import absolute_import, division, print_function, unicode_literals

""" ForestWalk but with stereoscopic view - i.e. for google cardboard

NB in this example the cameras have been set with a negative separation i.e.
for viewing cross-eyed as most people find this easier without a viewer!!!
If a viewer is used then the line defining CAMERA would need to be changed
to an appropriate +ve separation.

NB also, no camera has been explicitly assigned to the objects so they all
use the default instance and this will be CAMERA.camera_3d so long as the
StereoCam instance was created before any other Camera instance.
"""

import math,random

import demo
import pi3d

# Setup display and initialise pi3d
DISPLAY = pi3d.Display.create(w=1200, h=600)
DISPLAY.set_background(0.4,0.8,0.8,1)      # r,g,b,alpha
# yellowish directional light blueish ambient light
pi3d.Light(lightpos=(1, -1, -3), lightcol=(1.0, 1.0, 0.8), lightamb=(0.25, 0.2, 0.3))
CAMERA = pi3d.StereoCam(separation=-0.5, interlace=0)
""" If CAMERA is set with interlace <= 0 (default) then CAMERA.draw() will produce
two images side by side (each viewed from `separation` apart) i.e. -ve
requires viewing slightly cross-eyed.

If interlace is set to a positive value then the two images are interlaced
in vertical stripes this number of pixels wide. The resultant image needs
to be viewed through a grid. See https://github.com/pi3d/pi3d_demos/make_grid.py
"""

# load shader
shader = pi3d.Shader("uv_bump")
shinesh = pi3d.Shader("uv_reflect")
flatsh = pi3d.Shader("uv_flat")

tree2img = pi3d.Texture("textures/tree2.png")
tree1img = pi3d.Texture("textures/tree1.png")
hb2img = pi3d.Texture("textures/hornbeam2.png")
bumpimg = pi3d.Texture("textures/grasstile_n.jpg")
reflimg = pi3d.Texture("textures/stars.jpg")
rockimg = pi3d.Texture("textures/rock1.jpg")

FOG = ((0.3, 0.3, 0.4, 0.8), 650.0)
TFOG = ((0.2, 0.24, 0.22, 1.0), 150.0)

#myecube = pi3d.EnvironmentCube(900.0,"HALFCROSS")
ectex=pi3d.loadECfiles("textures/ecubes","sbox")
myecube = pi3d.EnvironmentCube(size=900.0, maptype="FACES", name="cube")
myecube.set_draw_details(flatsh, ectex)

# Create elevation map
mapsize = 1000.0
mapheight = 60.0
mountimg1 = pi3d.Texture("textures/mountains3_512.jpg")
mymap = pi3d.ElevationMap("textures/mountainsHgt.png", name="map",
                     width=mapsize, depth=mapsize, height=mapheight,
                     divx=32, divy=32) 
mymap.set_draw_details(shader, [mountimg1, bumpimg, reflimg], 128.0, 0.0)
mymap.set_fog(*FOG)

#Create tree models
treeplane = pi3d.Plane(w=4.0, h=5.0)

treemodel1 = pi3d.MergeShape(name="baretree")
treemodel1.add(treeplane.buf[0], 0,0,0)
treemodel1.add(treeplane.buf[0], 0,0,0, 0,90,0)

treemodel2 = pi3d.MergeShape(name="bushytree")
treemodel2.add(treeplane.buf[0], 0,0,0)
treemodel2.add(treeplane.buf[0], 0,0,0, 0,60,0)
treemodel2.add(treeplane.buf[0], 0,0,0, 0,120,0)

#Scatter them on map using Merge shape's cluster function
mytrees1 = pi3d.MergeShape(name="trees1")
mytrees1.cluster(treemodel1.buf[0], mymap, 0.0, 0.0, 400.0, 400.0, 50, "", 8.0, 3.0)
mytrees1.set_draw_details(flatsh, [tree2img], 0.0, 0.0)
mytrees1.set_fog(*TFOG)

mytrees2 = pi3d.MergeShape(name="trees2")
mytrees2.cluster(treemodel2.buf[0], mymap, 0.0, 0.0, 400.0, 400.0, 80, "", 6.0, 3.0)
mytrees2.set_draw_details(flatsh, [tree1img], 0.0, 0.0)
mytrees2.set_fog(*TFOG)

mytrees3 = pi3d.MergeShape(name="trees3")
mytrees3.cluster(treemodel2, mymap,0.0, 0.0, 300.0, 300.0, 20, "", 4.0, 2.0)
mytrees3.set_draw_details(flatsh, [hb2img], 0.0, 0.0)
mytrees3.set_fog(*TFOG)

#Create monument
monument = pi3d.Model(file_string="models/pi3d.obj", name="monument")
monument.set_shader(shinesh)
monument.set_normal_shine(bumpimg, 16.0, reflimg, 0.4)
monument.set_fog(*FOG)
monument.translate(100.0, -mymap.calcHeight(100.0, 235) + 12.0, 235.0)
monument.scale(20.0, 20.0, 20.0)
monument.rotateToY(65)

#screenshot number
scshots = 1

#avatar camera
rot = 0.0
tilt = 0.0
roll = 0.0
avhgt = 3.5
xm = 0.0
zm = 0.0
ym = mymap.calcHeight(xm, zm) + avhgt

# Fetch key presses
mykeys = pi3d.Keyboard()
mymouse = pi3d.Mouse(restrict = False)
mymouse.start()

omx, omy = mymouse.position()

# Display scene and rotate cuboid
while DISPLAY.loop_running():
  CAMERA.move_camera((xm, ym, zm), rot, tilt, roll)
  roll = 0.0
  myecube.position(xm, ym, zm)
  for i in range(2):
    CAMERA.start_capture(i)
    monument.draw()
    mymap.draw()
    if abs(xm) > 300:
      mymap.position(math.copysign(1000,xm), 0.0, 0.0)
      mymap.draw()
    if abs(zm) > 300:
      mymap.position(0.0, 0.0, math.copysign(1000,zm))
      mymap.draw()
      if abs(xm) > 300:
        mymap.position(math.copysign(1000,xm), 0.0, math.copysign(1000,zm))
        mymap.draw()
    mymap.position(0.0, 0.0, 0.0)
    myecube.draw()
    mytrees3.draw()
    mytrees2.draw()
    mytrees1.draw()
    CAMERA.end_capture(i)
  CAMERA.draw()

  mx, my = mymouse.position()
  buttons = mymouse.button_status()

  rot = -(mx-omx)*0.02
  tilt = (my-omy)*0.02
  omx=mx
  omy=my

  #Press ESCAPE to terminate
  k = mykeys.read()
  if k >-1 or buttons > mymouse.BUTTON_UP:
    if k == 119 or buttons == mymouse.LEFT_BUTTON:  #key W
      xm += CAMERA.camera_3d.mtrx[0, 3] 
      zm += CAMERA.camera_3d.mtrx[2, 3]
      ym = mymap.calcHeight(xm, zm) + avhgt
    elif k == 115 or buttons == mymouse.RIGHT_BUTTON:  #kry S
      xm -= CAMERA.camera_3d.mtrx[0, 3] 
      zm -= CAMERA.camera_3d.mtrx[2, 3]
      ym = mymap.calcHeight(xm, zm) + avhgt
    elif k == ord('a'):
      roll += 0.02 #radians
    elif k == ord('d'):
      roll -= 0.02
    elif k == 112:  #key P
      pi3d.screenshot("forestWalk"+str(scshots)+".jpg")
      scshots += 1
    elif k == 10:   #key RETURN
      mc = 0
    elif k == 27:  #Escape key
      mykeys.close()
      mymouse.stop()
      DISPLAY.stop()
      break

    halfsize = mapsize / 2.0
    xm = (xm + halfsize) % mapsize - halfsize # wrap location to stay on map -500 to +500
    zm = (zm + halfsize) % mapsize - halfsize
Where essentially I've just added a roll (into screen rotation) using the 'a' and 'd' keys and made rot, tilt and roll incremental values from one frame to the next. I've put a video here https://youtu.be/1guKBfqYMC0
also https://groups.google.com/forum/?hl=en-GB&fromgroups=#!forum/pi3d

User avatar
paddyg
Posts: 2616
Joined: Sat Jan 28, 2012 11:57 am
Location: UK

Re: PI3D - Rotating around local axis

Sat Jul 23, 2016 5:38 pm

I've pushed up some modifications to github pi3d develop branch to make this functionality available without hacking the module. There are also two Camera methods for helping to rebase gyro dead reckoning orientations to magnetometer vectors. See revised demo here.
also https://groups.google.com/forum/?hl=en-GB&fromgroups=#!forum/pi3d

chris-bunting
Posts: 7
Joined: Wed Jul 03, 2019 7:43 pm

Re: PI3D - Rotating around local axis

Wed Jul 03, 2019 7:53 pm

Try setting the camera's 'absolute' argument to True, then you can give it the angle in x, y, and z with each iteration, because these then apply to local camera coordinates, and not world coordinates.

The code would look something like this:

Code: Select all

camera = pi3d.Camera(absolute=True)

while display.loop_running():
    camera.rotateY(angley)
    camera.rotateX(anglex)

cowminer27
Posts: 10
Joined: Thu May 12, 2016 3:04 pm

Re: PI3D - Rotating around local axis

Wed Jul 03, 2019 9:53 pm

Thanks, I've not messed with this project for a while (I realise now that I never replied to paddy). I suspect that parameter didn't exist back then, but if it did, I missed something super obvious, my bad. Either way, I believe I ended up solving it using Paddy's solution above, but I lost the code a while ago so I can't be certain.

Thanks anyways!
Currently Working On:
- A DIY HTC Vive
- Extra stuff for my AIY project

User avatar
paddyg
Posts: 2616
Joined: Sat Jan 28, 2012 11:57 am
Location: UK

Re: PI3D - Rotating around local axis

Wed Jul 03, 2019 10:07 pm

Chris & @cowminer27, I think setting absolute=True would make the rotation about y axis act on the world vertical axis and rotation about the x axis act on the horizonal axis (horizontal in world space, but rotated with camera rotation about vertical axis) To test this (in ForestWalk with absolute=True and later mx, my = mymouse.position()) try looking straight down then rotating from side to side; you will see that the rotation is still about a vertical axis.

If you set absolute=False then rotation is always relative to the local axes so if you look straight down then rotate from side to side you will see horizon appears vertical, which is what you would expect (and is what happens with your head as the camera). absolute=False is the VR headset mode for the Camera but it is very difficult to use in normal mouse or joystick controlled game!
also https://groups.google.com/forum/?hl=en-GB&fromgroups=#!forum/pi3d

Return to “Python”