PS4 + Z3 tablet compact + Zeiss headtracker + Arduino Lenoardo – CHANGING GAMEPLAY

we started playing WarThunder on PS4 with standard PS4 setup, consisting of PS4 console and controller and FullHD monitor display.
The aim circle/cross is alway the center of your screen!

when we start using headtracking, we get new possibilities and new problems too.
Using headtracking version 1 ( moving head 45degree to left/right while looking to screen )  the center of our look isnt the same as the center of aiming!

to get better gameplay, we should set up a new color for our aiming cross, war thunder supports this setting..  not white (flying in the alps, not black, not brown.. better use any not natural terrain color, like pink..), while left/right is max 45degree so most time you will see your aiming cross, your HUD on your screen

later using headtracking version 2 ( HMD head mounted display, display moves with your head ), you can look around, above, behind your shoulders… you will not see your aiming cross, you will not see your artifical HUD altiude… you will loose controll of your  plain!
you can learn how to control in this new environment, but i guess you will not have advantages, you will have disadvantages.. as long as gamers play with unidentical equipment…

my choise number 1 is… using joystick to fly and mouse to look around
my choise numer 2 is..  using joystick to fly and ZEISS headtracker + arduino set up for impressive headtracking  version 1 ( moving head 45degree to left/right while looking to screen )… setting up color of aim cross
the other option, the HMD setup would be benefit in not so fast scenarios like semi/full-simulation battles

SONY Z3 tablet + PS4 + Remote Play + Zeiss Headtracker (playing WarTHunder)

 

took a helmet,
added a wooden system to carry the
Z3 tablet compact (268grams),
fixed Zeiss headtracker at the helmet, take care of correct alignment of axes
Zeiss headtraker connected to Laptop and then via USBSerial Adapter to Arduino Leonardo via USB to PS4

starting PS4,
starting RemotePlay from Z3,
finaly it works, flying the plane, looking around you to left, right, up, down..works great
BUT disadvantages are
– added little lag from streaming, not much, but in dogfights…
– the helmet construction with great enough distance to do not need eyeglasses creates leverage.. in addition to static weight…
– streaming video quality (not 1:1 what you know from TV screen) on Z3 in remote play app, we can use the settings icon, sitting in the upper right corner, to set various stuff and there we can set streaming quality, from standard to HIGH and fps from standard to HIGH!
http://manuals.playstation.net/document/gb/pstv/ps4link/settings_l.html

 

main problem is, how do we get 1:1 quality hdmi signal from PS4 onto our Z3 tablet WITHOUT  additional lag milliseconds ,
a solution using a cable would be accepted too
but MHL seems to be by design one direction only…

if you want to go further,
– get the Z3 in smaller distance, closer to your eyes..needing lenses, cross-eye-problem?
– get the Z3 in smaller distance will give lower leverage force/weight
– getting construction of arm as low as possible
– find a tablet or full HD panel+controller with cabled HDMI..
– check out 60GHz hdmi streamers (but missing point to use Z3 as TV screen to output incoming hdmi signals)
www.pcworld.com/article/2030041/meet-60ghz-wi-fi-the-insanely-fast-future-of-wireless-networking.html
www.wirelesshd.org/
www.siliconimage.com/solutions/wireless/

www.eurogamer.net/articles/digitalfoundry-2014-hands-on-with-playstation-tv-remote-play
http://manuals.playstation.net/document/gb/pstv/ps4link/settings_l.html

android using ethernet: http://www.bobjgear.com/which-tablets-support-ethernet.html


Zeiss Headtracker with PS4 – Solved

Zeiss Cinemizer Headtracker, USB:

works not realy with PS4, you can plug it in via USB, the Zeiss Headtracker gets recognized by PS4, for 2 seconds the incoming signals are working on PS4, but then the Headtracker gets disconneced by PS4 …

Asking Zeiss for the SDK, but never got an anwser. Ok…

But from previous project we have a base for a solution.
We use the headtracker with a Laptop, using the Zeiss Tracker Tray Configuration Tool to adjust sensitivity.
Then we use our python script to read the x,y coordinates from laptop screen. From that we calculate the relative movement between each reading of x,y. When the mouse pointer reaches the max or min values for x or y we set the x,y coordinates of actual mouse to center position of the screen and reinit some values. With that we can look around 360 degree and more .. mouse on laptop screen can hit min, max borders and gets set back to center… so we get continously relative values for our Arduino Leonardo.. connected to PS4 working as mouse input …  yeahhhh works fine… y-coordinates inverted and multiplied x 2..

remember: laptop->usb->usbserial adapter->RxTx->ArduinoLeonardo->usb->PS4

here is the script

# python C:\python_scripts\win32_read_mouse\win32_read_set_mouse.py
import win32gui
import time
import sys

import win32api, win32con
##########################################
# setting up serial communication

import serial
#import time
import os
"""
C:\Users\MrRonsen>python -m serial.tools.list_ports -a
Usage: list_ports.py [options] [<regexp>]

list_ports.py: error: no such option: -a
"""
# C:\Python27\Lib\site-packages\serial\tools\list_ports.py <<< look for options.. discoverd -v !
write_to_path = "C:\\python_scripts\\py_serial\\actual_com_ports.txt"
os.system("python -m serial.tools.list_ports -v > " + write_to_path)

fr = open( write_to_path , "rb")
frc = fr.read()
fr.close()

"""
COM14
desc: Arduino Uno (COM14)
hwid: USB VID:PID=2341:0001 SNR=55330343831351D03232
COM9
desc: Arduino Leonardo (COM9)
hwid: USB VID:PID=2341:8036
2 ports found
"""

frc = str(frc).split("\n")

com_dict = {}
COM_id = ""
for line in frc:
#print line
if line[0:3] == "COM":

COM_id = line.strip()
#print "COM_id: ",COM_id
com_dict[COM_id] = ""

if line.find("desc:") != -1:
line2 = line
line2 = line2.replace("desc: ","")
line2 = line2.replace("("+COM_id+")","")
line2 = line2.replace("\t","")
line2 = line2.strip()
#print "line2: >>",line2,"<<"
com_dict[COM_id] = line2


arduino_leonardo_COMport = ""
arduino_uno_COMport = ""

for e in com_dict:
print e, ": ", com_dict[e]

if com_dict[e].lower().find("leonardo")!=-1:
arduino_leonardo_COMport = e
if com_dict[e].lower().find("uno")!=-1:
arduino_uno_COMport = e

print "arduino_uno_COMport: ",arduino_uno_COMport
print "arduino_leonardo_COMport: ",arduino_leonardo_COMport
if 1==1:

#COMport = 14
#serial_port = COMport-1
#serial_port = "COM14"
#serial_port = "COM11"
#serial_port = arduino_leonardo_COMport
serial_port = arduino_uno_COMport

baudrate = 9600

connected = False
"""
ser = serial.Serial(port=serial_port,
baudrate=baudrate,
parity=serial.PARITY_NONE,
stopbits=serial.STOPBITS_TWO,
bytesize=serial.EIGHTBITS,
timeout=0,
xonxoff=0,
rtscts=0)
"""


ser = serial.Serial( serial_port, baudrate) # open first serial port
#ser.DtrEnable = "true";

print("initialising")
time.sleep(2) # waiting the initialization...

print ser.name # check which port was really used
# EOF setting up serial
######################################################

if 1==1:
flags, hcursor, (x,y) = win32gui.GetCursorInfo()
print "flags: ",flags
print "hcursor: ",hcursor
print "x: ",x
print "y: ",y
print

#if 1==11:
# print "..............."
# import msvcrt
# print msvcrt.getch()
# print "..............."

screen_max_width = 1280
screen_max_height = 800

screen_max_width_half = screen_max_width/2
screen_max_height_half = screen_max_height/2

screen_max_width_minusX = screen_max_width-1
screen_max_height_minusX = screen_max_height-1

win32api.SetCursorPos((screen_max_width_half,screen_max_height_half))

#current_frame_x = screen_max_width_half
#current_frame_y = screen_max_height_half

prev_frame_x = screen_max_width_half
prev_frame_y = screen_max_height_half
if 1==1:

collected = 0
attempts = 50

while collected < attempts :

try:
x,y = win32gui.GetCursorPos()

collected += 1

to_print = x,y ," , attempt: ",collected
to_print = str(to_print) + "\r"

# 1280 x 800 Laptop Bildschirm

do_reset = 0

if x == 0 or x == screen_max_width_minusX:
do_reset = 1
if y == 0 or y == screen_max_height_minusX:
do_reset = 1

if do_reset == 1:
x = screen_max_width_half # mitte
y = screen_max_height_half
win32api.SetCursorPos((x,y))
prev_frame_x = x # avoid jump
prev_frame_y = y #avoid jump

sys.stdout.write( to_print )
sys.stdout.flush()

if collected == attempts:
collected = 0

#rel_x = 0
#rel_y = 0

rel_x = x - prev_frame_x
rel_y = y - prev_frame_y

if rel_x != 0 or rel_y != 0:

#print
print "rel_x: ",rel_x
print "rel_y: ",rel_y

rel_y2 = -1 * rel_y # inertieren
rel_y2 = rel_y2*2 # doppelt umsetzen
ser.write( str(rel_x) + "," + str(rel_y2) + "\n" )

prev_frame_x = x
prev_frame_y = y
#time.sleep(0.01) # set the inveral to read next
time.sleep(0.001) # set the inveral to read next
except:
print "exception"