PS4 + Z3 tablet compact + Zeiss headtracker + Arduino Lenoardo – CHANGING GAMEPLAY

we started playing WarThunder on PS4 with standard PS4 setup, consisting of PS4 console and controller and FullHD monitor display.
The aim circle/cross is alway the center of your screen!

when we start using headtracking, we get new possibilities and new problems too.
Using headtracking version 1 ( moving head 45degree to left/right while looking to screen )  the center of our look isnt the same as the center of aiming!

to get better gameplay, we should set up a new color for our aiming cross, war thunder supports this setting..  not white (flying in the alps, not black, not brown.. better use any not natural terrain color, like pink..), while left/right is max 45degree so most time you will see your aiming cross, your HUD on your screen

later using headtracking version 2 ( HMD head mounted display, display moves with your head ), you can look around, above, behind your shoulders… you will not see your aiming cross, you will not see your artifical HUD altiude… you will loose controll of your  plain!
you can learn how to control in this new environment, but i guess you will not have advantages, you will have disadvantages.. as long as gamers play with unidentical equipment…

my choise number 1 is… using joystick to fly and mouse to look around
my choise numer 2 is..  using joystick to fly and ZEISS headtracker + arduino set up for impressive headtracking  version 1 ( moving head 45degree to left/right while looking to screen )… setting up color of aim cross
the other option, the HMD setup would be benefit in not so fast scenarios like semi/full-simulation battles

SONY Z3 tablet + PS4 + Remote Play + Zeiss Headtracker (playing WarTHunder)

 

took a helmet,
added a wooden system to carry the
Z3 tablet compact (268grams),
fixed Zeiss headtracker at the helmet, take care of correct alignment of axes
Zeiss headtraker connected to Laptop and then via USBSerial Adapter to Arduino Leonardo via USB to PS4

starting PS4,
starting RemotePlay from Z3,
finaly it works, flying the plane, looking around you to left, right, up, down..works great
BUT disadvantages are
– added little lag from streaming, not much, but in dogfights…
– the helmet construction with great enough distance to do not need eyeglasses creates leverage.. in addition to static weight…
– streaming video quality (not 1:1 what you know from TV screen) on Z3 in remote play app, we can use the settings icon, sitting in the upper right corner, to set various stuff and there we can set streaming quality, from standard to HIGH and fps from standard to HIGH!
http://manuals.playstation.net/document/gb/pstv/ps4link/settings_l.html

 

main problem is, how do we get 1:1 quality hdmi signal from PS4 onto our Z3 tablet WITHOUT  additional lag milliseconds ,
a solution using a cable would be accepted too
but MHL seems to be by design one direction only…

if you want to go further,
– get the Z3 in smaller distance, closer to your eyes..needing lenses, cross-eye-problem?
– get the Z3 in smaller distance will give lower leverage force/weight
– getting construction of arm as low as possible
– find a tablet or full HD panel+controller with cabled HDMI..
– check out 60GHz hdmi streamers (but missing point to use Z3 as TV screen to output incoming hdmi signals)
www.pcworld.com/article/2030041/meet-60ghz-wi-fi-the-insanely-fast-future-of-wireless-networking.html
www.wirelesshd.org/
www.siliconimage.com/solutions/wireless/

www.eurogamer.net/articles/digitalfoundry-2014-hands-on-with-playstation-tv-remote-play
http://manuals.playstation.net/document/gb/pstv/ps4link/settings_l.html

android using ethernet: http://www.bobjgear.com/which-tablets-support-ethernet.html

SONY Z3 tablet + PS4 + Remote Play (playing WarTHunder)

Modem->Router(wifi 2,4+5GHz,lanports)->ethernet cable->Router2(wifi 2,4+5GHz,lanports)

Router2(wifi 2,4+5GHz,lanports)->ethernet cable->PS4
Router2(wifi 2,4+5GHz,lanports)->5GHz->Sony Z3 tablet compact

description how to set up remote play are available on internet.. it works..

BUT graphic gets streamed as video stream from PS4 to Z3 tablet and so the Video Encoding gets involved.. resulting in not the same graphic brilliance like we see on main tv screen.

UPDATE: on Z3 in remote play app, we can use the settings icon, sitting in the upper right corner, to set various stuff and there we can set streaming quality, from standard to HIGH and fps from standard to HIGH!
http://manuals.playstation.net/document/gb/pstv/ps4link/settings_l.html

 

Maybe we could get better results when using a hdmi-splitter, external hdmi streamer, a app that reads our own stream, controlling quality, codec..  we need not realy remote play, only the screen on a tablet infront of our eyes, mimik a 2D head mounted display for flight simulations..

Zeiss Headtracker with PS4 – Solved

Zeiss Cinemizer Headtracker, USB:

works not realy with PS4, you can plug it in via USB, the Zeiss Headtracker gets recognized by PS4, for 2 seconds the incoming signals are working on PS4, but then the Headtracker gets disconneced by PS4 …

Asking Zeiss for the SDK, but never got an anwser. Ok…

But from previous project we have a base for a solution.
We use the headtracker with a Laptop, using the Zeiss Tracker Tray Configuration Tool to adjust sensitivity.
Then we use our python script to read the x,y coordinates from laptop screen. From that we calculate the relative movement between each reading of x,y. When the mouse pointer reaches the max or min values for x or y we set the x,y coordinates of actual mouse to center position of the screen and reinit some values. With that we can look around 360 degree and more .. mouse on laptop screen can hit min, max borders and gets set back to center… so we get continously relative values for our Arduino Leonardo.. connected to PS4 working as mouse input …  yeahhhh works fine… y-coordinates inverted and multiplied x 2..

remember: laptop->usb->usbserial adapter->RxTx->ArduinoLeonardo->usb->PS4

here is the script

# python C:\python_scripts\win32_read_mouse\win32_read_set_mouse.py
import win32gui
import time
import sys

import win32api, win32con
##########################################
# setting up serial communication

import serial
#import time
import os
"""
C:\Users\MrRonsen>python -m serial.tools.list_ports -a
Usage: list_ports.py [options] [<regexp>]

list_ports.py: error: no such option: -a
"""
# C:\Python27\Lib\site-packages\serial\tools\list_ports.py <<< look for options.. discoverd -v !
write_to_path = "C:\\python_scripts\\py_serial\\actual_com_ports.txt"
os.system("python -m serial.tools.list_ports -v > " + write_to_path)

fr = open( write_to_path , "rb")
frc = fr.read()
fr.close()

"""
COM14
desc: Arduino Uno (COM14)
hwid: USB VID:PID=2341:0001 SNR=55330343831351D03232
COM9
desc: Arduino Leonardo (COM9)
hwid: USB VID:PID=2341:8036
2 ports found
"""

frc = str(frc).split("\n")

com_dict = {}
COM_id = ""
for line in frc:
#print line
if line[0:3] == "COM":

COM_id = line.strip()
#print "COM_id: ",COM_id
com_dict[COM_id] = ""

if line.find("desc:") != -1:
line2 = line
line2 = line2.replace("desc: ","")
line2 = line2.replace("("+COM_id+")","")
line2 = line2.replace("\t","")
line2 = line2.strip()
#print "line2: >>",line2,"<<"
com_dict[COM_id] = line2


arduino_leonardo_COMport = ""
arduino_uno_COMport = ""

for e in com_dict:
print e, ": ", com_dict[e]

if com_dict[e].lower().find("leonardo")!=-1:
arduino_leonardo_COMport = e
if com_dict[e].lower().find("uno")!=-1:
arduino_uno_COMport = e

print "arduino_uno_COMport: ",arduino_uno_COMport
print "arduino_leonardo_COMport: ",arduino_leonardo_COMport
if 1==1:

#COMport = 14
#serial_port = COMport-1
#serial_port = "COM14"
#serial_port = "COM11"
#serial_port = arduino_leonardo_COMport
serial_port = arduino_uno_COMport

baudrate = 9600

connected = False
"""
ser = serial.Serial(port=serial_port,
baudrate=baudrate,
parity=serial.PARITY_NONE,
stopbits=serial.STOPBITS_TWO,
bytesize=serial.EIGHTBITS,
timeout=0,
xonxoff=0,
rtscts=0)
"""


ser = serial.Serial( serial_port, baudrate) # open first serial port
#ser.DtrEnable = "true";

print("initialising")
time.sleep(2) # waiting the initialization...

print ser.name # check which port was really used
# EOF setting up serial
######################################################

if 1==1:
flags, hcursor, (x,y) = win32gui.GetCursorInfo()
print "flags: ",flags
print "hcursor: ",hcursor
print "x: ",x
print "y: ",y
print

#if 1==11:
# print "..............."
# import msvcrt
# print msvcrt.getch()
# print "..............."

screen_max_width = 1280
screen_max_height = 800

screen_max_width_half = screen_max_width/2
screen_max_height_half = screen_max_height/2

screen_max_width_minusX = screen_max_width-1
screen_max_height_minusX = screen_max_height-1

win32api.SetCursorPos((screen_max_width_half,screen_max_height_half))

#current_frame_x = screen_max_width_half
#current_frame_y = screen_max_height_half

prev_frame_x = screen_max_width_half
prev_frame_y = screen_max_height_half
if 1==1:

collected = 0
attempts = 50

while collected < attempts :

try:
x,y = win32gui.GetCursorPos()

collected += 1

to_print = x,y ," , attempt: ",collected
to_print = str(to_print) + "\r"

# 1280 x 800 Laptop Bildschirm

do_reset = 0

if x == 0 or x == screen_max_width_minusX:
do_reset = 1
if y == 0 or y == screen_max_height_minusX:
do_reset = 1

if do_reset == 1:
x = screen_max_width_half # mitte
y = screen_max_height_half
win32api.SetCursorPos((x,y))
prev_frame_x = x # avoid jump
prev_frame_y = y #avoid jump

sys.stdout.write( to_print )
sys.stdout.flush()

if collected == attempts:
collected = 0

#rel_x = 0
#rel_y = 0

rel_x = x - prev_frame_x
rel_y = y - prev_frame_y

if rel_x != 0 or rel_y != 0:

#print
print "rel_x: ",rel_x
print "rel_y: ",rel_y

rel_y2 = -1 * rel_y # inertieren
rel_y2 = rel_y2*2 # doppelt umsetzen
ser.write( str(rel_x) + "," + str(rel_y2) + "\n" )

prev_frame_x = x
prev_frame_y = y
#time.sleep(0.01) # set the inveral to read next
time.sleep(0.001) # set the inveral to read next
except:
print "exception"

PS4 remote play + Head Tracking

Headtracker from ZEISS (USB, ready to use any system, mouse-wise) http://www.zeiss.de/cinemizer-oled/de_de/produktinformation.html
but 150,- bugs.. euros?

an own headtracker, based on Arduino + sensors would cost hardware at least 20+20+20 60 Euros an lot of work…

 

a CARL ZEISS cinemizer costs 650,- bugs, insane, crazy..
http://www.rcmodellbau-online.de/ 560,- euros
and we have no chance to test? ok, order and send back within legal time.
without testing such a device i would hang a Xperia 3 tablet infront of my eyes..
the tablet would be usefull as tablet too.. when you are not playing..
but with a tablet infront, we need exact headtracking… is a 20 bugs air mouse accurate enough?
first test with arduino and
Gyro: Triple Axis Gyro – L3G4200D (BK-L3G4200D)
Accelerometer: MMA7361L 3-Achsen Beschleunigungssensor (IM120712010)
show, that accurate headtracking isnt done with a snip..

 

for DIY  headtracker we have to leave single sensors behind us.. gyro, accelerometer, magnetometer… there are complete sensors available, doing combining all of them
http://en.wikipedia.org/wiki/Inertial_measurement_unit

War Thunder – PS4 – Headtracking SELBSTGEMACHT – FINAL

2 Point Head Tracking,
1 not moving referenz point (at your nacklace)
1 moving point on a wooden bar on your helmet
60 frames per second, stable

 

benötigte Hardware:
– PS3 Eye Cam ( very high framerates )
– PC/Laptop ( DualCore 2×1,8GHz, 4GB, Win7 )
– USB Serial Light Adapter
– Arduino Leonardo
– PS4
– evtl. USB Hub
– LED infrared, 2 pieces
– Resistors 100 Ohm, 2 pieces
– Batteries/Accus, AA 1.2V , 4 pieces
– Batterie Case with switch, 2 pieces
– helmet or similar else
– wooden bar or similar else
– duct tape

Hardware Hacks:
– PS3 Eye Cam – IR blocking Filter entfernen und Tageslicht blocking filter (Diskette) einsetzen

Software:
– PC/Laptop – Windows Treiber der PS3 Eye Cam
– PC/Laptop – Python 2.7.3
– PC/Laptop – openCV
– PC/Laptop – pyserial
– PC/Laptop – Arduino IDE
– Arduino Leonardo – extra sketch

Following script holds everything together:
– reading camera stream,
– analysing, finding blobs,
– calculating distances,
– calculating movement between current versus previous frame
– sending relative movement over USBSerialLight Adapter to ArduinoLeonardo

import cv2
import time
import numpy 

# python C:\python_scripts\opencv\cam_vid_60fps_binarized_flipped_blobs_SERIAL.py

##########################################
# setting up serial communication

import serial
#import time
import os




"""
C:\Users\XXXXX>python -m serial.tools.list_ports -a
Usage: list_ports.py [options] []

list_ports.py: error: no such option: -a
"""
# C:\Python27\Lib\site-packages\serial\tools\list_ports.py  <<< look for options.. discoverd -v ! write_to_path = "C:\\python_scripts\\py_serial\\actual_com_ports.txt" os.system("python -m serial.tools.list_ports -v > " + write_to_path)

fr = open( write_to_path , "rb")
frc = fr.read()
fr.close()

"""
COM14               
    desc: Arduino Uno (COM14)
    hwid: USB VID:PID=2341:0001 SNR=55330343831351D03232
COM9                
    desc: Arduino Leonardo (COM9)
    hwid: USB VID:PID=2341:8036
2 ports found
"""

frc = str(frc).split("\n")

com_dict = {}
COM_id = ""
for line in frc:
   #print line
   if line[0:3] == "COM":
    
     COM_id = line.strip()
     #print "COM_id: ",COM_id
     com_dict[COM_id]  =  ""
     
   if line.find("desc:") != -1:
    line2 = line
    line2 = line2.replace("desc: ","")
    line2 = line2.replace("("+COM_id+")","")
    line2 = line2.replace("\t","")
    line2 = line2.strip()
    #print "line2: >>",line2,"<<"
    com_dict[COM_id] = line2
    
    
arduino_leonardo_COMport = ""
arduino_uno_COMport = ""

for e in com_dict:
    print e, ": ", com_dict[e]
    
    if com_dict[e].lower().find("leonardo")!=-1:
      arduino_leonardo_COMport = e
    if com_dict[e].lower().find("uno")!=-1:
      arduino_uno_COMport = e

print "arduino_uno_COMport: ",arduino_uno_COMport
print "arduino_leonardo_COMport: ",arduino_leonardo_COMport




if 1==1:

  #COMport = 14
  #serial_port = COMport-1
  #serial_port = "COM14"
  #serial_port = "COM11"
  #serial_port = arduino_leonardo_COMport
  serial_port = arduino_uno_COMport

  baudrate = 9600

  connected = False

  
  """
  ser = serial.Serial(port=serial_port,
baudrate=baudrate,
parity=serial.PARITY_NONE,
stopbits=serial.STOPBITS_TWO,
bytesize=serial.EIGHTBITS,
timeout=0,
xonxoff=0,
rtscts=0)
        """
        
        
  ser = serial.Serial( serial_port, baudrate)  # open first serial port
  #ser.DtrEnable = "true";
  
  print("initialising")
  time.sleep(2) # waiting the initialization...

  print ser.name          # check which port was really used


# EOF setting up serial
######################################################
#

# python C:\python_scripts\opencv\cam_vid_60fps_binarized_flipped_blobs_SERIAL.py

# http://docs.opencv.org/trunk/doc/py_tutorials/py_gui/py_video_display/py_video_display.html


#cam = cv2.VideoCapture(0) # laptop eingebaute cam
#cam = cv2.VideoCapture(1) # wenn direct am laptop
cam = cv2.VideoCapture(1)

print "cam.isOpened(): ",cam.isOpened()

# - CV_CAP_PROP_FPS Frame rate.
#print "FPS: ", cam.get( 10 ) # ? ret
#print "x: ", cam.GetCaptureProperty( CV_CAP_PROP_FPS )
print "CV_CAP_PROP_FRAME_WIDTH:  ", cam.get(3)
print "CV_CAP_PROP_FRAME_HEIGHT: ", cam.get(4)
print "CV_CAP_PROP_FPS:          ", cam.get(5)
# cam.get(propId)   
# propId integer 0 bis 18

if 1==11:
   size = (int(cv2.GetCaptureProperty(cam , cv2.CV_CAP_PROP_FRAME_WIDTH)), int(cv2.GetCaptureProperty(cam , cv2.CV_CAP_PROP_FRAME_HEIGHT)))
   print "size: ",size


"""
Property identifier. It can be one of the following:
0 CV_CAP_PROP_POS_MSEC Current position of the video file in milliseconds or video capture timestamp.
1 CV_CAP_PROP_POS_FRAMES 0-based index of the frame to be decoded/captured next.
2 CV_CAP_PROP_POS_AVI_RATIO Relative position of the video file: 0 - start of the film, 1 - end of the film.
3 CV_CAP_PROP_FRAME_WIDTH Width of the frames in the video stream.
4 CV_CAP_PROP_FRAME_HEIGHT Height of the frames in the video stream.
5 CV_CAP_PROP_FPS Frame rate.
6 CV_CAP_PROP_FOURCC 4-character code of codec.
7 CV_CAP_PROP_FRAME_COUNT Number of frames in the video file.
8 CV_CAP_PROP_FORMAT Format of the Mat objects returned by retrieve() .
9 CV_CAP_PROP_MODE Backend-specific value indicating the current capture mode.
10 CV_CAP_PROP_BRIGHTNESS Brightness of the image (only for cameras).
11 CV_CAP_PROP_CONTRAST Contrast of the image (only for cameras).
12 CV_CAP_PROP_SATURATION Saturation of the image (only for cameras).
13 CV_CAP_PROP_HUE Hue of the image (only for cameras).
14 CV_CAP_PROP_GAIN Gain of the image (only for cameras).
15 CV_CAP_PROP_EXPOSURE Exposure (only for cameras).
16 CV_CAP_PROP_CONVERT_RGB Boolean flags indicating whether images should be converted to RGB.
17 CV_CAP_PROP_WHITE_BALANCE Currently not supported
18 CV_CAP_PROP_RECTIFICATION Rectification flag for stereo cameras (note: only supported by DC1394 v 2.x backend currently)
"""

if 1==11:

        cam.set(3,320) # CV_CAP_PROP_FRAME_WIDTH
        cam.set(4,240) # CV_CAP_PROP_FRAME_HEIGHT

cam.set(5,100) # fps
print "cam get fps: ", cam.get(5) # fps


# 'module' object has no attribute 'createBackgroundSubtractorMOG'
#fgbg = cv2.BackgroundSubtractorMOG()
#fgbg = cv2.BackgroundSubtractorMOG2()

counter_frames = 0
start_time = time.time()
fps = 0

# http://opencv-python-tutroals.readthedocs.org/en/latest/py_tutorials/py_imgproc/py_morphological_ops/py_morphological_ops.html
#kernel = numpy.ones((2,2),numpy.uint8)
#kernel = numpy.ones((5,5),numpy.uint8)
kernel = cv2.getStructuringElement(cv2.MORPH_RECT,(5,5))
#kernel = cv2.getStructuringElement(cv2.MORPH_ELLIPSE,(5,5))
#kernel = cv2.getStructuringElement(cv2.MORPH_CROSS,(5,5))
#kernel = numpy.ones((10,10),numpy.uint8)

cv2__THRESH_BINARY = cv2.THRESH_BINARY
cv2__flip          = cv2.flip
cv2__morphologyEx  = cv2.morphologyEx
cv2__MORPH_OPEN    = cv2.MORPH_OPEN
cv2__MORPH_CLOSE   = cv2.MORPH_CLOSE
cv2__imshow        = cv2.imshow
cv2__waitKey       = cv2.waitKey

if 1==11:
   flags = [i for i in dir(cv2) if i.startswith('COLOR_')]
   for flag in flags:
    if flag.find("RGB")!=-1:
       print flag
       # COLOR_RGB2GRAY


prev_frame_dist_x = 999999  #init  value
prev_frame_dist_y = 999999 # init value

while(True):     

        
        ret, frame = cam.read()
        
        #cv2.imshow('frame', frame)
        
        # Our operations on the frame come here
        frame = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
        
        #gray = cv2.cvtColor(frame, cv2.THRESH_BINARY)
        tresh = 253
        #tresh = 200
        ret,frame = cv2.threshold(frame,tresh,255,cv2__THRESH_BINARY )
        frame = cv2__flip(frame, 1) # 
        
        #print "frame.dtype: ",frame.dtype

        # http://opencv-python-tutroals.readthedocs.org/en/latest/py_tutorials/py_imgproc/py_morphological_ops/py_morphological_ops.html

        #frame = cv2.erode(frame,kernel,iterations = 1)
        frame = cv2.dilate(frame,kernel,iterations = 2)
        #frame = cv2.erode(frame,kernel,iterations = 3)
        #frame = cv2__morphologyEx(frame, cv2__MORPH_OPEN, kernel) # erosion followed by dilation
        #frame = cv2__morphologyEx(frame,  cv2.MORPH_TOPHAT, kernel) 
        #frame = cv2.morphologyEx(frame, cv2.MORPH_GRADIENT, kernel) # does outline
        cv2__imshow('frame', frame)
        
        #print "frame.dtype: ",frame.dtype

        #fgmask = fgbg.apply(frame)
        #cv2.imshow('frame',fgmask)
        
		
        #x = cv2.cvtColor(frame, cv2.CV_SHAPE_CROSS)    
        #cv2.imshow('frame', x)



        #frame = cv2.convertTo(frame, CV_8U)
     
        # http://opencvpython.blogspot.de/2012/04/contour-features.html
        contours, _ = cv2.findContours(frame, cv2.RETR_LIST, cv2.CHAIN_APPROX_SIMPLE)
        contour_number = 0
        contours_dict = {}

        for c in contours:

            rect = cv2.boundingRect(c)

            # rect[2]   width
            # rect[3]   height
            # sucht nur grosse blobs, alles kleiner 10x10 wird verworfen
            if rect[2] < 10 or rect[3] < 10: continue
            contour_number = contour_number+1
            
            print "contour_number: ",contour_number
            #print cv2.contourArea(c)
            print rect # x,y width,height
            contours_dict[contour_number] = rect
            print 

            if contour_number == 2:
               con1_x = contours_dict[1][0]
               con2_x = contours_dict[2][0]

               diff_con1_x_con2_x = con1_x - con2_x
               print "diff_con1_x_con2_x: ", diff_con1_x_con2_x


               con1_y = contours_dict[1][1]
               con2_y = contours_dict[2][1]

               diff_con1_y_con2_y = con1_y - con2_y
               print "diff_con1_y_con2_y: ", diff_con1_y_con2_y

               if diff_con1_x_con2_x != 999999: # 999999 is starting init val

                  diff_prev_frame_x = diff_con1_x_con2_x - prev_frame_dist_x
                  print "diff_prev_frame_x: ",diff_prev_frame_x
                  # x negativ value = move to the right
                  # x positiv value = move to the left
                  diff_prev_frame_y = diff_con1_y_con2_y - prev_frame_dist_y
                  # y negativ value = move to bottom
                  # y positiv value = move to top
                  print "diff_prev_frame_y: ",diff_prev_frame_y

                  # send these values to arduino
                  # diff_prev_frame_x
                  # diff_prev_frame_y
                  # or manipulate them multiple 2 or progressive
                  diff_prev_frame_x = diff_prev_frame_x * -1
                  
                  # ABSoluter wert, positiv.. d.h negativ number positiv machen
                  ABS_x = diff_con1_x_con2_x
                  if diff_con1_x_con2_x < 0:
                     ABS_x = diff_con1_x_con2_x * -1
                  ABS_y = diff_con1_y_con2_y
                  if diff_con1_y_con2_y < 0:
                     ABS_y = diff_con1_y_con2_y * -1

                  print "ABS_x: ",ABS_x
                  print "ABS_y: ",ABS_y

                  if ABS_x < 100:
                     diff_prev_frame_x = diff_prev_frame_x * 2 # 2
                  else:                     
                     diff_prev_frame_x = diff_prev_frame_x * 4

                  # 240 center point y
                  #  280
                  if ABS_y > 200 or ABS_y < 280:                      diff_prev_frame_y = diff_prev_frame_y * 4 # 4                   else:                      diff_prev_frame_y = diff_prev_frame_y * 8                   ser.write( str(diff_prev_frame_x) + "," + str(diff_prev_frame_y) + "\n" )                prev_frame_dist_x = diff_con1_x_con2_x                prev_frame_dist_y = diff_con1_y_con2_y                # now map the distances                # to mouse movements                # mouse movement                 # there is an posiblity to use absolute mouse via arduino,                 # http://forum.arduino.cc/index.php?topic=94140.0                # http://arduino.stackexchange.com/questions/3531/using-leonardo-and-getting-absolute-x-y-mouse-values                # WE NEED DATA FROM PREV FRAME                # CALC differences                # these differences map to mouse moves             #print rect.bx,rect.by #,c.bw,c.bh         #frame = cv2.drawContours(frame, contours, -1, (0,255,0), 3) 		         # to quit press key "q"         if cv2__waitKey(1) & 0xFF == ord('q'):             break 	          if 1==1:               counter_frames = counter_frames + 1                       #print "counter_frames: ",counter_frames 	                 cur_time = time.time()               #print "start_time: ",start_time               #print "cur_time: ",cur_time               diff_time = cur_time - start_time               #print "diff_time: ",diff_time               if diff_time > 5: # 5 sekunden
                 fps = counter_frames / 5 # frames pro sekunde		 
                 counter_frames = 0
                 start_time = time.time()
                 print "fps: ",fps
        print "fps: ",fps
	  
	  
	  
cam.release()
cv2.destroyAllWindows()

Sketch for Arduino Leonardo

/*
  Serial1-Mouse-Control 
 
 
 The mouse movement is always relative. 
 This sketch reads Serial1(RxTx) and uses incoming data 
 to set the movement of the mouse (over Serial, Main USB Port).
 
 WARNING:  When you use the Mouse.move() command, the Arduino takes
 over your mouse!  Make sure you have control before you use the mouse commands.
 
2015-02
Ronny Gey
  
*/


#include 


// read string
String mouse_x = "";
String mouse_y = "";
  
  
void setup() {
  
  // http://arduino.cc/en/pmwiki.php?n=Reference/Serial
  // http://arduino.cc/en/Main/ArduinoBoardLeonardo
  
  // Serial not needed, this is used by mouse control...
  // only for string tests, debugging
  //Serial.begin(9600); // first Serial Port, main USB port... OR sending to PS4
  //while (!Serial) {
  //  ; // wait for serial port to connect. Needed for Leonardo only
  //}
  
  // second Serial Port on Leonardo! pins Rx(Pin0)/Tx(Pin1) for receiving data from Laptop
  Serial1.begin(9600); 
  while (!Serial1) {
    ; // wait for serial port to connect. Needed for Leonardo only
  }

  
  // initialize mouse control:
  Mouse.begin();
}

void loop() {
  // read Serial_1, data coming from Laptop

  // send data only when you receive data:
  // read RxTx
  if (Serial1.available() > 0) {
    
      // getting x,y from Laptop over Serial1
      // Kommaseparated value: x,y\n
      // -10,5
      // 5,-10
      // 10,10
      mouse_x = Serial1.readStringUntil(',');
      mouse_y = Serial1.readStringUntil('\n');
      //Serial1.print("I received X,Y: ");
      //Serial1.print(mouse_x);
      //Serial1.print(" : ");
      //Serial1.println(mouse_y); 
      
      //Syntax:  Mouse.move(xVal, yPos, wheel);
      Mouse.move(mouse_x.toInt(), mouse_y.toInt(), 0);
      // http://arduino.cc/en/Tutorial/StringToIntExample     
     
  } // EOF serial1 available
   
 
  /* only for debugging purpose
  // send data only when you receive data:
  if (Serial.available() > 0) {
      
      // getting x,y from Laptop over Serial1
      // comma separated value: x,y\n
      // -10,5
      // 5,-10
      // 10,10
      mouse_x = Serial.readStringUntil(',');
      mouse_y = Serial.readStringUntil('\n');
      Serial.print("I received X,Y: ");
      Serial.print(mouse_x);
      Serial.print(" : ");
      Serial.println(mouse_y);     

     
  } // EOF serial available
  */  
        
}

thats all

 

 

War Thunder – PS4 – HMD HeadMountedDisplay – FPV FirstPersonView – 2D

War Thunder – PS4 –
HMD HeadMountedDisplay –
FPV FirstPersonView –
2D

folgendes Video von TannerRice zeigt
WarThunder mit iPhone HMD(cardboard version)
Wie kann man das anstatt mit iPhone und PC Technik nun für PS4 umsetzen?
Tip: auf Morpheus warten

 

theoretische Möglichkeiten:

Sony PS4
Game: War Thunder (Plane Simulation with Head Movement, FPV First Person View)
Remote Play
Sony Tablet xperia z3:
– 8″ FullHD 379,- to use 2D not 3D
– 213,3 x 6,4 x 123,6 mm
– 270 g
– 1900 x 1200 pixel
– micro USB OTG!(usb-otg to LAN possible ? instead wifi)

www.sony.de/electronics/xperia-tablet-z/sgp611-sgp612-sgp621
www.sony.de/electronics/xperia-tablet-z/sgp611-sgp612-sgp621/specifications

www.google.com/get/cardboard/ << keyFeature.. is 2D viewable?
www.mrcardboard.de/
www.pcardboard.com/index.php?route=common/home
www.ebay.de/itm/like/371175741936?lpid=106&chn=ps

www.vrnerds.de/  – Das deutsche Virtual Reality Showcase und Blog – VR∙Nerds

www.gaminggadgets.de/refugio-3d-billiger-papp-bausatz-fuer-vr-brille

Problems:
– bei 6″ ist Schluss mit HeadMounted Helmen mit Phone/Table-Aufnahmen?
– Side-by-Side Darstellung wird benötigt, diese müsste bereits die Kombo „PS4+TabletApp+RemotePlay“ bereitstellen

Zur reinen Bildübertragung benötigen wir noch das HeadTracking, allerdings in weit besser Qualität als in der bisherigen ersten Version.

IR InfraRed (active or reflective)
Sony PS3 Cam (60fps)
PC/Laptop with openCV/simpleCV/Python
Arduino Serial Light (frome PC usb to serial Arduino Leonardo)
Arduino Leonardo (hardware acting as mouse input at ps4 usb)

 

 

War Thunder – PS4 – Headtracking SELBSTGEMACHT – first tests

Das Headtracking auf der PS4 im Spiel WarThunder ist schon ganz nett,
aber für mich persönlich nicht 100% zufriedenstellend.
Können wir uns das Headtracking selbst zusammen-hacken?

Eine USB-Mouse an die PS4 angeschliessen, funktioniert.
Damit kann man die Ansicht im Flugzeug sehr gut steuern und
mit einem Links-Klick die Sicht sofort geradeaus zentrieren.
Ok.

Mit der Windows Software Freetrack http://www.free-track.net/english/
kann man am Windows PC mit einer Camera/WebCam
bereits MotionCapturing (1-Point,3-Point,…) benutzen.
Ok.

Kann man der PS4 eine Mouse vortäuschen,
deren Bewegungen man programmieren kann?
Yes we can…
Mit dem Arduino Leonardo und dem ButtonMouseControl Rezept http://arduino.cc/en/Tutorial/ButtonMouseControl
kann man am PC den Arduino als USB Mouse anschliessen, und mit den an Arduino angeschlossenen Buttons die Richtungen hoch/runter, links/rechts, Linksklick
ausführen… it works.
Ok.

Funktioniert Arduino Leonardo ButtonMouseControl
auch an der PS4, im Spiel War Thunder?
JA, es funktioniert!
Ok.

Was wir jetzt noch brauchen,
sind Daten von einem MotionCapturing Tool
und die Übertragung vom PC an Arduino Leonardo.

Wenn sich herausstellt, dass es mit Freetrack nicht möglich ist,
die Bewegungsdaten auszulesen und in geeigneter Weise an Arduino zu übertragen,
dann muss OpenCV / SimpleCV (Python) herhalten, was den Vorteil hätte,
dass es sich auch auf Linux portieren ließe.
Dabei müsste man diverse Berechnungen selbst programmieren.

andere MC Projecte:
FaceTrackNoIR, opentrack,freepie,..

Wenn Tools wie FaceTrackNoIR,Freetrack eine MouseEmulation haben,
dann könnte man mit einem einfachen Python Code die Mouse Position auslesen,
in einer Endlosschleife die Mouse permanent verfolgen.

Python 2.7 + win32 Extension
https://www.python.org/getit/windows/
http://sourceforge.net/projects/pywin32/files/pywin32/Build%20219/

# python C:\python_scripts\win32_read_mouse\win32_read_mouse.py

import win32gui
import time
import sys
      
if 1==1:
	
	collected = 0
	attempts  = 50
	
	while collected < attempts :
	
	    try:
               x,y = win32gui.GetCursorPos() 
				
               collected += 1
		
               to_print = x,y  ," , attempt: ",collected
               to_print = str(to_print) + "\r"
				
               sys.stdout.write( to_print )
               sys.stdout.flush()   

               if collected == attempts:
                  collected = 0

               time.sleep(0.01) # set the inveral to read next

	    except:
               print "exception"                

Wie bekommt man nun diese Koordinaten auf unseren Arduino Leonardo?
Standard wäre die USB Serial-Schnittstelle (COM-Port).
Aber ein Arduino Leonardo ist mit seinem einzigen USB bereits an der PS4!
Somit ist der Weg PC->USB->Arduino nicht benutzbar.
Oder wir benutzen einen zweiten Arduino, [oder einfaches USB Device wie zBsp…],
um dann per RX/TX zu unserem PS4-Arduino zu kommunizieren.
PC->USB->Arduino_A->SerialRxTx->Arduino_B->USB->PS4
recht komplex.. es geht auch einfacher.

Besser und wahrscheinlich günstigste Lösung wäre:
http://arduino.cc/en/Main/USBSerial
http://www.dfrobot.com/index.php?route=product/product&product_id=581
http://www.komputer.de/zen/index.php?main_page=product_info&products_id=319
USBSerial kostet ca. 12,- Euro
PC->USB->USBSerial Adapter->SerialRxTx->ArduinoLeonardo->USB->PS4

Achtung, wichtig:
Nicht vergessen neben Rx, Tx immer auch GND zwischen
USB Serial Light Adapter und Arduino zu verbinden!
Dann klappts auch mit Serial1.

„Serial“ ist die Serielle Schnittstelle am Arduino Leonardo USB Stecker.
„Serial1“ ist die Serielle Schnittstelle am Arduino Leonardo Pins 0(Rx)/1(Tx).

folgendes Sketch, das ich auf den Arduino Leonardo aufspiele,
dient zum testen der Seriellen Schnittstellen.

int my_char = 0;
int my_charB = 0;



void setup() {                
  // initialize the digital pin as an output.
  // Pin 13 has an LED connected on most Arduino boards:
  pinMode(13, OUTPUT);  


  Serial.begin(9600); 
  while (!Serial) {
    ; // wait for serial port to connect. Needed for Leonardo only
  }
  
  
  Serial1.begin(9600); 
  while (!Serial1) {
    ; // wait for serial port to connect. Needed for Leonardo only
  }
  
  
}

void loop() {
 

  
    while (Serial.available() > 0) {
       /* CODE */
       my_char = Serial.read(); // the first byte of incoming serial data available (or -1 if no data is available) - int
       //my_char = "Serial 0: " + my_char;
       
       
       Serial.print( "Serial 0: " );
       
       // Writes binary data to the serial port. 
       // This data is sent as a byte or series of bytes; 
       // to send the characters representing the digits of a number use the print() function instead.
       Serial.write( my_char ); // ausgabe des eingegebenen Zeichens "r"
       
       
       // Prints data to the serial port as human-readable ASCII text 
       // followed by a carriage return character (ASCII 13, or '\r') 
       // and a newline character (ASCII 10, or '\n'). 
       // This command takes the same forms as Serial.print().
       Serial.println( my_char ); // ausgabe des Int-Values des Zeichens "114" + NewLine
       
       delay(5);
       blink_once();
     }
     
     
    
    while (Serial1.available() > 0) {
      
      my_charB = Serial1.read();
      
      Serial1.print( "Serial 1: " );
      Serial1.write( my_charB );
      Serial1.println(  my_charB );
      
      delay(5);
      blink_once();      
    }
     
}



void blink_once(){

  digitalWrite(13, HIGH);   // set the LED on
  delay(1000);              // wait for a second
  digitalWrite(13, LOW);    // set the LED off
  
}

Mit folgenden Zeilen bekommt man jederzeit
die aktuellen Details der COM Ports angezeigt..
Das ist sehr nützlich um zu erkennen,
an welchem COM Port gerade der Arduino Leonardo
oder/und der USB Serial Adapter(Arduino UNO) hängt.

python -m serial.tools.list_ports -v
PAUSE

erzeugt folgende Ausgabe:

COM14
desc: Arduino Uno (COM14)
hwid: USB VID:PID=2341:0001 SNR=55330343831351D03232
COM20
desc: Arduino Leonardo (COM20)
hwid: USB VID:PID=2341:8036
2 ports found

etzt probieren wir noch
das Zusammenspiel zwischen Python und Arduino.

Dazu installieren wir noch PySerial
http://pyserial.sourceforge.net/

nun können wir mit folgendem py-script
Daten über USB Serial Adapter(ArduinoUNO) zum Arduino Leonardo senden
und empfangen sogleich sein Antwort.

#  python C:\python_scripts\py_serial\py_serial.py


import serial
import time
import os

"""
C:\Users\MrRonsen>python -m serial.tools.list_ports -a
Usage: list_ports.py [options] []

list_ports.py: error: no such option: -a
"""
# C:\Python27\Lib\site-packages\serial\tools\list_ports.py  <<< look for options.. discoverd -v !
write_to_path = "C:\\python_scripts\\py_serial\\actual_com_ports.txt"
os.system("python -m serial.tools.list_ports -v > " + write_to_path)

fr = open( write_to_path , "rb")
frc = fr.read()
fr.close()

"""
COM14               
    desc: Arduino Uno (COM14)
    hwid: USB VID:PID=2341:0001 SNR=55330343831351D03232
COM9                
    desc: Arduino Leonardo (COM9)
    hwid: USB VID:PID=2341:8036
2 ports found
"""

frc = str(frc).split("\n")

com_dict = {}
COM_id = ""
for line in frc:
   #print line
   if line[0:3] == "COM":
		
		 COM_id = line.strip()
		 #print "COM_id: ",COM_id
		 com_dict[COM_id]  =  ""
		 
   if line.find("desc:") != -1:
		line2 = line
		line2 = line2.replace("desc: ","")
		line2 = line2.replace("("+COM_id+")","")
		line2 = line2.replace("\t","")
		line2 = line2.strip()
		#print "line2: >>",line2,"<<"
		com_dict[COM_id] = line2
	  
	  
arduino_leonardo_COMport = ""
arduino_uno_COMport = ""

for e in com_dict:
		print e, ": ", com_dict[e]
		
		if com_dict[e].lower().find("leonardo")!=-1:
			arduino_leonardo_COMport = e
		if com_dict[e].lower().find("uno")!=-1:
			arduino_uno_COMport = e

print "arduino_uno_COMport: ",arduino_uno_COMport
print "arduino_leonardo_COMport: ",arduino_leonardo_COMport




if 1==1:

	#COMport = 14
	#serial_port = COMport-1
	#serial_port = "COM14"
	#serial_port = "COM11"
	#serial_port = arduino_leonardo_COMport
	serial_port = arduino_uno_COMport

	baudrate = 9600

	connected = False

	
	"""
	ser = serial.Serial(port=serial_port,
baudrate=baudrate,
parity=serial.PARITY_NONE,
stopbits=serial.STOPBITS_TWO,
bytesize=serial.EIGHTBITS,
timeout=0,
xonxoff=0,
rtscts=0)
        """
        
        
	ser = serial.Serial( serial_port, baudrate)  # open first serial port
	#ser.DtrEnable = "true";
	
	print("initialising")
	time.sleep(2) # waiting the initialization...

	print ser.name          # check which port was really used

	i=0
	
	while i >" + str(xValue) + "<<" )
			
		time.sleep(2) # waits for 2 second

	 
	if 1==1111:
		print "reading: "
		while 1:
			xValue = ser.readline()
			print("something: " + str(xValue) )
	 
	print "END"



	ser.close()             # close port


Wo stehen wir jetzt?
Serial: Arduino Leonardo MouseEmulation an PS4 USB – WORKS
Serial1: senden/empfangen von Daten zw. Arduino Leonardo und PC mittels Arduino USB Serial Light Adapter mittels Python – WORKS
Python Script nutzen, um mit Arduino zu kommunizieren – WORKS
next steps:
– PS3 Eye Toy modifizieren
– HeadTracking Software Schnittstellen anzapfen, MouseEmulation->Mouse auslesen oder HeadTracking Schnittstellen per IP:PORT stream lesen
– alternativ Bildberechnungen bzgl. HeadTracking mit OpenCV/simpleCV selbst umsetzen

erste Tests mit FreeTrack:
ohne Zubehör wie den 3-Punkt Gestellen,
bleibt uns nur das FaceTracking, Gesichtserkennung.
Das funktioniert in ersten einfachen Tests ganz gut.
FreeTrack bietet verschiedene Schnittstellen/Protokolle an,
um die anfallenden Daten bereitzustellen.
Zudem kann man die Mouse steuern lassen, das klappt aber tatsächlich NICHT,
zumindest nicht bei mir, die Mouse sitzt in der Mitte des Bildschirms und flackert hin und wieder ganz links auf dem Bildschirm auf, keinerlei hoch/runter Mouse-Bewegung.

Mit der Software „eViacam“
http://eviacam.sourceforge.net/
kann man per FaceTracking die PC Mouse steuern.
Das funktioniert ganz gut, das FaceTracking und Mouse steuern funktioniert aber leider nicht 100prozentig.. es gibt einen Drift! Alle Kalibrierungsversuche verliefen negativ.

Offenbar müssen wir selbst ran, die Bildberechnungen selbst anstellen.

PS3 Eye – Camera,
funktioniert nicht sofort mit Win7/32bit.
Den Treiber dafür erhält man auf
www.codelaboratories.com/downloads/
für 2,99 $ (2,60 Euro)..

Mit einem PythonScript und den Frameworks/Libraries SimpleCV/OpenCV
kann man die Camera Settings nicht einfach lesen/schreiben,
wie es bei anderen Cams der Fall ist,
aber man kann mit SimpleCV die Camera starten,
wenn man nur Camera(Number) benutzt, OHNE Settings mitzugeben!

Um die PS3 Eye – Camera
in FreeTrack benutzen zu können,
muss man einen Fix/Hack einspielen
www.maximumpc.com/article/how-tos/how_build_your_own_ir_head_tracker
www.maximumpc.com/article/how-tos/how_build_your_own_ir_head_tracker?page=0,1
www.forum.free-track.net/index.php?showtopic=2416
www.sendspace.com/file/2stcof
also Freetrack deinstallieren,
Freetrack neu installieren OHNE Freetrack zu starten,
dann die Hackfiles in Freetrack Ordner kopieren,
dann Freetrack starten.
Wenn man sofort auf den Button „Camera“ klickt friert Freetrack ein!!!
Daher zuvor den Button „Stream“ klicken, hier kann man Auflösung und FPS auswählen,
dann den Button „Camera“ klicken, voila… nun funktionierts.

Vom Kollegen habe ich seine 3 Punkt Infrared BaseCap zum Testen erhalten.
Ohne Modifikationen an der PS3 Cam war von den InfraRed LEDs nichts zusehen.
Daher habe ich als nächstes anhand von Anleitungen, die PS3 Cam auseinandergebaut, die IR-blocking Linse entfernt und von einer Diskette einen Schnipsel zurecht geschnitten und eingesetzt um das Tageslicht zu filtern.
Nach dieser Modifikation sind die IR-LEDs der BaseCap sehr deutlich zu sehen.
www.youtube.com/watch?v=7jJfuP7YgPA
www.youtube.com/watch?v=CbMqsrm2TTs

Nachdem ich lange nach Beschreibungen der FreeTrack Schnittstellen gesucht und nichts gefunden habe,
werde ich den weiteren Weg mit OpenCV/SimpleCV gehen.

Fazit:
das nächste Puzzleteil ist also
eine eigene Software/Python-Script zu schreiben,
das die Kopfposition/ausrichtung anhand der IR-LEDs
aus den Kamerabildern/stream auswertet (SimpleCV, grayscale, binarize, findBlobs,..)

#############################

PC / Laptop mit Windows sollte man bereits haben und verwenden können.

Kosten:
Arduino Leonardo – 25,- Euro
wer noch keine Camera hat müsste sich eine zulegen, vielleicht
liegt ja noch eine benutztbare im Keller..
PS3 Eye Cam – 40,- Euro (sehr hohe Framerate, IR Infrarot Filter muss hier noch entfernt werden)
Infrarot-LEDs + Widerstände + AA-Batteriehalter   – ca. 10,- Euro

PS:

„TrackIR kann man nicht mit der PS4 verwenden“
das ist ein Zitat/Antwort vom deutschen Distributor 2connect.
Es braucht immer die PC Software.
www.trackir.eu/shop/
Zudem sind die Daten verschlüsselt…
man kann die anfallenden Daten also nicht verwenden.
www.en.wikipedia.org/wiki/TrackIR

SimpleCV
www.simplecv.org/
www.simplecv.org/book Page:206
ein erstaunlich kleines Script zur Gesichtserkennung, aber ob wir die wirklich einsetzen werden….
Tip: für bessere Performance kann man die Kameraauflösung herabsetzen
cam = Camera(0, {„width“: 320, „height“: 240})

 

 

das folgende Script ist ein Arduino-Sketch,
http://arduino.cc/en/Tutorial/ButtonMouseControl

/*
mouse steuerung,
parallel i2c LCD shield showing "clicked"
and sending "clicked" over Serial to PC... reading with Serial Monitor
and sending PC keyboard pressed key over serial to Arduino, Arduino reads and 
displays int value of pressed key on LCD and back to PC for reading on Serial Monitor
*/

/*
  ButtonMouseControl
 
 Controls the mouse from five pushbuttons on an Arduino Leonardo or Micro.
 
 Hardware:
 * 5 pushbuttons attached to D2, D3, D4, D5, D6  
!!! CHANGED  D2 to D8 and D3 to D9 for using I2C-LCD on Leonardo !!!
 
 
 The mouse movement is always relative. This sketch reads 
 four pushbuttons, and uses them to set the movement of the mouse.
 
 WARNING:  When you use the Mouse.move() command, the Arduino takes
 over your mouse!  Make sure you have control before you use the mouse commands.
 
 created 15 Mar 2012
 modified 27 Mar 2012
 by Tom Igoe
 
 this code is in the public domain
 
 */

#include 
#include 
LiquidCrystal_I2C lcd(0x27,16,2);  // set the LCD address to 0x27 for a 16 chars and 2 line display



// set pin numbers for the five buttons:
const int upButton = 8;  // 2   
const int downButton = 9; // 3        
const int leftButton = 4;
const int rightButton = 5;
const int mouseButton = 6;

int range = 5;              // output range of X or Y movement; affects movement speed
int responseDelay = 10;     // response delay of the mouse, in ms

int incomingByte = 0;


void setup() {
  
  Serial.begin(9600);

  //display  
  lcd.init();                      // initialize the lcd 
  lcd.backlight();
  lcd.home();
  
  lcd.setCursor(0, 0);
  lcd.print("Hello world...");
  lcd.setCursor(0, 1);
  lcd.print("dfrobot.com");
  
  
  // initialize the buttons' inputs:
  pinMode(upButton,    INPUT);       
  pinMode(downButton,  INPUT);       
  pinMode(leftButton,  INPUT);       
  pinMode(rightButton, INPUT);       
  pinMode(mouseButton, INPUT);
  // initialize mouse control:
  Mouse.begin();
}

void loop() {
  // read the buttons:
  int upState    = digitalRead(upButton);
  int downState  = digitalRead(downButton);
  int rightState = digitalRead(rightButton);
  int leftState  = digitalRead(leftButton);
  int clickState = digitalRead(mouseButton);

  // calculate the movement distance based on the button states:
  int  xDistance = (leftState - rightState)*range;
  int  yDistance = (upState - downState)*range;

  // if X or Y is non-zero, move:
  if ((xDistance != 0) || (yDistance != 0)) {
    Mouse.move(xDistance, yDistance, 0);
  }

  // if the mouse button is pressed:
  if (clickState == HIGH) {
    // if the mouse is not pressed, press it:
    if (!Mouse.isPressed(MOUSE_LEFT)) {
      Mouse.press(MOUSE_LEFT); 
      
      Serial.println("mouse klick");
      
      writeLCD("mouse klick");
      delay(500);
      
    }
  } 
  // else the mouse button is not pressed:
  else {
    // if the mouse is pressed, release it:
    if (Mouse.isPressed(MOUSE_LEFT)) {
      Mouse.release(MOUSE_LEFT); 
    }
  }

  // a delay so the mouse doesn't move too fast:
  delay(responseDelay);
  
 
  
  // send data only when you receive data:
  if (Serial.available() > 0) {
     // read the incoming byte:
     incomingByte = Serial.read();

     // say what you got:
     Serial.print("I received: ");
     Serial.println(incomingByte, DEC);
     writeLCD(String(incomingByte));
     
  } // EOF serial available
        
}


void writeLCD(String input){
  
      lcd.clear();
      lcd.setCursor(0, 0);
      lcd.print( input );    

}

War Thunder PS4 – Saitek X55

works great, with the PS4
better than the previous thrustmaster force feedback joystick.

but PS4 is not always recognizing the axis of the devices,
throttle has an own usb, joystick has an own usb.
so at the beginning of gaming you need to set the axis for throttle, gearing, rolling, …
but a good tip could be: plug out the devices before you start the PS4,
when you are in the game in the control options, then plug in the joystick at first… when this has the axis with the lower IDs, after that plug in the Throttle, when the throttle has the higher axis IDs.
I will find out.. or Gaijin GameDeveloper could fix this next time?
The Buttons seems to stay correct, axis are lost.
Normaly a device has an own USB ID? Which is a real identifier not only a number of the n-th plugged in usb device. Then the game should/could store the information, regardless at which usb port the devices are plugged in, but maybe this time its on Gaijins implementation.

how to change the spring of the joystick

good: many, many buttons…
crazy possibilities to programm all these buttons/axis

when you attach a command to a keyboard keystroke, then you can programm the joysticks buttons/axis to immitate the keyboard keys!
you can programm axis in a special style: divide up the full range of an axle in two, three, four or more separate parts.. which you can bind to a key…

but i have to „screw“ to stick and throttle to the desk,
especially the throttle, when its used in „strong“ manner, when you pull back to you, the throttle could pullout from desk.

War Thunder PS4 – seeing is winning

to get better in the game
its important to see the other teammates and your opponents.

Head Tracking feature would quite good, if it would work perfectly,
what is actualy not the case.

So which option are left?

i use a mouse (usb,optical,Razer), directly connected to an usb hub, which is connected to the PS4.
http://www.amazon.de/PlayStation-USB-Hub-5-fach-Design-passend/dp/B00JQJKYT8/ref=sr_1_1?s=videogames&ie=UTF8&qid=1420564727&sr=1-1&keywords=PlayStation+4+USB-Hub+5-fach

With the mouse i can look around quite fast and comfortable, left, right, top, bottom…
and very important, with the mouse left click, i can recenter instantly!

One thing is not so good,
for a mouse you need some place on the desk to move the mouse around.

So you can think about to test a real 3D Mouse like this:
http://www.3dconnexion.de/products/what-is-a-3d-mouse.html
http://www.3dconnexion.de/products/spacenavigator.html << http://www.rakuten.de/produkt/3dconnexion-spacenavigator-standard-edition-1045245985.html?portaldv=3&sclid=a_rakneu_pla_DE&cid=google_base_pla&utm_source=gpla_DE&utm_medium=pla&utm_campaign=googlebase-pla-2012-02_DE&gclid=COCkmtTx_8ICFSdn7Aod0h0AFw use hot key to: - look back - look to bottom ground - follow your actual tagged opponent