Sunday, June 23, 2013

Emulating touchscreen interaction with sendevent in Android

I know this has been explained many times before. But I keep getting asked  about it over and over again. So I decided to summarize it in one place I could refer to in the future.


Android uses Linux kernel so it processes input events the same way as any other linux system. In case of touchscreen displays it uses the same multi-touch protocol. There are 2 different versions of the protocol - Type A and Type B. Usually it does not matter which protocol to use for injecting events since the kernel driver supports both protocols at the same time. In my automation scripts I use Type A just because I tried it first and it worked right away and I did not have to try Type B.

Android provides these two convenient tools for dealing with input events:

  • getevent - for dumping input events and providing information about input devices
  • sendevent - for injecting input events

Every sendevent command requires 4 parameters:
  • device_name (string)
  • event_type (decimal int)
  • event_code (decimal int)
  • value (decimal int)

First you need to find the name of touchscreen device. The following command requires busybox to be installed:

getevent -pl | busybox sed -e ':a;N;$!ba;s/\n / /g' | busybox awk '/ABS_MT_TOUCH/{print $4}'



Let's assume it printed out "/dev/input/event0" - this would be the first parameter.

For touch events only 2 event types are used:
  • EV_ABS (3)
  • EV_SYN (0)

Touching the display (in case of Type A protocol) will result in an input report (sequence of input events) containing the following event codes:
  • ABS_MT_TRACKING_ID (57) - ID of the touch (important for multi-touch reports)
  • ABS_MT_POSITION_X (53) - x coordinate of the touch
  • ABS_MT_POSITION_Y (54) - y coordinate of the touch
  • ABS_MT_TOUCH_MAJOR (48) - basically width of your finger tip in pixels
  • ABS_MT_PRESSURE (58) - pressure of the touch
  • SYN_MT_REPORT (2) - end of separate touch data
  • SYN_REPORT (0) - end of report

Let's say we want to emulate a touch down event at the point with coordinates x=300, y=400. We will need to execute the following sendevent commands:

sendevent /dev/input/event0 3 57 0
sendevent /dev/input/event0 3 53 300
sendevent /dev/input/event0 3 54 400
sendevent /dev/input/event0 3 48 5
sendevent /dev/input/event0 3 58 50
sendevent /dev/input/event0 0 2 0
sendevent /dev/input/event0 0 0 0

I used arbitrary values of 5 for the ABS_MT_TOUCH_MAJOR (makes for very small finger tip for high precision) and 50 for the ABS_MT_PRESSURE (a slight tap) which work good enough for most applications.

For most touch screens on the market it takes 20 to 50 milliseconds to reliably register the touch. So I would recommend to wait at least 50 milliseconds before sending the release event report. If you want to emulate the long touch - then you wait longer. You can use busybox usleep command:

busybox usleep 50000



The release report is really simple. To let the input device know that all previous touches have been released - you just send the empty report with ABS_MT_TRACKING_ID = -1:

  • ABS_MT_TRACKING_ID (57)
  • SYN_MT_REPORT (2)
  • SYN_REPORT (0)

sendevent /dev/input/event0 3 57 -1
sendevent /dev/input/event0 0 2 0
sendevent /dev/input/event0 0 0 0



This is how injecting touch events could be implemented in python:


4 comments:

  1. At least on my Motorola Defy with CM7 (Android 2.3.7) things look a bit different.
    It looks like
    0 2 0
    0 0 0
    is NOT a release report, but some kind of activation of the previous sendevent commands.
    To release you have to send an additional command (including activation):
    sendevent /dev/input/event3 3 57 -1
    sendevent /dev/input/event3 0 2 0
    sendevent /dev/input/event3 0 0 0
    As you can see, the device name is event3 on the Defy.

    ReplyDelete
    Replies
    1. I finally got a device which did not work without providing the ABS_MT_TRACKING_ID in the release report - I added that event into the script

      Delete
  2. Hmm I can't get this to work. Afterwards no real touch events are accepted anymore and I have to reboot the phone. Even with the 3 57 -1 before the release, it doesn't work. getevent shows that the parameters are in hex values. Could this be necessary?

    ReplyDelete
    Replies
    1. Here is the output of getevent when I just tap the screen:

      /dev/input/event2: 0003 0039 0000000e
      /dev/input/event2: 0003 0035 000003a7
      /dev/input/event2: 0003 0036 00000499
      /dev/input/event2: 0003 003a 0000002e
      /dev/input/event2: 0003 0030 00000002
      /dev/input/event2: 0000 0000 00000000
      /dev/input/event2: 0003 0035 000003a3
      /dev/input/event2: 0003 0036 00000498
      /dev/input/event2: 0003 003a 00000031
      /dev/input/event2: 0000 0000 00000000
      /dev/input/event2: 0003 0039 ffffffff
      /dev/input/event2: 0000 0000 00000000

      The 0 2 0 seems to be missing. Is this important? And what about the base of the numbers? getevent shows hexadecimal numbers, shouldn't I also use hex on sendevent?

      Delete