How To Run Compatibility Test Suite (CTS) ?



How does the CTS work?

The CTS is an automated testing harness that includes two major software components:

The CTS test harness runs on your desktop machine and manages test execution.

Individual test cases are executed on attached mobile devices or on an emulator. The test cases are written in Java as JUnit tests and packaged as Android .apk files to run on the actual device target.

Pre-Requirements :

  1. Linux System 
  2. Mobile or Emulator with User Build.
  3. Download latest CTS.
  4. Download CTS Media files.
  5. Download Android SDK for Linux and keep it ready. 

Setup the System for CTS:

1.Create 3 folders in your system root directory. 

  1. CTS
  2. CTS_Media
  3. ADT

2. Extract and Copy downloaded CTS files into the CTS folder which you have created now . It will have below folder structure 
    
     android-cts
               repository
         plans
                      results
         testcases
                      host_config.xml
      tools
3. Extract and Copy downloaded CTS Media files into the CTS_Media folder which you have created now .It will have below folder structure 
      
        1. Change the permission for copy_media.sh file
                $ export chmod 544 copy_media.sh

4.  Extract and Copy downloaded android-adt files into the ADT folder which you have created now .It will have below folder structure 

5. Open Terminal and Set ADT path by running below command form root directory.

    $ export PATH=$PATH:/home/root/ADT/android-sdk-linux/sdk/platform-toots
    Note : Run any adb command to check path is set properly or not. (exp:  adb devices )

Set up the Device/Emulator :

CTS can be executed only on consumer devices. This section is important as not following these instructions will lead to test timeouts/ failures:
1. Your device should be running a user build (Android 4.0 and later)from source.android.com
2. Please refer to this link on the Android developer site and set up your device accordingly.
3. Make sure that your device has been flashed with a user build (Android 4.0 and later) before you run CTS.
4. You need to ensure the Text To Speech files are installed on the device. You can check via Settings > Speech synthesis > Install voice data before running CTS tests. (Note that this assumes you have Android Market installed on the device, if not you will need to install the files manually via adb)
5. Make sure the device has a SD card plugged in and the card is empty. Warning: CTS may modify/erase data on the SD card plugged in to the device.
6. Do a factory data reset on the device (Settings > storage > Factory data reset). Warning: This will erase all user data from the device.
7. Make sure no lock pattern is set on the device (Settings > Security > Screen Lock should be 'None').
8. Make sure the "USB Debugging" development option is checked (Settings > Developer options > USB debugging). Google Confidential
9. Make sure Settings > Developer options > Stay Awake is checked
10. Make sure Settings > Developer options > Allow mock locations is checked
11. Make sure device is connected to a functioning Wi-Fi network (Settings > Wi-Fi)
12. Make sure the device is at the home screen at the start of CTS (Press the home button).
13. While a device is running tests, it must not be used for any other tasks.
14. Do not press any keys on the device while CTS is running. Pressing keys or touching the screen of a test device will interfere with the running tests and may lead to test failures.
15. Set up accessibility tests:
   1. adb install CTS/android-cts/repository/testcases/ CtsDelegatingAccessibilityService.apk
   2.On the device, enable Settings > Accessibility > Delegating Accessibility Service
16. Set up device administration tests: 
  1. adb install CTS/android-cts/repository/testcases/CtsDeviceAdmin.apk
  2. On the device, enable Settings > Security > Device Administrators > android.deviceadmin.cts.CtsDeviceAdmin* settings

17. Copy the CTS media files to your device using ADB with the instructions below:
   The CTS media stress tests require video clips to be on external storage (/sdcard). Most of the clips are from Big Buck Bunny which is copyrighted by the Blender Foundation under the Creative Commons Attribution 3.0license. The required space depends on the maximum video playback resolution supported by the device. By default, 176x144 and 480x360 SHOULD be supported. Note that the video playback capabilities of the device under test will be checked via the android.media.CamcorderProfile APIs.
Here are the storage requirements by maximum video playback resolution:
  • 480x360: 91.4MB
  • 720x480: 151.9MB
  • 1280x720: 401.6MB
  • 920x1080: 1008.2MB

Instructions to copy the media files to a device:
  • To copy clips for just the default resolutions, run ./copy_media.sh. Google Confidential 
  • To copy clips up to a resolution of 720x480, run ./copy_media.sh 720x480. 
  • If you are not sure about the maximum resolution, try 1920x1080 so that all files are copied. ○ If there are multiple devices under adb, add -s serial option to the end. 
Syntax  :  $.<space>./CTS_Media/copy_media.sh <resolution>
Example :  $. ./CTS_Media/copy_media.sh 1920x1080


For example, to copy up to 720x480 to device with serial 1234567, run copy_media.sh 720x480 -s 1234567.
Now , Start Using CTS commands :

1. Run the below command to start cts . 
   Syntax  $./CTS/tools/cts-treadfed run cts --plan <plan-name>
   Command $./CTS/tools/cts-treadfed run cts --plan CTS
    
     Note : It will take around 10 hours to complete test and it will create report in CTS/android-cts/repository/results folder.  To list the reports follow the below step.

2. Run the below command to list results . 
   Command $./CTS/tools/cts-treadfed list results
      Results will be shown as below :


3. Run the below command to continue the session if there is a "NotExecuted" cases  . 
      Syntax   $./CTS/tools/cts-treadfed add derivedplan --p <new-plan-name> --s <session-id> --r notexecuted
   Command $./CTS/tools/cts-treadfed add derivedplan --p CTS-FAIL --s 0 --r notexecuted

4. Run the below command to create new plan with Failed cases . 
      Syntax   $./CTS/tools/cts-treadfed add derivedplan --p <new-plan-name> --s <session-id> --r fail 
   Command $./CTS/tools/cts-treadfed add derivedplan --p CTS-FAIL --s 0 --r fail

5. Run the below command to create new plan with Passed cases . 
       Syntax   $./CTS/tools/cts-treadfed add derivedplan --p <new-plan-name> --s <session-id> --r pass
   Command $./CTS/tools/cts-treadfed add derivedplan --p CTS-FAIL --s 0 --r pass

6. Run the below command to run individual methods in class . 
       Syntax   $./CTS/tools/cts-treadfed run cts --class <class-name> --method <method-name>
      Command $./CTS/tools/cts-treadfed run cts --class android.permission.cts.FileSystemPermissionTest --method testOtherRandomDirectoriesNotWritable
7. Run the below command to run all methods in class . 
       Syntax   $./CTS/tools/cts-treadfed run cts -c <class-name>
       Command $./CTS/tools/cts-treadfed run cts -c android.permission.cts.FileSystemPermissionTest

Comments

  1. Unknown1 July 2017 at 13:11

    hi thank you soo soo much
    your explanation with ex. is very good
    from 1 day i was searching for continue session now i got
    really thanks for your kind sharing

    ReplyDelete
  2. Unknown1 July 2017 at 13:12

    hi brother have you got answer for this ?,
    if got please help me out by sharing

    ReplyDelete
  3. Mahfuz9 July 2017 at 23:56

    Hi
    How to run multiple modules at a time or under one plan? Please explain with example.

    ReplyDelete
  4. Mahfuz9 July 2017 at 23:59

    This comment has been removed by the author.

    ReplyDelete
  5. Android App Development Company in Indore3 November 2017 at 06:43

    Thank for the Post, i am glad you have shared the complete process on how to run compatibility test suite. its an awesome article you are shared.Try to keep boost these types of article patterns.Pretty effective post that competent to me.


    Mobile Application Development Company

    ReplyDelete
  6. Sukesh23 September 2018 at 08:57

    Document is very much appreciated well furnished thanks a lot for such a nice one
    A query below/
    $ export chmod 544 copy_media.sh, when i executed this i could see below error what could be the issue..
    bash: export: `544': not a valid identifier
    bash: export: `copy_media.sh': not a valid identifier


    ReplyDelete
  7. Mounika27 January 2019 at 21:53

    I prefer to study this kind of material. Nicely written information in this post, the quality of content is fine and the conclusion is lovely. Things are very open and intensely clear explanation of issues
    Python Online certification training
    python Training institute in Chennai
    Python training institute in Bangalore

    ReplyDelete
  8. Darshana M4 April 2019 at 04:17

    This comment has been removed by the author.

    ReplyDelete
  9. Keerthana11 May 2020 at 22:16

    lots of solutions are giving Regarding Studies and getting new innovative ideas Through this articles.Thanks for sharing your valuable time with new ideas
    python training in chennai | python training in annanagar | python training in omr | python training in porur | python training in tambaram | python training in velachery

    ReplyDelete
  10. Revathi4 August 2020 at 00:44

    Your good knowledge and kindness in playing with all the pieces were very useful. I don’t know what I would have done if I had not encountered such a step like this.keep it up!!

    android training in chennai

    android online training in chennai

    android training in bangalore

    android training in hyderabad

    android Training in coimbatore

    android training

    android online training

    ReplyDelete

Post a Comment

Popular posts from this blog

Dalvik Virtual machine VS Java Virtual Machine:

Dalvik Virtual machine :          (DVM) is Register Architecture, designed to run on low memory, uses its own byte code and runs .Dex file (Dalvik Executable File) Java Virtual Machine :         (JVM) is Stack Architecture, uses java byte code and runs .class file having JIT. Java  source code is compiled by the Java compiler into .class files. Then the dx (dexer) tool, part of the Android SDK processes the .class files into a proprietary file format called DEX that contains Dalvik bytecode. Difference between .class and .dex file format           ● In DEX all the classes of the application are packed into one file.           ● All th e classes in the same DEX file share field, method, etc if they are same.           ● Classes from the same DEX file are l...
Read more

NFC : Reading a MiFare Classic 1K from Android Devices

Image
Ever since Near Field Communication was embedded on mobile phones, loads of new ideas and business proposals made people very busy. So does the Android platform with its API's supporting NFC. Nexus S looks like a state of the art - good starting point if one wants to get past the monotonic Nokia's piece of the cake. I just want to share with you my experience on reading a MiFare Classic tag using the Nexus S..and the Android platform.  You need to have : A MiFare Classic 1k Tag - ( hopefully you know the keys for its Blocks :=) ) Android SDK and IDE Preferable a Nexus S (Make sure if the Android version is 2.3.3 and above). Some Basics about the card: MiFare classic cards store data in its Sectors. In MiFare classic 1k card there are 16 of them. Each Sector contains 4 blocks.  You can store 16 bytes in each block. Making about 1024 bytes of storage space..that explains the 1K part of the card. You can perform common tasks like reading, writing data on these blocks, authen...
Read more