Saturday, March 28, 2015

iOS App Testing



                            

iPhone/iPad features


   If you are completely new to iPhone testing and haven't used iPhone before then you should get yourself familiarize with iPhone/iPad features.
    Watch  youtube videos on iPhone/iPad/iOS features so that you get to know about physical features(like screen size, resolution etc) and phone features like how to set pass code on lockscreen, How to download Apps, Know about Touch ID,Control Panel, Notification Center,Multitasking, Siri, Spotlight Search,  Move & Delete Apps, Stock Apps (Built in Apps), Settings etc.
    You should also watch videos on iOS 7 and iOS 8 features and what are the new features available on iOS 8 from youtube and most importantly try out all the functionality on actual phone itself..


Screen Size, Aspect Ration, Resolutions:

iphone 6 plus: Screen Size: 5.5 inches, Resolution: 1920x1080, 401ppi, Retina HD Display
iPhone 6: Screen Size:4.7 inches, Resolution: 1334x750, 326ppi, Retina HD Display
iPhone 5s: Screen Size:4 inches, Resolution: 1136x640, 326ppi

iPhone  6 plus


1.iPhone 6 plus, 64 bit architecture, A8 chip, M8 Motion Coprocessor
2. Faster than iPhone 5s in terms of processing power and graphics processing
3. iOS 8 preinstalled.
4. Some default app shows more information on the screen, e,g ability to show pictures against all messages in messages app.
5. 8 MP iSignt camera, True time flash, 1080p HD video at 60fps


Please check out following links for more details:

  •     http://en.wikipedia.org/wiki/IPhone
  •     https://www.youtube.com/watch?v=zHwkJJ3Vmm4  (iPhone 6 & iPhone 6 plus Complete Features Guide)
  •     https://www.youtube.com/watch?v=uWxwig4Tr2o (Top 20 iOS 8 features)
  •     http://www.phonearena.com/phones/compare/Apple-iPhone-6,Apple-iPhone-5s/phones/8346,7710

iOS Human Interface Guidelines


Apple has publishes iOS Human Interface Guidelines. The guidelines are Apple's recommendations and suggestions to developers for designing cohesive and usable apps that follow similar user interface principles to other iOS applications.  As a tester you should understand whether application is following apple recommended user interface guidelines. You can also suggest changes to the mock-ups/wire-frames of your application if needed.

Basic understanding of iOS application stack


  •     iOS is the opertaing system that runs on iPad, iPhone and iPod touch devices.
  •     The iOS SDK contains tools and interfaces needed to develop, install, run and test native apps.
  •     Native apps are build using Objective C and iOS system framework
  •     Consists of four Abstraction layers • Core OS Layer • Core service Layer • The Media Layer • The Cocoa Touch Layer
  •     Uses Darwin foundation and therefore a unix like OS
  •     XCode is a suite of tools used for developing software for both Mac OS and iOS

Certificates, App ID and Provisioning Profile


   If you want to run your app on iOS simulator then you do not need provisioning profiles. But if you want to run your app on a device, you need to create a provisioning profile.

   A provisioning profile is a collection of digital entities that uniquely ties developers and devices to an authorized iPhone Development Team and enables a device to be used for testing.So basically a provisioning profile says this particular app (app ID) can be install on this particular device (device ID) by a particular developer/development team (developer certificate). In other words, it verify whether an application is permitted to run on a specific device.

It mainly involves following steps

  •     Join iOS Developer Program
  •     Create a Certificate Signing Request (after logging into app developer portal)
  •     Create a Development Certificate ,download and add it to your login keychain.
  •     Adding Device/Devices (using device's UDID/Serial number)
  •     Create an App ID(An app ID is an identifier that uniquely identifies an application on iOS) and add it to XCode
  •     Create a Provisioning Profile
  •     Configuring the Project(XCode Build Settings=> Code signing authority)
  •     Create archive (IPA package) for distribution or build=>run directly on connected iPhone

In most of the cases, the company you are working for,  will join iOS developer program and will create team provisioning profile. This  profile allows all your apps to be signed and run by all team members on all their devices. (provides those devices and apps are added in the profile)

For more details, please refer:

  • https://developer.apple.com/library/mac/documentation/IDEs/Conceptual/AppStoreDistributionTutorial/AppStoreDistributionTutorial.pdf
  • http://escoz.com/blog/demystifying-ios-certificates-and-provisioning-files/
  • http://code.tutsplus.com/tutorials/how-to-test-your-app-on-an-ios-device--mobile-13861

Setup iOS Development Environment and compile an app

  • Setup Development Environment: Download iOS 8 SDK and Xcode 6 from the Mac App Store and install.
  • Check out your source code from the report. You can do it from command line or you can use any UI based client e.g. Sourcetree.
  • Build your source code and create IPA package.
You may also want to setup continuous integration system and finally integrate it with automation system.
“Continuous Integration is a software development practice where members of a team integrate their work frequently, usually each person integrates at least daily - leading to multiple integrations per day. Each integration is verified by an automated build (including test) to detect integration errors as quickly as possible.” Martin Fowler,

There can be two approaches
1) Continuous Integration Process by setting up Mac OS X Server
2)  Using Jenkins + GIT Plugin + XCode Plugin

Approach 1: Continuous Integration Process using Mac OS X Server

  • Install OS X Server on OS X Mavericks from the Mac App Store
  • Start XCode service from Server app ( It will advertise on the network using Bonjour discovery.)
  • Add server to an Apple Developer Program Member Center team
  • Add an iOS development device to the Xcode service (Connect device and it should be provisioned : Provisioning Your iOS Device for Development)
  • Add  OS X Server account to Xcode on a development Mac
  • Set Up the Xcode Service for Team Members(Add user account to the server, change permissions for who can create or view bots)
  • Enable Access to Your Source Code Repositories(Connect to remote GIT) (in XCOde server: repositories tab)
  • Configure Bots to Perform Continuous Integration (Share build scheme and create Bots)
  • Manage Bots in the Log Navigator
  • Review Integration Details in the Integration Viewer
  • Manage and Monitor Bots from a Web Browser (5 tabs => Summary, Tests, Commits, Logs, Archives (*.ipa))

Approach 2: Using Jenkins

  1. Install JDK 1.7 on Mac OS X
  2. Install Jenkins (Custom installation: Deselect 'Start at boots as Jenkins)
  3. Start Jenkins (Application -> Jenkins.war)
  4. Go to http://localhost:8080 in browser.
  5. Jenkins=> Manage Jenkins=> Manage Plugins=> Go to Available tab=> Install XCode plugin
  6. Follow the process described here (http://www.erikzaadi.com/2011/09/09/connecting-jenkins-to-self-signed-certificated-servers/). This is to overcome git cloning issue.
  7. Create a new Job
    • Provide project name (select build a free style s/w project)
    • Repository: 
      • Select git:  provide repo UR
      • Credentials: Add your credentials.
      • Branches to build
    • Build Trigger: 
      • We keep it blank as of now. (we can build periodically or based on certain triggers)
    • Add build Step: Select XCode
    • XCode: (Build , signing and creating package)
      • Target: target name
      • Clean before build: Yes
      • Configuration: Release
      • Pack application and build .ipa (Select check box)
      • Code signing and OS X Key chain options (Code signing identity: iOS Team Provisioning Profile:*)
    • XCode: (running unit test)
      • Target: Tests 
      • Configuration:Debug
      • SDK: iphonesimulator
    • Post-build Actions:
    • Publish JUnit test results report:
      • test-reports/*.xml  (Note: Xcode plugin will convert OCUnit console out to Junit XML format.)
    • Upload to TestFairy/iTunesConnect(TestFlight): Provide required auth tokens. (API token, team token)
    • Send email


App Testing


IPhone different versions, models, differences

Functionality testing of app and example test cases

Check similar apps

Install app on iPhone


  1.     Installing from TestFlight/TestFairy/iTunesConnect
  2.     Get IPA file attached in email, available in FTP
  3.     Install from App Store
  4.     Checkout release tag from repository SVN/GIT , build IPA package using XCode and install

·


How to install App(from IPA) on iPhone from Windows 7 ?

  •     Connect iPhone to Win 7 Laptop
  •     Launch iTune
  •     Select your phone, click on 'Apps'
  •     From menu-> 'Add file to Library'
  •     Install App (you may also need to install provisioning profile)

                                                                                    

How to see console logs on Windows m/c ?


  •     Install iPhone configuration utility or iTools. (iTools has lot of additonal features)
  •     Connect Phone using USB cable
  •     Launch iPhone config utility. Go to Devices->Abhijit's phone ->Console

How to collect crash logs ?


    Connect Phone to PC using USB cable

    Crash logs are available at C:\Users\Abhijit\AppData\Roaming\Apple Computer\Logs\CrashReporter\MobileDevice\Abhijit's iPhone
    File name : <app name>_2013-12-19-201433_<Abhijits-iPhone>.crash

    There are other types of logs also available at following locations:
    C:\Users\Abhijit\AppData\Roaming\Apple Computer\Logs\CrashReporter\MobileDevice\Abhijit's iPhone\Baseband
    C:\Users\Abhijit\AppData\Roaming\Apple Computer\Logs\CrashReporter\MobileDevice\Abhijit's iPhone\DiagnosticLogs
    C:\Users\Abhijit\AppData\Roaming\Apple Computer\Logs\CrashReporter\MobileDevice\Abhijit's iPhone\Message

     You can also use iTools to collect crash logs

How to take screenshots using phone and transfer to PC?


  •     Press Home + On/Off Button at the same time.
  •     Connect Phone using USB cable
  •     From explorer browse to phone folder and transfer images (e.g screenshots available in my phone at following location
  •     Computer\Abhijit's iPhone\Internal Storage\DCIM\860OKMZO)

Analyze crash logs with XCode

1) Plug in the device and open Xcode
2) Choose Window -> Devices from the Xcode menu
3) Under the DEVICES section in the left column, choose the device
4) To see crash logs, select the View Device Logs button under the Device Information section on the right hand panel
5) To see the device console, click the up-triangle at the bottom left of the right hand panel to show the device console, then click the down arrow on the bottom right to save the console as a file
6) Find your app in the Process column and select the Crash log to see the contents.
7) Xcode 6 will also list low memory logs here; these will be shown with a Process name "Unknown" and Type "Unknown. You can examine the contents of the log to determine whether any of these are caused by your app. For more information about low memory logs, see Understanding and Analyzing iOS Application Crash Reports


How to capture video of the issue ?

   Sometimes you need to attach video of the issue in the bug report. You can use some other phone to record the video or you can use software like Airserver or Reflector.

Tools


iTools


iTools is a tool to manage all your iOS devices in a very similar way to iTunes, but it adds a few new and interesting options,e.g collecting system logs, crash logs etc.

It is 100% freeware and  you do not need to install onto your PC, it has no ads or plugins in it. You just unzip and directly use it from the folder.

To start using the program you must first connect all of your iOS devices (iPhone, iPod, iPod Touch, iPad) to your PC.

   You can collect system logs(console logs) and Crash logs easily.

Note : you can also use iTools to record screencast and take screenshot though I have not tried this feature.


iPhone Configuration Utility


To access iOS Console Logs you can also use iPhone Configuration Utility.

Once you have installed and started the iPhone Configuration Utility, connect your iOS device into your computer. In the sidebar on the left, select the device you are testing with; then click on the Console tab.  This will show you the iOS system console log.  There is an option below the displayed log to save it as text file.


Test for Compatibility

iOS 8 supports iPhone 4s or later, iPod touch (5th gen), and iPad 2 or later. New apps and app updates submitted to the App Store should support iOS 8 and be optimized for iOS devices with Retina display.
App Review Guidelines and App Store Release

Please read the details
https://developer.apple.com/app-store/review/guidelines/


In my upcoming blogs, I will try to cover following topics
  • Testing with the XCTest Framework
  • UI automation using  Instrumentation
  • UI automation: Commercial tools: like Ranorex , TPLAN-ROBOT, Eggplant
  • UI automation: Opensource tools: Appium
  • Battery and data consumptions testing
Please stay tuned.
Posted by at

1 comment:

Subscribe to: Post Comments (Atom)