Importance of Automated Softwate Testing

Posted by Zhimin Zhan on August 18, 2010|Comments|Read full article

The election campaign is reaching the climax in Australia (election day on this Saturday). Last night, one voter asked Prime Minster a question of impact of troubled QLD health payroll system on upcoming E-Health system. In Queensland, this yet another IT project failure attracts a lot political attention, just have a look at news headlines below:

QLD health payroll failure is after similar one in the same city: Brisbane City Council to dump troubled payroll (this may not be done IBM though). And surprisingly, in this IT world, reputation or recent track records seems doesn't matter much, Gold Coast City Council (80KM from Brisbane) pays $55m to health payroll firm.

Globally, the biggest IT disaster believed to be UK NHS (based on this ZdNet article): : "experts have warned that the project could actually cost more than £30 billion, making it the greatest IT disaster in history."

The especially sad part of all this is that while these big companies walked away with millions, we as tax payers are ultimate victims.

Auditor-General's report on QLD Health Payroll: Pay system not properly tested: report.

Once one asked me, "What didn't they just fix the system? Given all the attentions (including demonstrations), 3 months is a long time, isn't it? I saw you fixing defects in our system very quickly."

My answer: "If they don't have automated regression testing in place (most likely not), fixing a defect may break other functions. Manual testing is not good finding out that quickly. Before I check-in fixes, I run some related regression test suites in TestWise, and our continuous build server runs full regression tests every day. That's why Steve McConnell says 'The only practical way to manage regression testing is to automate it'"

So will our elected politicians learn the lesson? Let's hope.

Watir-WebDriver support in TestWise2

Posted by Zhimin Zhan on July 27, 2010|Comments|Read full article

Selenium and Watir, as we know, are the two most popular open-source testing frameworks. The next big thing is WebDriver, which is now merged with Selenium 2. Thanks Jari Bakken (who is also the creator of Celerity), we have Ruby bindings for Selenium 2 and even Watir wrapper of it: watir-webdriver, which some might refer it as Watir 2.0. With watir-webdriver, users can pretty much reuse existing Watir scripts, with minor changes.

What are the benefits of using watir-webdriver comparing existing Watir 1.6.5?

  • Run same test scripts in IE, Firefox, Google Chrome and HTMLUnit (headless)
  • No JSSH plugin required for running tests in Firefox
  • Synergy of combination of Selenium and Watir

In last appearance at Watir Podcast, I said future TestWise versions will support WebDriver. Now you can try web-driver in IDE!

  1. Download TestWise-2.0a1-setup.exe and install
  2. In TestWise, open sample project c:\program files\testwise\samples\demo\demo.tpr (if you are using TestWise first, it shall be pre-loaded)
  3. Select the first test script file (left mouse click the file name on the left): ajax_rwebspec_spec.rb,
    choose your browser (IE, Firefox or Chrome), then run it
     

Our initial impression with Watir-WebDriver is good and very promising, with a few issues currently:
  • Can't reuse existing browser window, has to start up a new one
  • Test execution is slower than Watir 1.6
  • Opening browser operation seems quite CPU intensive

Please note that both Selenium 2 and TestWise2 are at early development stage, so please use Watir 1.6.5 and TestWise 1.9 for production use.


Test sites with native Windows authentication

Posted by Zhimin Zhan on July 14, 2010|Comments|Read full article

For some sites (especially in .NET world), users are presented a native windows' authentication window (like below) before can access to it.

The standard open_browser in Watir won't work (get stuck), as it will be waiting response from the browser.

  open_browser("http://secure.site.com") # will stuck 

To add a bit more complexity, we expect the same tests to pass whether user has already logged in or not.

The solution is:

  • Detecting Login Window using Ruby's timeout
  • Use AutoIt to fill user name and password in native Windows Login Window.

    begin
      Timeout::timeout(5) {
        open_browser("http://secure.site.com")
      }
    rescue Timeout::Error => e
      # debug "Timeout error on get to site, maybe asking for login"
      autoit = WIN32OLE.new('AutoItX3.Control')
      win_title = "Connect to edam.bluefirems.com.au"
      ret = autoit.WinWait(win_title, '', 10)
      if ret
        autoit.ControlSetText(win_title, "", "Edit2", username)
        autoit.ControlSetText(win_title, "", "Edit3", password)
        autoit.ControlClick(win_title, "", "Button3")
      end
   end

The above test scripts will work, but not readable. By applying 'Extract Function' and "Move to Helper" Refactorings in TestWise, we get:

    # ...
    rescue Timeout::Error => e
      # debug "Timeout error on get to site, maybe asking for login"
      # calling helper function in test_helper.rb
       
      login_as("zhimin@secure.site.com", "Secret")    
    end