TestWisely RSS Feed

Verification efforts in total software development cost: more than half!

Sun, 22 Jan 2012 00:00:00 GMT

Verification efforts in total software development cost: more than half!

We all know that testing/verification activities are important in any software project, but how important? Most referenced paper on this topic I could find is B. Hailpern and P. Santhanam's paper on IBM Systems Journal in 2002. They stated:
"In a typical commercial development organization, the cost of providing this assurance via appropriate debugging, testing, and veriﬁcation activities can easily range from 50 to 75 percent of the total development cost." - IBM SYSTEMS JOURNAL, VOL 41, NO 1, 2002

This article 'The Case for Automated Software Testing' by Bernie Gauf and Elfriede Dustin, IDT contains further survey statistics on the time spent on testing out of total software development cycle:

Although the numbers vary, it is safe to say mostly will agree to the ratio in the range of 30%-75%. The average: >50%. Nowdays software are released more frequently, i.e, requires more testing effort. If we take this into consideration, the figure is even higher.

Some might find about more than half effort on testing/verification hard to believe, my observation is in align with the authors above, that's is reality in many software projects. Now I let me explain, verification in software department is not only limited to testers (what's why I used 'verification' instead of 'testing'), most of team members in a software projects doing verification:

Testers test a build against requirements
Programmers verify functions being developed or bug fixes
Business Analysts verify the implementation matches customer's expectation.

Please note that they do these activities pretty much every day.

If your software project is late, over budget and looks like a mission impossible, verification is the first area you shall look into for improvement.

Print User Story Cards in StoryWise

Sat, 31 Dec 2011 00:00:00 GMT

Print User Story Cards in StoryWise

Agile projects, even fake ones (which don't do automated testing and continuous integration at all), use story cards, that is, writing requirements on cards (commonly, index cards used). But the fact is that nowadays most of people's handwriting, like mine, are not very recognisable (a less polite word: sucks). Story cards are conversation starters in agile projects, but only can be effective if team members can read the text on them.

Almost every project team I am involved likes the idea of printing story cards, this can be easily done in StoryWise. After entered or imported user stories in StoryWise, you can generate story card PDF from user story list

or from the story details page

, and you will get

Then you set up your printer with specified paper size 152x102mm to print it out on index cards.

Testing tags not supported by Watir

Tue, 22 Nov 2011 00:00:00 GMT

Testing tags not supported by Watir

Watir supports most HTML tags, and we use them to drive the testing:

link(:text, "some text").click
td(:id, "special_cell").click

How to test tag (such as TH) which are not supported by Watir? You can use xpath to identify the element. Let me illustrate with an example: a tester wants to click the table heading 'LOGIN' on the page below:

It's source:

<th abbr="Login" scope="col">Login</th>

It does not even have a ID! We could use the xpath to find the element: tag is 'TH' and attribute 'abbr' is 'Login'. Here is the test script:

ie.element_by_xpath("//th[@abbr='Login']").click

To verify the solution, I modified the source to show a JavaScript popup window on clicking this table heading. And out test script worked:

Why more Scrum than XP?

Sat, 22 Oct 2011 00:00:00 GMT

Why more Scrum than XP?

Today I picked up Kent Beck 'Extreme Programming' book from the bookshelf, what's a great book! Last time I read it was about 5 years ago. The XP practices now make even more sense to me. However, over this period, I saw more projects branded themselves 'Agile', 'Scrum', few doing XP. Why?

With the help of James Shore's Scrum and XP Practices: Cross Reference, we can do a quick comparision

Stand-Up Meetings XP: Yes, Scrum: Yes
Continuous Integration: XP: implied, Scrum: n/a
Ten-Minute Build XP: Yes, Scrum: n/a
Simple Design XP: Yes, Scrum: n/a
If a programmer thinks it is easy, then he/she is still in "don't know what you don't know" stage.
Pair Programming XP: Yes, Scrum: n/a
Maybe a controvsial one, but there many studies approving pair programming achieve better productity with lower rates
Single Code Base XP, Scrum: n/a
I have seen too many times, several junior level developers (not by age) got excited talking about branches, always end up with disappointments.
Refactoring: a core practice in XP, n/a in Scrum
Note if you don't have unit tests, it is NOT refactoring. What's your project's unit test code coverage?
Customer Tests XP: implied, Scrum: n/a
Getting customer to write/review automated tests prevents miscommunication.
No bugs: XP: Implied, Scrum: n/a

As you can see, XP practices are more real, for my experience, found more used in successfuly projects. Few people claim their doing XP as it is can be easily checked against the above list.

Scrum is, in my opinion, a more abstract version of XP and the word 'sprint' sounds cool. I remembered that in a group session at one conference, one attendant called out "Who else hate Scrum?" For me, what flag you fly up dosn't matter, how you do it matters. Story cards and stand-up meetings don't make projects 'Agile' (I am sure you have seen enough Fake agile projects like me), automated testing (unit and functional UI) and continuous integration do.

Sample projects: RWebSpec, Watir, Selenium, RSpec, and Cucumber

Thu, 15 Sep 2011 00:00:00 GMT

Quite often, we receive the following questions:

Which test framework do you suggest, RWebSpec or Watir or Selenium?
Which BDD framework is good for automated test scripts, xUnit or RSpec or Cucumber?

These kind of questions are not easy to answer, as people have different preferences. Now we created a public GitHub repository, you can try out and compare them yourself:
https://github.com/zhimin/adminwise-ui-tests

A test web site is created for this, you just checkout (or download) the test projects (six of them), open one in TestWise 2.

HP TouchPad marketing irony

Fri, 19 Aug 2011 00:00:00 GMT

HP TouchPad marketing irony

Posted by Zhimin Zhan on August 19, 2011| Comments

Two days ago, I received a marketing email from HP with subject "Be the first to own the HP TouchPad". Inside, "Works Like Nothing Else", "Everybody on HP TouchPad", "There is nothing like it", you may still visit its web version on HP.

Guess how much how much could change in 2 days? Today, 2011-08-19, news come out that "HP will discontinue its TouchPad as well as its WebOS phones". I feel sorry for people who just bought it.

This is just another example of sales/marketing guys will sell you dying animals. It happens so common in our industry that whole project team suffers from some manager/architect's decision on a product (usually expensive), knowing there are much cheaper or free products out there are a lot better. I am sure these managers/architects won't do grocery like this (as that will be own money), he/she will try and pick up most value for money products.

My point in the context of testing, when choosing a testing tool/framework, try it out for a week, see how much effort for the team to learn and how it cope with application changes? This is common sense really, isn't it?

TestWise 2.0

Thu, 18 Aug 2011 00:00:00 GMT

TestWise 2.0 - Next Generation Functional Testing Tool Gets Even Better

After over one year's development, we are excited and proud to release TestWise 2.0.

TestWise 2.0 Highlights

	Support latest Watir 2 TestWise continues to provide better support for Watir, the latest Watir 2 is also close to be released. If your main target browser is Internet Explorer, in our opinion, Watir is the framework to go for its reliability, concise syntax and performance. If you require testing against Firefox or Chrome and on non-windows platform, you can also use Selenium 2.0 in TestWise 2.
	Selenium 2.0 (WebDriver) Support The long-waited Selenium 2.0 is released in July 2011, and now you can start using TestWise 2 to develop your Selenium 2.0 web tests, as well as Watir. Selenium 2 supports all main-stream web browsers: Internet Explorer, Firefox and Google Chrome, which fits well for multi-browser testing. In TestWise, you can run tests against different type of browser with a just single click of button. Screencast: running same test in IE, Firefox and Chrome
	Cucumber Cucumber allows testers describe how software should behave in plain text, like this: It is not hard to see why it is getting popular. Screencast: Watir in Cucumber
	TestWise IE Recorder Our TestWise recorder works in Firefox (as an extension). Based on feedback, we created one for IE for web sites not working well under Firefox. Plus, this recorder also records operations in Selenium 2.0 ruby syntax as well Watir and RWebSpec. Please note: recorders are just helpers and recorded test scripts are hard to maintain. Don't forge to refactor! (Tip: using TestWise)
	And many more ... We listened your feedback and suggestions, a long list of improvements went into TestWise 2.0: Take screenshots when a test fails Test reports with screenshots (when failures occur) JIRA integration Show call tips when auto-complete page functions Intelligent file finder Editor themeing ...
	More Help Our users like video tutorials, and we added many more. Shall I use RWebSpec, Watir or Selenium 2.0? Which BDD framework suites me better: RSpec or Cucumber? We created a public repository containing six projects with above combinations. Just simply try them out in TestWise, this also serves a good learning tool for testers who are familiar with one framework and is willing to learn another.

Are Software Testing same as VCS, worst are most expensive ones?

Sat, 06 Aug 2011 00:00:00 GMT

Are Software Testing same as VCS, worst are most expensive ones?

Today I came cross to the Martin Fowler's Version Control Software Survey. Ironically, commercial and expensive VCS such as TFS and ClearCase are behind, correction, far far behind free ones such as Git and Subversion rated by thoughtworks consultants. Given the nature of version control software, it is hard to be critical against free good working ones, as they are widely approved by large software projects, why there are people still paying money for more problematic ones? Sounds crazy, but this happens all the time, remember old expensive J2EE containers, it turns out free Tomcat suits most projects, free and more stable.

This kind lack of common sense: prefer very expensive and useless rather than working and affordable happens to our testing world as well. Do you know projects who buy very expensive testing tools which generate unmaintainable test scripts, eventually left on shelves, I bet you do, as I heard all the time during testing conferences. But there is a hope, I see increasingly more testing job ads on Watir and Selenium.

Zero Bug Tolerance

Sun, 10 Jul 2011 00:00:00 GMT

Zero Bug Tolerance

Occasionally we got questions on use of defect tracking system, my answer is "try not using them if the team is co-located". To back up my claim, I quoted from the two classic books below:

From 'Agile Testing' book, "New agile teams usually have a hard time believing it can be done." "The objective is not to get ride of a defect tracking system -- it is to not need to a defect tracking system. There is a big difference. Ths trick is to expose a defect the moment it occurs."

From 'Extremem Programming 2nd Edition', "At first, American workers can't believe this. Chet Hendrickson told me the story of his brother-in-law who worked at a Toyota plant in kentucky. He saw a defective door go by. His buddy said, "Pull the cord." Nah, he didn't want to get in trouble. Another defective door. Another. Finally, he pulled the cord. He was praised for telling the truth and pointing out flaws. Unlike mass-production lines where someone "down the line" is responsible for quality, in TPS the goal is to make the quality of the line good enough that there is no need for downstream quality assurance. This implies that everyone is responsible for quality."

Google lost to scripting languages?

Sun, 03 Jul 2011 00:00:00 GMT

Google lost to scripting languages?

News: Oracle Seeks ‘Billions’ From Google in Java Patent Lawsuit for use of Java in Andriod. When Google acquired Android, why tied themselves to a technology belong to a potential competitor (Sun, then Oracle). With talents in Google, it shouldn't be too difficult to implement the idea in free languages such as Ruby or Python. Surely it will be of some work extending the language, but it won't cost 'billions', let alone on-going license fees. Apple is smarter on this.

To extend the thought further, It seems Google, with all respect, is not winning against companies embracing free and scripting languages, namely:

FaceBook, implemented in PHP
Twitter, originaly created in Ruby on Rails

TestWisely RSS Feed

Verification efforts in total software development cost: more than half!

Verification efforts in total software development cost: more than half!

Print User Story Cards in StoryWise

Print User Story Cards in StoryWise

Testing tags not supported by Watir

Testing tags not supported by Watir

Why more Scrum than XP?

Why more Scrum than XP?

Sample projects: RWebSpec, Watir, Selenium, RSpec, and Cucumber

HP TouchPad marketing irony

HP TouchPad marketing irony

TestWise 2.0

TestWise 2.0 - Next Generation Functional Testing Tool Gets Even Better

TestWise 2.0 Highlights

Support latest Watir 2

Selenium 2.0 (WebDriver) Support

Cucumber

TestWise IE Recorder

And many more ...

More Help

Are Software Testing same as VCS, worst are most expensive ones?

Are Software Testing same as VCS, worst are most expensive ones?

Zero Bug Tolerance

Zero Bug Tolerance

Google lost to scripting languages?

Google lost to scripting languages?