Venturous: Web Forms WAT framework

Venturous is a library for testing ASP.NET Web Forms applications through the browser. In many ways it is the same as already existing frameworks like Selenium, Watir, WatiN and WebAii. The reason for writing it is because I have some specific design goals in mind that I believe will be important to the success of our testing efforts. I’ll go through some of them in this post.

As I mentioned, there are already frameworks out there that do basically the same thing. In fact Venturous uses the excellent WatiN framework behind the scenes to do the actual work of interacting with the browser. Venturous is written as a layer on top of it, exposing it’s own API surface.

Venturous is specialized towards ASP.NET Web Forms applications. The Web Forms framework has some peculiarities that can make testing through the browser problematic. It is important that tests fail for the right reasons. They should be written in a way that makes them tolerant to changes that don’t break the functionality that is being tested. The goal of Venturous is to help you with this.

Control Hierarchy

As you probably know, Web Forms is built on the concept of a control hierachy where a page contains controls like panels and grids, and these controls again contain more controls. The IDs of the HTML elements are generated by combining the ID given to the control with the IDs of all the parent controls like the path in a folder structure. So whenever the structure of the page changes, if your tests rely on the ID of an HTML element, it will break.

To overcome this, Venturous allows you to write test-controls that follow the control hierarchy. In your test project you define controls that represent (more or less) the actual controls in the page. These end up looking a lot like the ASP.NET user controls they represent and are easy to maintain. (Venturous uses a scoped seach within the parent element for the partial ID.)

Other frameworks support a similar pattern of pages and controls, but not quite as aligned with the Web Forms model.

Built in UpdatePanel support

Web Forms allows you to do “partial postbacks” using the UpdatePanel control. Using WatiN (or the other frameworks) directly, you would have to put some logic inside your tests to wait for the update panel to refresh. This makes the tests require more maintenance (someone added or removed an UpdatePanel in the application, and now the tests fail even though the functionality works for the user). Venturous will automatically handle the partial postbacks so you don’t have to think about it.

HTML element agnostic

With some frameworks you have to specify in the tests the type of HTML element you are working with. This ties the test to an implementation detail of the page markup. Venturous treats all elements the same so you don’t have to worry about it.

Server Errors throw exception

Venturous will check for server errors and immediately throw an exception containing the exception message from the error page. This makes it easier to see what the problem was with a failed test, instead of wondering why some element wasn’t found in the page.

Developer focused

Venturous is code and developer focused. Tests are written as unit tests in a regular unit testing framework. There is no record/playback GUI. This is a tool targeted at the developers working on the application under test so they can add WAT testing as part of their overall automated testing approach.

Opiniated API

The API surface is deliberately kept simple and specialized. Venturous is meant for writing happy-path feature tests that verify that user functionality works. It is not meant for detailed testing of validation errors, javascript behaviour or visual aspects. Therefore there is only the needed support for finding and interacting with elements, and only limited support for inspection. Tests should interact with other entrypoints of the system under test as needed, like web services, to arrange or assert persistent state.

Is there a performance difference between the Visual Studio test runner and TD.Net?

I've semi-recently moved a suite of unit tests from MbUnit to MSTest (ok, I won't groan about why right now), and started using the built-in test runner in Visual Studio. I've noticed that the built-in runner feels slower than TestDriven.NET. Not knowing if this was real or just a feeling I decided to investigate a little.

I did a quick timing of the two on the solution I'm currently working on. I'm using VS 2010 and TD.Net 3.0. I'm timing them manually by starting and stopping a stopwatch so this isn't completely accurate timings, but they give a good enough indication for me.

Full test suite

I did a fresh Get Latest and built the solution first. Then I ran all tests in the solution using TD.Net first and then the built-in VS runner afterwards. Here are the timings:

First run:

• TD.Net: 101 seconds
• VS: 143 seconds => 41% slower

It was noticable how VS spent a long time initalizing. It would first do nothing for a while, then show the test results with a "Test run pending" message, still doing nothing, before it would start running the tests. I'm not sure what it's doing during this time. TD.Net also has a bit of initialization time, but it's not as noticable.

Without changing anything I ran the same tests again. This time the results were more in line with each other:

Second run:

• TD.Net: 78 seconds
• VS: 85 seconds => 9% slower

Both were faster the second time. VS started running tests without too much delay. VS is still slower, but not by as much (9% instead of 41%).

I made a code change in one of the core-level projects on which most other projects depend, causing almost all projects to rebuild. After building successfully I ran both testers again, giving these results:

After rebuild:

• TD.Net: 86 seconds
• VS: 103 seconds => 20% slower

Again, the Visual Studio runner spent a gread deal of time in a "Test run pending" state (about 50 seconds) before starting the testing.

Single test class

It might not be a big deal that VS is a bit slower running the full test suite if you're doing TDD and most frequently run only the few tests covering the module you're working on. I therefore timed the performance of running a single test class only. As before I first ran TD.Net and then VS and timed them:

First run:

• TD.Net: 2-3 seconds
• VS: 8 seconds => +5 seconds, 166% slower

This time the timings were pretty consistent accross repeated runs, and it didn't matter if I ran all 30 tests in the class or just one of them. So it seems almost all of this time is spent in initialization.

If you're actually doing some work you'll have made some code changes before running the tests again, so build time also contributes to the cycle time between tests. I know it's not possible to measure the build time since this varies wildly accross projects, but I wanted to see how this affected the times for my current project. I made a whitespace code change to the SUT and ran the same test again without building first:

After code change, including build time:

• TD.Net: 6 seconds
• VS: 11 seconds => +5 seconds, 83% slower

VS is still the same 5 seconds slower as above, but now this is down to around 83% slower instead of the 166% above.

Conclusion

It's apparent that the VS runner is indeed slower that TD.Net. VS seems to use a lot more time in initialization. It seems to be about 20-40% slower for a full test run, and 80% slower for TDD.

Build times can affect your cycle time a lot more than the test runner, but with well factored modules the build time can be pretty low as you only need to rebuild the project under test and not the whole solution.

These tests were run on a codebase with relatively few tests compred to the amount of code (72 projects and 718 tests). I might have gotten different results if the number of tests matched the code size since initialization time might be more offset by the testing time.

It should be interesting to measure the timing difference between MSTest and another testing framework like N/Mb/XUnit as well.

God works in incremental ways

Man has evolved from cavemen to farmers to our current hi-tech lifestyle. Throughout time we have learned to create many things. Cavemen created knives from bone, farmers created tools from melting metal. But only in our technology era have we become able to create something from nothing. All our earlier creations have been mere modifications upon something physical - bone or metal or empty rolls of toilet paper. But software is not made from something physical. It is something that does exist, but it's not something you can kick, and - like God only could - we are creating it from nothing.

But creating software involves great complexity. Any software project deals with transforming what you can imagine into something working and usable, and this transformation is most of the time not trivial. There are so many ways of doing things that it is very easy to take a wrong step. Managing this complexity is what software creation is all about.

This is why so many software projects run into problems. The complexity is too great, and it is not being tackled accordingly. There have been many attempts to formalize away this complexity with methodologies that aim to tell everyone involved what to do and when, and to limit the number of changes that come over time by a nailing down detailed specification prior to implementing anything. The problem is that these methodologies promote a system so rigid that very limited value ends up coming out in the other end, especially for large projects. There is a system built, but it is often not what the customer really wanted, just what they thought they did when the spec needed to be approved.

Some have come up with a different approach. Just take things one small step at a time. Figure out what small thing will be the most valuable to have up and running first. Now do just that; code it, test it, document it, and deliver it. Now the customer can try it out, get some experience with how it works in practice, and then figure out the next thing that will give them the most added value. Step by step the customer gets a more and more valuable system. Call every such step an increment, and this process is what is called Incremental Development.

Now that we all know that the Bible contains a detailed description of how the brain works, it makes you wonder what else is hidden in there in plain sight. I thought I would do some research of my own to see if I could find anything interesting. Expecting to read through a lot of material with little hope of finding anything substantial, I was quite surprised to find great value from the very first paragraph.

What I found is that the Bible actually teaches us incremental development. This method is demonstrated precisely in the story of how God created the world. God has documented for us just how he executed his own development project. Lets go carefully through the passages of Genesis and look at the details:

¹In the beginning God created the heaven and the earth. ²And the earth was without form, and void; and darkness was upon the face of the deep. And the Spirit of God moved upon the face of the waters.

Here He has first set up an empty project to get started. This, as you see, was simply a kind of "Hello World" starting point to work from. And from this He started his first increment:

³And God said, Let there be light: and there was light. ⁴And God saw the light, that it was good.

Here are the main steps of an increment in a nutshell. As the first step He decided what functionality would give the most value to implement first. As "darkness was upon the face of the deep", He committed His first increment to implement light. He was starting with something of great value, yet not too complex. Pure energy, no shape or form, just beams of light.

As the second step He went ahead and implemented it. And so, "there was light".

As the third step, to verify His implementation, He did the proper testing. He took a good critical look at the light, and made sure that "it was good".

And God divided the light from the darkness. ⁵And God called the light Day, and the darkness he called Night. And the evening and the morning were the first day.

To complete the increment, He documented what He had made. He documented that light was something separate from darkness. He documented the light as being "Day" and darkness being "Night", And with this He concluded the first increment - the first Day.

The rest of Creation was completed by several more such increments, following this same pattern. Now that He could see His world in a new light, He decided to do a second increment to separate the sky and the earth, then a third to separate the sea from the land and add vegetation, a fourth to add the sun, moon and the stars, a fifth to add fish to the sea and birds to the sky, and a sixth to add animals to the land, and people to rule it all.

¹Thus the heavens and the earth were finished, and all the host of them.

After six increments He had completed enough features to call the first release finished. He has a working heaven and hearth, with a fully functional ecosystem. And as Man was made with a creative spirit, like his Father, He documented His incremental method in the Bible so that men can benefit from His wisdom in their own projects.

Motivating software developers

I've seen how a lot of people don't understand what motivates a software developer. What motivates them to do good work, to work extra hours when needed, to come to work each day, to even stay working for the company.

The problem I think is related to how different types of people are motivated by different things. People choose their profession by where they feel motivated to work. Sales people work towards getting their hands on as much money as they can, and these are people that are motivated by money. So for them incentive pay works well. To give them a percentage of what they bring in aligns their personal motivation with the interest of the company.

On the other hand, software developers are motivated by other things. They love solving problems. If you give me the choice between a highly paid but mindless job, and a medium paid job with interesting problems to work on, I will most of the time go for the fun one. (That is, the one with the problems.)

Many companies have a salesperson type of guy in a management position, and his money-focused approach to motivation is gossly misaligned with the interests of developers. Throwing money around for overtime work will apparently produce results in the short term, but in the long term this actually undermines the basic motivation. Developers need to be motivated differently from sales personnell in order to thrive.

So how is a developer motivated then? I am a developer, and I can tell you what motivates me. It is actually not hard to do, or even expensive. I'm largely self-motivating when I'm just allowed to be. I am a developer because I love to develop software, and if I'm not being held back I will do just that, and do it well. If developers seem unmotivated there is probably something in their way that needs to be removed. If I'm just allowed certain benefits there is no need to spend money on bonuses, I'll still be doing more than I'm paid for.

I'll present here a list of requests that, if granted, will let me work at my full potential. And these aren't costly things, these are just things that take a little bit of understanding and goodwill, and perhaps some small adjustments in the workplace.

Allow me to focus

A developer is best left doing what he loves. Anything else you put me up to is just another something that gets in my way of doing my real work. And when I get too much of this I feel like I'm not doing anything useful. And another thing: Allow me to work in a good working environment. Don't put me the same room with users or salesmen that will interrupt me with every dumb question they can think of, or be talking all day on the phone. I need to concentrate in order to solve problems. If I feel that I can get some work done during the day, I'll be motivated to do so.

Having people around me that work on the same problem as I am, though, is only positive. That's considered collaboration, not interruptions.

Allow me to feel that I am making progress

Building a system bottom-up, starting with backend systems and over time moving towards user interface functionality, makes it hard to see and feel the progress. All I've done so far is to create the building blocks that will later become real functionality. Looking at this isn't much to get excited about. It's not like "Oh my god, my unit test for this business object runs now! This is so cool!". It is much more motivating to implement some working function to completion, all the way from GUI. Something that I can actually see and show off.

This goes not only for the developers, but for everyone involved, from managers to customers. Everyone gets to see that the team delivers something functional, that the project is coming together. And I get to show off what I've made and feel that someone is actually interested in what I do.

A good way to achieve this is to have frequent small releases where a set of functions are chosen to be included each time. These functions should be made release-ready - as in coded, integrated, tested, bugfixed and documented. By demonstrating these releases, everyone sees exactly how far the project has come.

Allow me to make something I can take pride in

I take pride in making good code. Code that not only works, but that is elegant, neat, well designed and tested. Being allowed to finish one piece of functionality before starting something new is a good way to achieve quality. By letting me focus on one thing at a time, and by demanding that it must be both tested and documented before I can call it complete, I can make something that I can take pride in. I'll know I have done solid work. My spirit is high and I can stand proud to show what I've accomplished. So now I am motivated to take on the next challenge - the next release - knowing that I haven't left behind a complete mess that will haunt me with bugs for a long time.

Allow me to do it for me, not for the money

As I am taking pride in my work - my craft - I'm doing the best I can to make quality code and deliver on time. If I'm pressed on time I'll work overtime to deliver something good rather than mediocre. This additional effort is motivated by my commitment to the team, to the company, by my pride and by getting to do what I love. By tying cash bonuses up to the reaching of some goal, milestone or whatever within some timeframe I'll feel that I'm not trusted to be putting in that extra effort otherwise. It turns all that goodwill into greed, and before you know it I am not really committed to the work anymore. This is extremely dangerous. As hard as it is for anyone but the developers to tell what level of quality is being delivered, when my motivation is tied to the bonus I can get away with a lot of shortcuts and crappy code.

Also, bouses that are not distributed evenly between everyone in the team - or even between teams - will be grounds for envy and give root to discusions. It contributes to splitting the team, not building it. Also, if I got less than someone else I'll feel that my effort is not appreciated. And, anyway, how can management objectively and correctly determine what each person's contribution to the group is worth?

Allow me to work on interesting problems

If I am given a difficult task I will take it as a challenge, feel good about being trusted to handle it, and be motivated to show how good I am. Of course not all tasks are challenging, so I can't be given these all the time. But just make sure that there are at least some exciting tasks between the trivial and boring.

If the same types of tasks are assigned to the same people all the time - designers do only design work, testers to only testing, and programmers do only coding by a detailed spec - they'll all get bored quite soon. People are interested in more than just one thing. By rather putting all these people on the same team and letting them work together through all phases, they will challenge each other and learn from each other, and they'll be having lots of fun doing it.

Allow me to be part of a team

Solving problems together, bouncing ideas around, getting challenged on my own ideas. There are so many joys of working in a team. If I get to work in a team with people of all different fields, and we all focus on the same set of problems, we all will be having a great time. Motivation will be high, and we will be able to get some great work done. What's often done instead is to isolate everyone by having them work on individual tasks, separate them with walls or in cubicles, demand that they communicate through written documentation rather than casual discussions, and having them all so pressed on time that there is no room for collaboration.

There are so many benefits to teamwork I can't really list them all. Quality will improve as we get to discuss ideas, we solve problems together, and I get reviewed on my work. Social aspects improve for obvious reasons. Commitment to the team is high and everyone will notice if I'm not delivering. Knowledge transfer is high from constant discussions. Everyone's total overview of the product is better, and any potential problems or bad decisions will be detected sooner. It all contributes to increase motivation.

Allow my ideas to be taken seriously

I'm the one closest to my own situation, so if there's anything holding me back I'll be the first to know. Provide me with frequent opportunities to express any ideas or concerns that I might have, and let whoever I'll be talking to have the teams interests at heart. Listen to my ideas and concerns, I might suggest something that would improve the whole team's productivity, or point out something that is holding us back. It goes a long way to motivating me just by having me feel I am taken seriously - that what I do matter.

So, there you have my list. It's not too much to ask is it? There's nothing too fancy. Nothing that costs a lot of money. Just some simple requests to help me thrive in my workplace, to let me actually do my job. That's where my motivation comes from.

Operators - The dice game for geeks

A friend of mine and me had some time to kill at a cabin trip some time ago. As we got tired of Yatzy we started inventing our own game. All we had was the Yatzy dice and a deck of cards, so that's what the game was based on.

The basics

What you need to play Operators is five dice and a deck of cards. The basics of the game are to flip a card, toss the five dice, and then find a way to calculate the value on the card from the numbers on the dice. Like, if your card shows a 7, and your dice shows 1, 6, 2, 5, and 1 then you can combine them like this:

6 + 5 - 2 - 1 - 1 = 7

All the dice must be used in the solution. In this solution there are two mathematical operators in use: + and -. If you wanted to use more operators you could find a more intricate solution:

6 / 2 + 5 - 1 * 1 = 7

This one uses four different operators.
The fun part starts when you introduce more operators to the game, especially binary operators. How about this solution?

6 SHL ((2 SHR 1) AND 1) - 5 = 7

The operators

The operators we first started to use were the normal mathematical operators, +, -, * and /. These should be familiar to all geeks. If they aren't to you then I'm afraid this game is not your cup of tea. A rule of the game is that only operations that give whole-numbered results are allowed. Operations resulting in fractions can not be used. I.e 2 / 5 is not valid.

After a while we started adding more advanced math operators. These are SQR (square), SQRT (square root), POW (power), and MOD (modulo). It was at this point we started losing players. Remember that the rule of whole numbers still apply, so SQRT(8) is not allowed.

At some point we started using also binary operators, which include AND, OR, XOR, SHL (shift left), SHR (shift right), ROL (rotate left), ROR (rotate right) and NOT. For the operators that rely on a given number of bits, namely NOT, ROL and ROR, the number of bits is set to 8. This means that NOT 1 = 254 and that 1 ROR 1 = 128. These operators can't be used on numbers larger than 255. - Now we were getting to play alone.

Tip: When using binary operators, you can visualize by counting on your fingers.

We also tried out some other operators that we decided to disallow in the game. They made it too easy. Most notable is string-concatenation (&) as in 1 & 2 = 12.

How to play

1. Puzzle

A simple way to play is to just try and find a solution. This is like a puzzle more than a game. Just toss the dice and to try and find a correct solution. Sometimes it's easy, other times it's impossible. This way of playing is good practice and a good brain-teaser.

2. Points

A more competitive game is to count points from the solutions. Play like the puzzle, but when the solution is found you calculate a score of points from it. There can be many ways to count points. One way can be to give a certain number of points just for finding a solution, and then add points related to the intricacy of it - like the number of operators used, or based on a "trickyness level" of the operators.

3. Timed

For the quick-headed of you an option is to decide a certain amount of time between tossing the dice and coming up with a solution. You get, say, 1 minute, and you come up with as many solutions as you can within that time. The solution with the highest number of points is used and the score is noted for each round. This can be either turn-based, or all players can work on the same dice simultainously. After a given number of rounds the geek with the highest total score is the winner.

Open letter to the makers of shampoo bottles

This my view with blurred lenses


Which one is the shampoo?


Eenie.. Meenie..

I have a complaint about the accessibility properties of shampoo and conditioner bottles, and I believe most people that have less than 20-20 vision agree with me on this.

My vision is quite poor, but the use of contact lenses let me function normally in my daily life. But there is one situation in particular where contacts aren't very helpful - in the shower. The water makes them blur or shift around on my eyeballs, and the result is uncorrected vision.

I can manage to adjust the water and wash myself in all the important places without perfect vision, but when I come to the point where I'm about to pick up the shampoo bottle I run into problems.

I must admit, I'm not very interested in the different brands of shampoo on the market. To me, it doesn't really matter much if I end up washing my hair with Dove, Naturelle or Fructis. I leave the shopping for haircare products to my wife. That also means that I don't really know much about how the different bottles look, I just want to find one that states Shampoo on the front of it. This is where the problem arises. I can't read it!

I can perfectly well read the brand name on the bottle, because this is written in huge letters covering the width of the bottle. But like I said, I don't care about the brand! All I really want to know is whether it contains shampoo or conditioner - or something else that I really don't want to put in my hair. But the letters are written so small it's impossible to make out.

And on top of it, they make the shampoo and conditioner bottles almost identical. There is usually some difference between them, like a slightly different color, or one of them having the opening on the top while the other one having it on the bottom. But this just helps me nada since I don't know which is which - due to my previously stated lack of interest in haircare products and their bottle designs.

So my request to the makers of shampoo bottles it this: Could you please print the words Shampoo and Conditioner in a large, clear, easily readable type so that also people in the shower - where the bottles will be used most likely - with water in their eyes, can read it?

Until then, I'll have my wife buy me one of those combined products like the Wash-n-Go with both the shampoo and conditioner in the same bottle.

Count to 31 on one hand

We are used to counting with our fingers. We have been doing this from we first learned to count. This is quite useful in many circumstances, but the number of fingers are quite limited - 10 fingers in total per person. What if you want to count beyond 10? Do you start using toes? Or another person's fingers?

I've heard that this number of fingers is actually the basis for the 10-digit number system we have been using for centuries. In our modern computer age we should really move to the much more capable binary system. When we count with our fingers, each finger is in one of two states: up or down. Down means the finger is not counted, and up means the finger is counted. If you extend the three first fingers it signifies the number three.

But what if we reconsider these states in a binary system, where down means binary zero, and up means binary one. Then we define that the right thumb is the least significant - being the rightmost finger when you turn your palm inwards. Now you can on one hand - 5 fingers = 5 bits - count all the way to 31! And if you add the other hand, the number doesn't just double, but the number of bits doubles. That means you now have 10 fingers = 10 bits, and you can count all to the way to 1023!

It takes just a little practice to recognize the pattern of fingers as the correct numbers. It's a bit unusual to see three fingers and then understand it as the number 7. But when you've used it a few times you learn most combinations by heart.

Now you can also put numbers to well known finger gestures. Like the the "giving the finger" gesture where the middle finger is extended can now be recognized as the number 4. Or the victory-V gesture with index and middle finger extended is the number 6. The common stop-gesture with all fingers extended is the number 31, and the satan-rock sign where the index and pinky fingers are extended is the gesture for the number 18. As a tip for satan-rockers, you should try another gesture: Left hand extends only thumb and middle finger, and right hand extends all but the thumb and middle finger. This is the gesture for the number 666!

This new counting gives a little different meaning to the term "No more than you can count on one hand". If you hear this you can politely ask if they mean about 5 or about 31.

The JIT-D Development Cycle

This diagram shows the project lifecycle called JIT-D, Just-In-Time Development.