Data driven UIDs for automated WPF UI testing

The Ultimate and Premium versions of Visual Studio 2010 include a new set of tools for creating automated tests that interact with application UI known as Coded UI Tests. One of the tools is a recorder that will generate the appropriate code for you as it monitors your interactions with your application.

Although it is based on older mechanisms for examining the UI (it makes me yearn for the clarity of Snoop), it is still effective at identifying most WPF interactive controls in simple applications. Problems start to arise when working with the more complex data-driven type of applications where WPF excels.

Continue reading

What is RECESS ?

Here at InterKnowlogy, as a Microsoft Gold Partner, we pride ourselves on being able to keep up with the latest and greatest technologies, and to bring that breadth of knowledge and experience to the table for our customers.  As an example, the latest projects we’re working on are based on WPF, Silverlight, Surface, Windows Phone 7, and even iPad development.  As we all know, the pace at which new technologies come out of Microsoft and other industry leaders these days is crazy, so it becomes difficult to keep up. 

At IK, we have a perk we call RECESS

Research (and)
Experimental
Coding (to)
Enhance
Software
Skills

The company gives us some time each week to work on whatever we want – learn a new technology, write an app for a different platform, investigate the feasibility of some new pattern, catch up on new language features, etc.  It’s a great investment that IK makes in us to spend a few hours away from our current project, doing something completely different, and then share that knowledge amongst the rest of the company.  We got the idea from one of our devs working at Microsoft a couple years ago – that team had a similar program.

We don’t always end up with a finished “product”, maybe we just cut some sample code, read some articles, etc. but once in a while, we end up with some very cool stuff.  As an example, some of the actual software we’ve created during RECESS:

  • Surface Craps
  • Surface JukeBox
  • Wish 43
  • Surface Curling
  • Firebrick
  • Atlas (Virtual Earth on Surface)
  • 3D Boxes (Surface physics engine display)
  • Surface PixMatch (child picture matching game)
  • Surface YouTube viewer
  • Blackjack
  • . . .

Anyway, thought the RECESS concept is worth mentioning – I think it’s a very cool “feature” of working here. 

Well, … this afternoon is RECESS, so I have to get busy learning something new…

Surface Craps

Disclaimer:  InterKnowlogy is not in the business of writing gambling or casino games.  This application was only written for research, with no intention of using it in a real gaming situation.

VIDEO

Here at InterKnowlogy, we have a program called RECESS (Research and Experimental Coding to Enhance Software Skills) – where we get to spend Wednesday afternoons working on whatever we want, to further our skill set, maybe in a technology area we don’t touch on our everyday projects.  crapsbasic_61F83FBC

We’ve been writing apps for the Microsoft Surface table for a while, and I decided it would be cool to write a Craps game. One of the main ideas of the Surface table is to encourage multiple people to use an application simultaneously while sitting around the table.  What better way to do that than to place bets, “roll” dice, and win “money”?

Before we get too far, check out the video of the game in action.

Now I’ll get into some of the cool design and implementation aspects of the application.

Tag Recognition

Obviously, you need to place bets to play craps.  At a real craps table, you place bets with your chips on CrapsChipsAndDice_6A87B0D4the table, so in Surface Craps you do the same. We use a stack of chips taped together just so they’re easier to hold, and we use “object tags” stuck onto the bottom of the stack to uniquely identify you as a player.  The Surface has cameras inside the case that recognize the object tag patterns and wake us up in event handlers with information about the tag that was placed.  When your chip stack touches the table in a “bettable area” the bet is recorded, and a visual indication of your bet is added to the table.  Up to 4 players can play at the same time, each having their unique stack of chips and placing bets on the table simultaneously.

Bettable Areas

I came up with a UserControl that I call a BettableArea that allows me to know if a tag (stack of chips) is placed somewhere on the table that allows bets (as opposed to dead space around the table, etc).  This control’s content is a Path that defines the actual region that’s bettable (to allow for funny shapes) and also has 4 properties that are filled with the location where the virtual chips should be placed for each player in that region.  The screenshot below shows the BettableArea controls with their opacity turned up to show them at design time – notice the curved and angled paths used to follow the actual betting zones.

 

CrapsBettableAreas_19F20BA2 A quick aside: on a real craps table, the dealers will place the chips for a player at a spot on a bettable area that’s related to where they are standing around the table.  (if you’re at the right corner of the table, your bets will be at the right corner of the bettable area).  When a bet is placed on my Surface Craps table, the visual chips representing your bet show up at one CrapsPlaceOddsSmall_7DE08B0Fof the 4 spots defined in the BettableArea control depending on which player you are.

Many craps players know that the “Place Odds” bet behind the Pass Line is the best bet in all of Vegas.  So, YES, Surface Craps does support placing odds behind the pass line bet, and even enforces the fact that you can’t do it until a point has been established!

 

 

Ratchet Gesture

CrapsRachetBet_658D70F4After you place your initial bet with your chips, we had to come up with an easy way to change the amount you’re betting.  I came up with something I call the “Ratchet” gesture.  You place your chips back down on the table over the existing virtual chips and I show a couple arrows on each side with the bet amount.  While holding the chips down, you turn them left or right (say as if you’re unscrewing a bolt).  As you turn, the bet amount is adjusted up or down depending on the direction of the turn.

3D Dice

I also used this application as a way to learn WPF 3D.  Cubes are about the easiest object to make since they’re CrapsDiceMidSpin_5C5135B3square, and thus only require 2 triangles on each face.  They material is semi-translucent to be able to see through the dice faces to the back of the opposing face.  I use WPF animations to rotate the dice along 2 of the 3 axis when the user flicks the dice to “roll them”.  The dice spin at a rate consistent with the force of the flick detected by the Surface cameras.

Rules / Payout Engine

I came up with a pretty straight forward architecture to allow me to process bets and payouts after each role.  There is a BetLogicBase abstract class from which I derive a specific bet logic class for each type of bet (pass line, field, hardway, etc).  They all implement the virtual ProcessRollResult( ) method, applying their specific logic for that type of bet, and determine what they should do with the bet they’re attached to: payout (win), remove chips (lose), or nothing.  This design makes it super easy to generically process the bets after each roll.  I just loop through all the BettableAreas and call ProcessRollResult( ) on any bets in that area.

On-Table Help

Instead of On-line help, I came up with “On-Table Help”.  If you’re wondering how the hell to play cCrapsTableHelp_68067CE5raps, or can’t remember what the rules are for a particular bet, you just touch the faint question mark button next to that area, and up pops a ScatterViewItem with the information you’re looking for.  Among other things, it tells whether it’s a single roll bet, or multi-roll, and what the payout table is.

Acrylic Dice

Finally and definitely one of the coolest features of the game are the acrylic dice.  We only just got these dice in the last month, so until then you only play Surface Craps by flicking the virtual dice on the table.  We got these dice that are transparent, and have nearly transparent byte tags on them.  My understanding is that the IR sensitivity of the dots in the stickers on the dice is set so that the Surface cameras will recognize them.  VERY COOL! CrapsAcrylicDice_3B3E3FC5

So I recently went about implementing the ability to roll those physical dice on the table, and have it recognize which face they land on.  I show a visual of the correct face of the dice under each physical die, and process results just as if the user had flicked the virtual dice.

Whew – long winded I know, but I’m super stoked on how this game came out.  Take a look at the video if you haven’t already, and let me know what you think.  Any ideas for other options or functionality in the game?

Thanks to Kevin Kennedy for the graphic design of the UI – he’s one of the InterKnowlogy graphic design studs.  Another thanks to Joe Seymour who helped me with the betting/payout engine.

 

 

Disclaimer:  InterKnowlogy is not in the business of writing gambling or casino games.  This application was only written for research, with no intention of using it in a real gaming situation.

Improvements to the WPF Snoop Utility

I made a couple of improvements to the WPF Snoop utility that I posted a while ago.  (as before, this is an update to the original created and posted by Peter Blois, with his permission)

Property Filter ComboBox info now comes from config

After publishing the last version with property filter support, I continue to get suggestions from people using the tool for other properties and groups that should be included.  This is obviously begging for the data to be based on configuration, so I dove in and made it so.

It’s not as straight forward as a simple app.config setting, since the code that displays the property grid and needs to know about those property filter groups is running within the process space of the EXE that you’re snooping.  So you still edit the config file that’s in the directory where have snoop.exe installed, but at runtime, the config is read “remotely” from the snooped EXE.

To add/edit property filters, edit snoop.exe.config.  The layout looks like this:

<PropertyFilters>        <add DisplayName="Layout"         PropertyNames="width,height,actual,margin,padding,canvas,align" />        <add DisplayName="Grid/Dock"         PropertyNames="grid,dock" />        <add DisplayName="Color"         PropertyNames="color,background,foreground,borderbrush,fill,stroke" />        <add DisplayName="ItemsControl"         PropertyNames="items,selected" />        <add DisplayName="Text"        PropertyNames="font,align,trim,wrap" /></PropertyFilters>

 

The comma-separated values in the PropertyNames attribute are split and used as case-insensitive substring searches.

Mouse Wheel Property Editing

I just finished a WPF project where we used Visual Studio 2010, and it required lots of WPF canvas layout.  Normally I would fire up Expression Blend to lay things out exactly where I want them, and be done.  Unfortunately, Blend 3 does not (yet) like VS 2010 project files.  So this made for a very tedious couple months of run-time adjustments of elements.  I would run the app, and use Snoop to adjust Canvas.Left/Top, Margin, Width/Height, etc, each time, typing in a new value, seeing the changes in the app.  UGH.

So that was the motivation for this new feature.  Now when you select a property in the property grid, for fields that support discrete changes to their values, you will see an additional copy of the current value in the right edge of the current value column.  The great thing about this copy of the property’s value is that you can mouse wheel over them to CHANGE THE PROPERTY VALUE!

MouseWheelEditing_4CE25570

There are many data types supported:  the obvious int, double, and boolean; and then maybe not so obvious.  Data types supported:

  • Int32
  • Double

(For the above numeric types, the default is to change the value by 1.  Hold down SHIFT while wheeling to change by 10, hold down CTRL while wheeling to change by 0.1)

  • Boolean
  • Thickness
  • Brush
  • Visibility
  • Horizontal/VerticalAlignment

For the “multi-part” data types (Thickness and Brush), you can change individual parts of the property by mouse-wheeling over just that part.

So this ends up giving me a run-time design experience where I can quickly tweak layout and style without a bunch of text typing / editing.

(Thanks to my co-worker Brad, who tested this version and blew it up when trying to mouse-wheel edit a LinearGradientBrush background.  Types that derive from supported types (i.e. GradientBrush from Brush) that are not supported are listed in the config file and can not be edited with the mouse wheel.  In the case of GradientBrush, you CAN delve into the GradientBrush, to the GradientStops, use the Indexer feature added last time, and then adjust the color or offset of an individual stop at runtime!!)

The Bits

The binaries – one folder for 32-bit and one for 64-bit.

The source code for the solution.

Give it a try!  Hope you like it.

64-bit (x64) Snoop for WPF Development

The popular Snoop tool from Peter Blois has been out for a long time, and I use it almost every day in my WPF development.  However, there has always been something missing:  a native 64-bit version. 

My coworker Joe and I finally decided that it was worth looking into building one.  There were a few tweaks to the code itself to compile in 64-bit, and then some project build directory settings changes … but for the most part, it was pretty straight forward to make. 

So with permission from Peter Blois, I have created and posted a true 64-bit version of Snoop.  Mind you, this is version 1 of Snoop – haven’t yet tried to tackle version 2.  I tend to use v1 in my everyday WPF dev – not having property editing in v2 is a showstopper for me.

Bit Versions

Snoop works its magic by injecting a DLL into the process space of the app you are snooping.  The problem with snooping a 64-bit app with the original snoop is that the injector DLL was always 32-bit, and when Snoop tries to load a 32-bit DLL into the 64-bit target application, you get the invalid image exception. 

To allow snooping either a native 32 or 64-bit target app, I have built both a 32 and 64-bit version of Snoop and its supporting ManagedInjector DLL.   So unfortunately, the only downside of this approach is that you have to keep both versions around, and use the one that’s right for the bit-level of the target app.  (There has already been a request for ONE version of Snoop that will load the appropriate version of the injector DLL, so maybe someday I’ll dig deeper into that.)

Enhancements

While I was in there building Snoop, I decided to throw in a couple other tidbits – either changes to small things that have always bugged me, or small features that I wish the original had. 

  • “ShowAllProperites” ToggleButton is off by default.  I find that more times than not, the properties I want to look at are in the “defaults” set, so this keeps the properties list much shorter.
  • Use a 2nd monitor if present.  My everyday use of Snoop is that I choose the app to snoop, the SnoopUI comes up and I move it over to my 2nd monitor.  I added some code to do this automatically if there’s a 2nd monitor present.
  • Scroll to Selection – when you snoop on an element in your app with CTRL+SHIFT pressed and the element is found in the tree, the selected item is automatically scrolled into view.
  • Custom property filter ComboBox.  In addition to filtering the properties list by text, there is now a ComboBox next it that allows to filter the properties by functionality.  For example, if you choose “Layout”, you see only properties like Width, Height, Margin, etc. that have to do with layout.  Choose ItemsControl for things like ItemsSource, SelectedItem, etc.   This screenshot shows snooping on a ListBox with the custom property filter set to ItemsControl to quickly see everything related to ItemsControls and Selectors.

 BlogSnoop641_3378ECEA

  • Indexer delve.  This is the small TextBox to the right of the Custom Property ComboBox.  If the current property type (that you’ve delved into) is an ICollection, you can type an index value in the TextBox and delve into that item in the collection.  The indexer delve plays along with all the other delves, so you can pop context to get back to where you came from.

Here is the properties window after delving into a collection data type (ListBox.ItemsSource which is ObservableCollection<Person> in this case).

BlogSnoop642_3A2BF66D

After entering a “1” in the Indexer Delve field (updates on text change), you see the Person object at index 1 in the above list.

BlogSnoop643_1A10E9B0

There you have it.   I hope these enhancements, along with a native 64-bit version are helpful in your WPF development!

Here is the zip with both 32-bit and 64-bit folders.  And since the original source was posted, here is the source code for this updated version.

Initial Thoughts on Blend 3 MIX Preview

After trying out the new features in Blend 3 I’m definitely looking forward to the full release. It already seems to be more responsive than Blend 2 SP1 while navigating around files, templates, selected elements, etc, even at this pre-Beta stage. I was disappointed to see SketchFlow missing from the Preview, but look forward to it being included when it’s stable enough.

There are a bunch of nice UI tweaks:

  • Tool Panels now allow pinning (like in VS) to autohide. Data and Objects and Timeline are also now their own panels so they can move around freely. The left side tool strip now floats and docks too.
  • The Active Container is no longer a static setting. It’s still possible to set it to a fixed object, but now it must be done from the right click menu (Pin Active Container) instead of double clicking. I was annoyed by this at first and wondering why there was no yellow selection border, but then realized that Blend was inferring a container based on what I was doing. Double clicking to add an element now uses the current selection as the container to add to, or its parent if it doesn’t support children. A blue border takes the place of the yellow indicator in these cases.

Even better, when adding by dragging directly on the design surface, the container is inferred graphically and switches (with blue border) as you move the cursor around.

  • Like the active container setting, elements are similarly highlighted with a blue border as you mouse around with the selection arrow tool, making it easy to find elements. This includes elements that are buried in layers of panels, and selecting an element is as easy as clicking on it when it’s highlighted, virtually eliminating the need for using the right click “Set Current Selection” menu item (although it is still available). I haven’t had to use it yet once.
  • Design-time sizing is available on more root container than before, including both Control and Data Templates. Rather than being set on the template itself the size is set on the first element in the tree.

  • The old Gradient tool that was pretty much only useful for rotating gradients now allows direct setting of stops on the design surface. Each one is shown as a circle and can be dragged to a different spot or edited in place (with popup color picker) by double clicking. Alt-clicking adds new stops and dragging a stop out of the gradient area removes it. The Properties panel brush editor itself has also added the ability to set a gradient stop position as a numeric percentage (great for translating from Illustrator) and reverse the order of gradient stops.

  • Annotations allow for design notes to be added directly to the UI that only show up in Blend (think comments for designers). I tried adding some in a real WPF project and the attached properties they created didn’t seem to have any effect on VS or Blend 2. I haven’t figured out where the AnnotationManager class lives so I’m hesitant to leave notes on files that other people need to edit without Blend 3 but at least they didn’t cause any immediate backward compatibility issues on my own box.
  • New keyboard shortcuts abound. Look around the menus to find them – the help hasn’t been updated yet.

My favorite part of the new version is design time data – something I’ve been doing awkward workarounds for since I started using Blend. There are lots of options, but my favorite so far is the generated sample data source. I first tried importing an XML file, which gave me a full set of data and showed all the available fields, including nested collections. I then set up another data source from scratch, using only the Blend UI to define the data structure:

Anywhere there’s a “+” a simple, complex or collection property can be added. Simple properties can be set as String, Number, Image, or Boolean types, with further options for each. String properties can be set to things like Names, Dates or Phone numbers to provide realistic looking data. Images of chairs are provided out of the box but can be replaced by pointing to a folder of image files of your choice.

All of this results in a generated class that includes a bunch of dummy data and is declared as a resource in location of your choice. Any element from the data source can then be dragged onto the design surface to set up a binding or generate a bound control. I haven’t quite gotten the hang of predictably hooking up data by dragging but through a combination of dragging elements and using Blend’s data binding window (which consistently displayed my test data structure through the Data Context tab) I managed to quickly set up a complete multi-level master detail UI for the structure I set up without jumping into XAML at all. An option allows you to switch between using the data only at design time (d:DataContext) or at design and run-time (DataContext) – this should be good for initial testing of a UI.

Through a few hours of working on a live WPF project I haven’t had any problems using the new version and hopping back and forth to VS. I even tried opening in Blend 2, 3, and VS and making changed in all 3 with no problems. I haven’t done much more than start up a new Silverlight 3 project but that worked fine too and the Preview install doesn’t appear to have negatively impacted my existing Silverlight 2 dev environment.

Get the preview here

and if you’re using TFS look at http://code.msdn.microsoft.com/KB967483 for the hotfix needed to hook up the Preview’s source control integration.