With the upcoming release of Visual Studio 2012, Portable Libraries will finally be built into the product. The 2012 iteration of Portable Libraries is much improved from the prior versions, adding not only new platforms (Windows 8 Metro) but greatly increasing the list of available framework library classes that can be used across the various platforms.
Presentation Materials for Fullerton Code Camp 2012
Additional downloads to try out the demo code:
Download the Visual Studio 11 Developer Preview: http://msdn.microsoft.com/en-US/vstudio/hh127353
Or get the Async CTP for 4.0 and Visual Studio 2010: http://msdn.microsoft.com/en-us/vstudio/async.aspx
Windows 8 Developer Preview for all WinRT/Metro samples: http://msdn.microsoft.com/en-us/windows/apps/br229516
Easy Async with .NET 4.5: Slides
Code: .NET 4.5 Code (requires VS11 Preview) | Same Code for .NET 4 CTP
The Future of XAML: Slides
Code: Sample Metro App (requires VS 11 on Windows 8 to build) | Sample Silverlight 5 App (shows some things that either don’t work or change in Metro)
Presentation Materials for NE Code Camp 16
Thanks to everyone for coming!
To try out the code we looked at:
Download the Visual Studio 11 Developer Preview: http://msdn.microsoft.com/en-US/vstudio/hh127353
Or get the CTP for 4.0 and Visual Studio 2010: http://msdn.microsoft.com/en-us/vstudio/async.aspx
Easy Async with .NET 4.5: Slides
Code: .NET 4.5 Code (requires VS11 Preview) | Same Code for .NET 4 CTP
Using Kinect in a Windows 8 / Metro App
We have been working with the Kinect for a while now, writing various apps that let you manipulate the UI of a Windows app while standing a few feet away from the computer – the “10 foot interface” as they call it. Very cool stuff. These apps make use of the Microsoft Kinect for Windows SDK to capture the data coming from the Kinect and translate it into types we can use in our apps: depth data, RGB image data, and skeleton points. Almost all of these apps are written in C# / WPF and run on Windows 7.
Last month a few of us went to the Microsoft //BUILD/ conference, and came back to start writing apps for the new Windows 8 Metro world. Then naturally, we wanted to combine the two and have an app that uses Kinect in Windows 8 Metro. At InterKnowlogy we have a “Kiosk Framework” that fetches content (images, audio, video) from a backend (SQL Server, SharePoint) and has a client for various form factors (Win7, Surface, Win Phone 7) that displays the content in an easy-to-navigate UI. Let’s use the Kinect to hover a hand around the UI and push navigation buttons! Here’s where the story begins.
One of the First Metro Apps to Use Kinect
Applications that are written to run in Windows 8 Metro are built against the new Windows Runtime (WinRT) API, which is the replacement for the old Win32 API that we’ve used since Windows 3.0. The problem when it comes to existing code is that assemblies written in .NET are not runtime compatible with WinRT (which is native code). There is a lot of equivalent functionality in WinRT, but you have to port existing source code over, make changes where necessary, and compile specifically against WinRT. Since the Kinect SDK is a set of .NET assemblies, you can’t just reference it in your WinRT / Metro app and start partying with the Kinect API. So we had to come up with some other way…
You CAN write a .NET 4.5 application in Windows 8, using Visual Studio 11 and it will run on the “desktop” side of the fence (alternate environment from the Metro UI, used for running legacy apps). So we decided to take advantage of this and write a “Service” UI that will run in the classic desktop environment, connect to the Kinect and receive all the data from it, and then furnish that data out to a client running in the Metro side. The next issue was – how to get the data over to our Kiosk app running in Metro? Enter web sockets. There is a native implementation of web sockets in the WinRT framework and we can use that to communicate on a socket channel over to the .NET 4.5 desktop which can reply to the client (Metro) socket with the Kinect data.
Some Bumps in the Road
Writing the socket implementation was not conceptually difficult. We just want the client to poll at a given frame rate, asking for data, and the service will return simple Kinect skeleton right-hand position data. We want to open the socket, push a “request” message across to the service, and the service will write binary data (a few doubles) back to the caller. When pushing bytes across a raw socket, obviously the way you write and read the data on each side must match. The first problem we ran into was that the BinaryWriter in the .NET 4.5 framework was writing data differently than the DataReader in WinRT was receiving the data.
As with any pre-release software from MIS, there is hardly any documentation on any of these APIs. Through a ton of trial and error, I found that I had to set the Unicode and Byte Order settings on each side to something that would match. Note the highlighted lines in the following code snippets.
// Send data from the service side using ( MemoryStream ms = new MemoryStream() ) { using ( BinaryWriter sw = new BinaryWriter( ms, new UnicodeEncoding() ) ) { lock ( _avatarPositionLock ) { sw.Write( _lastRightHandPosition.TrackingState ); sw.Write( _lastRightHandPosition.X ); sw.Write( _lastRightHandPosition.Y ); } } Send( ms.GetBuffer() ); }
// Receive data in the client DataReader rdr = e.GetDataReader(); // bytes based response. 3 ints in a row rdr.UnicodeEncoding = UnicodeEncoding.Utf16LE; rdr.ByteOrder = ByteOrder.LittleEndian; byte[] bytes = new byte[rdr.UnconsumedBufferLength]; var data = new JointPositionData(); var state = rdr.ReadInt16(); Enum.TryParse<JointTrackingState>( state.ToString(), out data.TrackingState ); data.X = rdr.ReadDouble(); data.Y = rdr.ReadDouble(); UpdatePositionData( data );
Once I got the socket channel communicating simple data successfully, we were off and running. We built a control called HoverButton that just checks whether the Kinect position data is within its bounds, and if so, starts an animation to show the user they’re over the button. If they hover long enough, we fire the Command on the button.
The next problem was connectivity from the client to “localhost” which is where the service is running (just over in the desktop environment). Localhost is a valid address, but I would keep getting refused connections. Finally re-read the setup instructions for the “Dot Hunter” Win8 SDK sample which tells about a special permission that’s required for a Win8 app to connect to localhost.
Open a command prompt as administrator and enter the following command (substitute your package name for the last param):
CheckNetIsolation LoopbackExempt -a -n=interknowlogy.kiosk.win8_r825ekt7h4z5c
There is no indication that it worked – I assume silence is golden here. (I still can’t find a way to list all the packages that have been given this right, in case you ever wanted to revoke it.)
Finally, a couple other minor gotchas: the service UI has to be running as administrator (to open a socket on the machine), and the Windows Firewall must be turned OFF. Now we have connectivity!
What’s Next?
Beyond those two problems, the rest was pretty straight forward. We’re now fiddling with various performance settings to achieve the best experience possible. The skeleton data is available from the Kinect on the desktop side at only about 10 frames per second. We think this lower rate is mostly due to the slower hardware in the Samsung Developer Preview device we got from BUILD. Given that speed, the Metro client is currently asking for Kinect data from the service at 15 frames per second. We are also working on better smoothing algorithms to prevent a choppy experience when moving the hand cursor around the Metro UI.
Windows 8 WinRT Development 101
Let there be no mistake made that it’s pretty exciting that Microsoft is starting to release early bits for some of its products for us to test out. Windows 7, Visual Studio 2008 (Orcas), VS2010, VS11, Win8, etc. are just a few that come to my mind. These early releases are full of bugs and are definitely not production quality, neither should they be. So I’m just going to go ahead and preface everything I’m going to write with, “The Win8 WinRT framework is pre-beta and therefore should not be expected to work exactly as prescribed.” With that said there are a few key features that don’t work as expected and I’ll be touching on some of those.
Motivation
I’m currently on a project with two other developers here at InterKnowlogy, Dan Hanan and Carmine Sampogna. We are at the tail end of porting what we call our Kiosk Framework to Windows 8 on the WinRT framework. The project has been a ton of fun and we’ve learned a lot. The project has not been as straight forward as we would have hoped. WinRT XAML is similar in capabilities to Silverlight 3 and 4. Sometimes you still feel like you’re in SL2, but again it is pre-beta software.
Environment
Carmine and I are running Win8 from bootable VHDs that we made following the steps in a post by another coworker Travis Schilling. Dan received one of the coveted Win8 Tablets from BUILD. And we’ve installed the full developer preview of VS11.
WinRT Gotcha’s
Only one merged resource dictionary supported: We like to organize our XAML Resources into Resource Dictionaries to help with readability. It also allows multiple controls to share a single resource and multiple developers to work on different resources at the same time without the need for merging XAML files. So naturally we went down this path only to find that this is not supported yet in WinRT. We got the following error:
We found some help from someone who had already encountered the problem: http://metrotrenches.wordpress.com/2011/10/07/tailored/.
DependencyProperties are not typed: DependencyProperties are very important when designing UserControls that you intend to be used in different contexts with different DataContexts. In the current version of WinRT DPs are not typed, which is a big bummer. This work around was found at: http://social.msdn.microsoft.com/Forums/en-US/winappswithcsharp/thread/4fed80a8-1343-41ca-8c27-b20a00689f65.
ElementName bindings only sorta-kinda work: ElementName bindings help simplify and centralize values. One element contains the values and a bunch of others can grab those values. They are very important when it comes to attempting to implement a MutltiBinding workaround. However, we have found that in many cases ElementName bindings don’t work while in other cases it does. Not sure why.
Can’t retrieve non-template resources such as double, GridLength, Thickness, etc.: I discovered this because ElementName bindings were failing in a very specific case.
var myGridLegnth = App.Current.Resources["MyGridLength"]; //fails var myDataTemplate = App.Current.Resources["MyDataTemplate"] as DataTemplate; //works
Works just fine if you are accessing a DataTemplate, ControlTemplate, or Style. However it seems to fail for structs, value types, and complex types.
DataTemplates cannot contain ContentControls that use ContentTemplateSelectors in some cases: I was trying to refactor a DataTemplate into App.xaml (because we can’t have multiple ResourceDictionarys) and I would continually get an Engine Execution Exception.
No Implicit DataTemplates: However there are implicit Styles.
Other Thoughts
One of the most annoying parts about development is that every run of our application is cut short by a FatalExecutionException which claims may be a bug in the CLR, or an ExecutionEngineException with no details. We can’t seem to get around these. Also, any exceptions thrown by bad XAML at runtime have zero details about what is wrong. Sometimes we get exceptions that have nothing to do with XAML or our UserControl but the exception is still thrown by a UserControl. Very Strange.
My coworker Dan has a post with more gotcha’s and his take on Win8. Overall, Win8 is cool, but the preview is severely lacking. We’re hoping it doesn’t take another 6+ months to get new bits!
Windows 8 Development–First Impressions
Along with 5000 other developers, I attended //BUILD/ in September and came home with the Samsung pre-release hardware loaded with Visual Studio 11 and the Win8 bits. Since then, I’ve been writing a few apps, getting my feet wet writing code for Metro / WinRT apps in C#. Here are some early impressions on the Windows 8 operating system in general, and then some thoughts on what it takes to develop Metro apps.
Don’t Make Me Hop That Fence Again
The grand new look of Windows 8 is the “Metro” style interface that you see right away on the Start Screen. The traditional start menu is gone, you don’t have a hierarchy of start menu icons that you wade through, and you don’t have a bunch of icons scattered on your desktop. Instead you have the clean looking, larger sized “tiles”, some of which are “live”, showing you changes to application data in real-time. You can find lots of info on the Win8 Metro interface here, so I won’t talk about that. Apps written for Metro, built against the Windows Runtime (WinRT) run over here on this “Metro” side of the fence.
What is so interesting to me though is that there exists an alternate world in Windows 8 – the classic desktop. This is where legacy apps run, where the familiar desktop can be littered with icons, the taskbar shows you what’s running,etc. Everything from old C and C++ code to .NET 4 apps run over here on this “Desktop” side of the fence.
So here then is the problem. Throughout the day, I run a few apps to check Facebook and Twitter (Metro), then I startup Visual Studio 11 (desktop), then I start Task Manager (Metro), then maybe a command prompt (desktop). Each time I’m in one environment and run an app that runs in the other, the OS switches context to the other side of the fence. This becomes SUPER ANNOYING over the course of a day. I’m in the desktop, all I want to do is fire up a command prompt. Click the start menu (which takes me to the Metro start screen), then choose command prompt, which switches me back to where I just came from and fires up the command prompt. I’ve become accustomed to pinning all my necessary apps to the desktop side taskbar so I don’t have to go to the Metro screen to run time.
Enough of the Rant – What About Development?
The Windows 8 Runtime (WinRT) is a completely re-written layer akin to the old Win32 API that provides lots of new services to the application developer. When sitting down to write a new app, you have to decide right away, are you writing a .NET 4.5 app, or are you writing a Metro app. (This dictates which side of the fence you’ll be hanging out in.) Some of the coolest features of the Win8 runtime are:
Contracts
Contracts are a set of capabilities that you declare your application to have, and the OS will communicate with your app at runtime based on the contracts. Technically they’re like interfaces, where you promise to implement a set of functionality so the OS can call you when it needs to. The most popular contracts are those that are supported by the “Charms” – Search, Share, and Settings. These are super cool. Implement a couple methods and your app participates as a search target, so the when the user enters some text in the search charm, your app is listed and you furnish the results.
protected override void OnLaunched( LaunchActivatedEventArgs args ) { var pane = Windows.ApplicationModel.Search.SearchPane.GetForCurrentView(); pane.QuerySubmitted += QuerySubmittedHandler; pane.SuggestionsRequested += SuggestionsRequestedHandler; pane.ResultSuggestionChosen += ResultSuggestionChosenHandler; pane.PlaceholderText = "Search for something"; // ... } private void QuerySubmittedHandler( SearchPane sender, SearchPaneQuerySubmittedEventArgs args ) { var searchResultsPage = new SearchTest.SearchResultsPage1(); searchResultsPage.Activate( args.QueryText ); } private void SuggestionsRequestedHandler( SearchPane sender, SearchPaneSuggestionsRequestedEventArgs args ) { var searchTerm = args.QueryText; var sugg = args.Request.SearchSuggestionCollection; for ( int i = 0; i < 5; i++ ) { // just faking some results // here you would query your DB or service sugg.AppendResultSuggestion( searchTerm + i, "optional description " + i, i.ToString(), Windows.Storage.Streams.StreamReference .CreateFromUri( new Uri( "someurl.jpg" ) ), "alternate text " + i ); } //defer.Complete(); } protected override void OnSearchActivated( SearchActivatedEventArgs args ) { var searchResultsPage = new SearchTest.SearchResultsPage1(); searchResultsPage.Activate( args.QueryText ); }
Sharing is just about as easy. Declare the contract in your manifest and your app is a share target. When the user chooses to share to your app, the OS sends you the data the user is sharing and you party on it.
protected override void OnSharingTargetActivated( ShareTargetActivatedEventArgs args ) { var shareTargetPage = new ShareTestTarget.SharingPage1(); shareTargetPage.Activate( args ); }
Settings leave a little bit to be desired in my opinion. You declare the contract, and add a “command” to tell the OS to put in the settings charm panel. But when the user chooses that command (button), your app has to show its own settings content (usually in a slide out panel from the right). This is a 4 step process to make any settings changes in your app (swipe from right, touch the button, change settings, dismiss the panel). I really hope in future versions they figure out a way to embed our application settings content right there in the settings panel, not in our own app.
Windows.UI.ViewManagement.ApplicationLayout.GetForCurrentView().LayoutChanged += LayoutChangedHandler; DisplayProperties.OrientationChanged += OrientationChangedHandler; Application.Current.Exiting += ExitingHandler; SettingsPane pane = SettingsPane.GetForCurrentView(); SettingsCommand cmd = new SettingsCommand("1", "Pit Boss Settings", (a) => { ShowSettingsPanel(); }); pane.ApplicationCommands.Add(cmd);
Async Everywhere
The WinRT team has built the APIs with asynchrony in mind from day one. At //BUILD/, we heard over and over that your UI should NEVER freeze because it’s busy doing something in the background. To that end, any API that could potentially take longer than 50 milliseconds is implemented as an asynchronous method. Using the async and await keywords (we’ve seen these in a CTP for .NET 4) that are built into the WinRT-based languages, we can write our code in a flowing, linear fashion that makes sense semantically. No longer do we have to use BackgroundWorkers and worry about marshalling results back to the calling thread.
HttpClient http = new HttpClient(); var task = await http.GetAsync( url ); string responseText = task.Content.ReadAsString();
“Free” Animations
XAML support is implemented in WinRT as native code, and is available to use from C#, VB, and C++. One of the coolest features they added to what we’re already used to in WPF & Silverlight is the “built-in” animation library. Simply add any number of transitions on a container element, and any of the associated operations in ANY OF ITS CHILDREN will use that transition. The list of built-in transitions include: Entrance (content appearing for the first time), Content (change content from old to new), Reposition (fluidly move elements when they get repositioned), Add/Delete (fade out/in and move existing items), and Reorder.
<Grid> <Grid.ChildTransitions> <TransitionCollection> <EntranceThemeTransition HorizontalOffset="500" /> </TransitionCollection> </Grid.ChildTransitions> </Grid>
Well, I’ve jumped all around in this post, and it’s long enough already. Look for future posts with more specifics about features and code that we’re working into our apps.
Some Development Gotchas (aka Bugs)
Here’s a quick list of some issues we’ve run into these first few weeks.
- You can only set DataContext in code-behind – future releases will allow setting in XAML
- There are a handful of bugs in FlipView.SelectedItem. If you’re binding to the SelectedItem property, you have to manually set the SelectedItem to null and then back to the correct property (get it from your ViewModel) in code-behind. This will obviously be fixed in a future release.
- You must use ObservableVector<object> based properties in your VMs if you are binding to ItemsControls.ItemsSource and want the UI to update when there are changes to the collection. ObservableVector<T> does not exist in the class library – you can find source code for it in many of the SDK samples.
- Use the correct INotifyPropertyChanged implementation (from Windows.UI.Xaml.Data, not System.ComponentModel)
- There is no VS item template for custom controls in C#, so VS won’t make you the Themes\Generic.xaml structure. We’re still tracking this one down. We’ve used a Style resource to describe the template for a Button-derived custom control and it works, but you don’t get access to the Template property in OnApplyTemplate, so we’ve resorted to using VisualStateManager instead of referring to template parts and changing UI that way.
- I’m sure there are more to come … in fact, check out my co-worker Danny’s post (should be up in the next couple days) with more details on what we’ve been encountering while writing our first few Win8 apps.
Check out the Win8 Metro developer forums for ongoing discussions of many other issues.
Icons for Windows 8 Metro Application Buttons
To make it easier to use standard button icons, a new Segoe UI Symbol font has been added to Windows 8 that currently includes 25 icons. The available icons are listed below with their hex and base 10 values. The icons can be used from XAML like: <TextBlock Text=”” FontFamily=”Segoe UI Symbol”/>