IoT: Ping Pong Scoring via Netduino

Combining InterKnowlogy’s thirst for using the latest and greatest technology with our world famous ping pong skills provides the following result:  during RECESS, I am making a small device that allows us to quickly keep a digital score of our ping pong matches.

Internet of Things (IoT) is a hot topic these days, so I decided to implement the ping pong scoring system on a Netduino board.  I had dabbled with an older board a year or more ago, and was frustrated: one of the first things I wanted to do was make a call to a web API service, but there was no network connectivity.  Enter the newest board, the “Netduino 3 WiFi“.  It has a built-in button and LED, but it’s extensible by way of the 3 GOBUS ports, where you can easily hookup external modules.

Hardware setup

Netduino 3 board with Gobus modules

My shopping list

Required Software

Netduino is a derivative of the Arduino board, with the ability to run .NET Micro Framework (NET MF) code.  This means you get to write your “application” for the board using familiar tools like Visual Studio and .NET.  Here are the steps I went through (loosely following this forum post, but updated to current day):

Network Configuration

This board has WiFi (sweet!), which means you need to get it on your wireless network before you go much further.

Use the .NET Micro Framework Deployment Tool (MFDEPLOY) to configure WiFi on the board.

  • Target, Connect
  • Target, Configuration, Network
  • Set network SSID, encryption settings, network credentials, etc.
  • Reboot the device to take on the new settings!
  • GREEN LIGHT means you’re successfully connected to the network (yellow means it’s searching for a network)

Write Some Code!

After installing the VS plug-in, you now have a new project template.

File, New Project, Micro Framework – Netduino Application (Universal)

Go to the project properties and confirm two things:

  • Application, Target Framework = .NET MF 4.3
  • .NET Micro Framework, Deployment.  Transport = USB, Device = (your device)

On-board light

static OutputPort led = new OutputPort( Pins.ONBOARD_LED, false );
led.Write( true );

On-board button

NOTE: There is a bug with the on-board button in the current Netduino 3 board firmware. While your application is running, pressing the on-board button will cause a reset of the device, not a button press in your application. The work-around until the next version of the firmware is to reference the pin number explicitly, instead of using Pins.ONBOARD_BTN. See my forum post for more information.

static InputPort button = new InputPort( (Cpu.Pin)0x15, false, Port.ResistorMode.Disabled );

GO Button

Now attach a GOBUS button module and the code is a little different.  The Netduino SDK provides classes specific to each module that you use instead of general input / output port classes.

The natural way in .NET to react to button presses is to wire up an event handler.  The GoButton class has such the ButtonPressed event, BUT, there’s a bug in the firmware and SDK:  If you react to a ButtonPressed event and in that handler method (or anywhere in that call stack), you make a call on the network, the call will hang indefinitely.  I discuss this and the work around with others in the a Netduino forum post.

It’s kind of ugly, but instead of wiring up to the events, for now (until the Netduino folks get it fixed), you just sample the IsPressed state of the button in a loop.

Add a reference to Netduino.GoButton.

var goButton = new NetduinoGo.Button();
if ( goButton.IsPressed ) { // do something }

Go Buzzer

Add a reference to Netduino.PiezoBuzzer.

var buzzer = new NetduinoGo.PiezoBuzzer();
buzzer.SetFrequency(noteFrequency);

Talk to the Web!

You bought this board because it has WiFi, so you must want to call a web API or something similar.  In my case, I wrote a simple OWIN based Web API service, hosted in a WPF app that is my ping pong scoreboard display.  This gives me the ability to receive HTTP calls from the Netduino board & client code, straight into the WPF application.

So a call from the Netduino application code to something like http://1.2.3.4:9999/api/Scoring/Increment/1 will give player 1 a point!

I do this using the HttpWebRequest and related classes from the .NET MF.

// error handling code removed for brevity...
var req = WebRequest.Create( url );
req.Timeout = 2000;
using ( var resp = req.GetResponse() )
{
	var httpResp = resp as HttpWebResponse;
	using ( Stream strm = httpResp.GetResponseStream() )
	{
		using ( var rdr = new StreamReader( strm ) )
		{
			string content = rdr.ReadToEnd();
			return content;
		}
	}
}

In my case, the results from my API calls come back as JSON, so I’m using the .NET Micro Framework JSON Serializer and Deserializer (Json.NetMF nuget package).

var result = Json.NETMF.JsonSerializer.DeserializeString( responseText ) as Hashtable;
if ( result != null )
{
    Debug.Print( "Score updated: " + result["Player1Score"] + "-" + result["Player2Score"] );
}

Putting that all together, I have a couple physical buttons I can press, one for each player, and a WPF based scoreboard on the wall that removes any confusion about the score!

Hope you too are having fun with IoT!

 

Create A Great User Experience For Your App

User Experience Design Pyrmid

If your app is infuriating to use, users will give up and find an alternative. User experience design also referred to as UXD or UED, is the process of enhancing user satisfaction by improving the usability, accessibility, and pleasure provided in the interaction between the user and the product. User experience design combines the user interface, user workflows, animations, gestures, artwork and the overall feeling your app gives to the user. The mobile UX design can make or break the entire user experience and high attention and effort is required on features and designs of an app.

Consider these questions before you start designing an app:

  • Usefulness – How can it simplify or make tasks easier?
  • Usability – Is it user-friendly?
  • Uniqueness – Is it offering something other apps don’t?
  • Accessibility – What is the load time? Do users quickly and easily find what they’re looking for?

Collaborate With the Design & Dev Teams Early On

Involve your UX designer and developers as early as possible into the project. Working with both a UX designer that has an understanding that software is not simply code and a developer team that appreciates the expertise of beautiful and intuitive design can lead to stunning results. The whole team can work together to perfect the app’s user interface.

Ensure There Is a Primary Action and a Next Action On Every Screen

Screens can often be cluttered with too many actions to take, but lower functionality pages often leave users with no next action to take to continue their journey. This also brings up the topic of habituation. Habituation is when a user becomes trained to do a certain action they no longer need to think about it. Using consistent actions to create a focused experience will allow users to quickly and easily explore the app.

Know Your Audience

When designing your app, you need to understand who you are designing it for. App developers tend to be in the age range of 30 to 55, also known as Generation Xers. The two largest demographics in the U.S. are Baby Boomers (55+) and Millennials (Under 30). Knowing who will be using your app can help you build what they want: an intuitive app that meets a need.

Want to take your app to the next level?

We would love to hear from you! Visit us at www.interknowlogy.com or email us at curious@interknowlogy.com

InterKnowlogy and Microsoft partner to bring a new technology era for 2016 Iowa caucuses

635501804068694446-2016IowaCaucus-720x163
The Iowa caucuses provide a unique forum for civic engagement.

Every four years, Iowa commands the national spotlight. As the first official contest in the nominating process for president of the United States, candidates from both political parties and the citizens of the State of Iowa take the process seriously.

At Microsoft, we understand that technology can empower people and organizations to discover creative ways to address enduring problems and opportunities. That is why we are honored to support the 2016 Iowa caucus via a new, mobile-enabled, cloud-based platform that will facilitate accuracy and efficiency of the reporting process.

We are excited to work with such a great partner to bring the citizens and political parties in the state of Iowa the technology, data, and business intelligence to help make the caucus reporting efficient, accurate, and secure. Watch this video by Microsoft to learn more about the exciting project!

View the press conference here (live from Des Moines on Friday, June 5 at 1:30 pm PT).

 

The big news was also picked up on several media outlets which was very exciting for us! Checkout these press releases for more coverage.

CNN’s Coverage

Washington Times

Iowa Democratic Party

Iowa Republican Party

Microsoft On The Issues

 

We’d love to know your thoughts on this article or if  you have any questions, feel free to drop a line.

Crossing The Finish Line With IMSA

This slideshow requires JavaScript.


As Tim called out the other day, we recently went live with a brand new mobile experience for IMSA, and I had the privilege of leading the engineering team on the project here at IK. The scope and timeframe of the project were both ambitious: we delivered a brand-new, content-driven mobile app with live streaming audio and video, realtime in-race scoring results, custom push notifications and more, across all major platforms (iOS, Android, and Windows), with custom interfaces for both phone and tablet form factors – all in a development cycle of about twelve weeks. It goes without saying that the fantastic team of engineers here at IK are all rockstars, but without some cutting-edge development tools and great partnerships, this would have been impossible to get across the finish line on time:

  • Xamarin allowed our team to utilize a shared codebase in a single language (C#, which we happen to love) across all of our target platforms, enabling massive code reuse and rapid development of (effectively) six different apps all at once.
  • Working closely with the team at Xamarin enabled us to leverage Xamarin.Forms to unlock even further code-sharing than would have been otherwise possible, building whole sections of the presentation layer in a single, cross-platform XAML-based UI framework.
  • On the server side, our partners at Microsoft’s continued world-class work on Azure made utilizing Mobile App Service (née Azure Mobile Services) a no-brainer. The ability to scale smoothly with live race-day traffic, the persistent uptime in the face of tens of thousands of concurrent users making millions of calls per day, and the ease of implementation across all three platforms, all combined to save us countless hours of development time versus a conventional DIY approach to the server layer.
  • Last but not least, being able to leverage Visual Studio’s best-of-breed suite of developer tools was essential to the truly heroic amounts of productivity and output of our engineering team at crunch time. And Visual Studio Online enabled the Project Management team and myself to organize features and tasks, track bugs, and keep tabs on our progress throughout the hectic pace of a fast development cycle.

The final result of this marriage between cutting-edge cross-platform technology and an awesome team of developers is a brand new app experience that’s available on every major platform, phone or tablet, and this is just the beginning – we have lots of great new features in store for IMSA fans worldwide. I’ll be following up with a couple more technical posts about particular challenges we faced and how we overcame them, and be sure to check out the next IMSA event in Detroit the weekend of May 29-30; I know I’ll be streaming the live coverage from my phone!

IMSA

Somewhat lost in the jaw-dropping awesome announcements at Build 2015 was going live on a pretty incredible suite of Mobile apps for IMSA (The International Motor Sports Association).

What we built in such a short space of time is nothing short of awesome. From idea to production on the cutting edge: an app that runs cross platform, built by our team in a little over 3 months

clip_image002

The App Experience

The new app includes enhanced features such live broadcast television, IMSA radio, live timing and scoring of the race events, multiple in-car cameras feeds, live tweets and custom viewing options for fans to access straight from their device.

Through a collaborative effort InterKnowlogy partnered with IMSA and Microsoft to deliver a new, game-changing, multi-platform mobile experience for all of IMSA’s fans, both at the race and from wherever they watch the action. 

Technical Stack

VS and MSFT stuff

Visual Studio 2013

Visual Studio Online

Resharper 9

Windows 8.1 Universal Phone and Tablet

Azure stuff

Azure Mobile Services

Azure Redis Cache

Azure App Insights

Xamarin stuff

Xamarin.Forms 1.4

Xamarin Studio

Xamarin Insights

Other stuff

Akamai CDN

iOS 8

Android Kitkat

PhoneSM.CodePlex.com (HLS video streaming media)

JSON.net

Little Watson

Slack

Get the app on your own device

Download the Apps for your devices here:

Windows Phone

Windows Tablet

Apple

Android

About IMSA:

The International Motor Sports Association, LLC (IMSA) was originally founded in 1969 with a long and rich history in sports car racing. Today, IMSA is the sanctioning body of the TUDOR United SportsCar Championship, the premier sports car racing series in North America. IMSA also sanctions the Continental Tire SportsCar Challenge and the Cooper Tires Prototype Lites Powered by Mazda, as well as four single-make series: Porsche GT3 Cup Challenge USA by Yokohama; Ultra 94 Porsche GT3 Cup Challenge Canada by Michelin; Ferrari Challenge North America; and Lamborghini Super Trofeo North America. IMSA – a company within the NASCAR Group – is the exclusive strategic partner in North America with the Automobile Club de l’Ouest (ACO) which operates the 24 Hours of Le Mans as a part of the FIA World Endurance Championship. The partnership enables selected TUDOR Championship competitors to earn automatic entries into the prestigious 24 Hours of Le Mans.

Apple Watch versus Microsoft Band?

Most people seem to have interest in product comparisons even when the products have different intended audiences. These comparison usually are quite slanted in one direction due to the predilections of the author and provide little to no value because it is an opinion piece and not a true evaluation. So the discussion either resonates with you or feels completely ridiculous. With devices, authors seem to enjoy rating similar products through the same lenses even if does not make sense.  Why would a purely home consumer ever want a business centric device? The sway is trying to make other people see the world through your lenses instead of educating people to make their own choices that are right for them. This is completely true for phones, and it’s also true for wearables.

I have been using a Microsoft Band for several months, and my son just received his Apple Watch. My world is app UX design and architecture for large enterprise systems, which mixes in complex mobile and large screen immersive data visualization experiences. My son’s world is that he is attending Art Center College of Design for Interaction Design and is about to embark on a summer internship at Facebook. We both own Apple and Microsoft products. His center of gravity is in elegance and beauty of design, where mine is in elegance and performance of design. But what is also different is what do in life which changes what we need from our devices.

For wearables, my world is fitness. I want to know steps, how well I’m sleeping, and, most importantly, detailed tracking of my mountain biking adventures with gps, heart rate, and calorie burn. On a second tier of importance are the notifications through email, calendar, Skype, SMS, Twitter, and Facebook. With my activities (mountain biking) I’m often in dirt and rocks and slamming into things on the mountain, so I can’t worry about the health of my wearable. At the end of every ride I shoot off an email to my buddies that has these attachments:

band

This is my lense. For $200, I feel I am getting my money’s worth out of my Microsoft Band. I don’t care that it is ugly and is not distinguishable from other wearables. It’s a touch bulky but I’ve gotten used to it. I wish the battery life were a little better. Putting that into context, I can do a long 20 mile mountain bike ride only if I’m near a full charge. It is hitting pretty much every sensor on the device with GPS sucking up most of the juice. I wish there was a little better control of how notifications are managed. For example, I may just want my work email pushed to my Band and not my personal email and not have to define a discrete list of people. No device is perfect and we will all need to judge whether or not the compromise you would need to make justifies its continued use.

My son would never use a fitness band, and would never use a Microsoft Band. He decided on an Apple Watch long ago. The only reason he has an Apple Watch so soon is that he prioritized a quarter of his net worth (i.e. $800) to this device. He has never worn a watch before. I would never wear a device so expensive on a mountain biking ride. I hear my son extol the sapphire glass on the watch, but he also used to talk about the shatterproof gorilla glass on his Android phone before it lost a battle against cement. The leather band is from a European country and costs $150 alone, and it’s handmade and smells of new leather. It is the stainless steel version, and the buckle on the leather band is milled from same metal as the housing. When my son was telling me all this, my wife laughed at the look of complete and utter disinterest on my face. Clearly, vanity is important to him and a core reason why he purchased the device.

So after about 24 hours living with the Apple Watch, I asked my son of his first impressions:

applewatch

To hammer home my point about my son and fitness, his daily caloric goal was set to 300. The screen real estate feels small after using iPhones. The force touch is a new and interesting interaction being shared to other types of Mac devices, meaning a tap versus a pressed touch will behave differently. My impression is that although the interaction is interesting, it may not be discoverable for the less sophisticated users and if apps sporadically adopt it. The digital crown has been designed very well as animations that react to its movement feel natural and unforced. After apps are loaded, reactions to touch and the crown are smooth making the device not feel underpowered.

App navigation is not quite worked out because tapping the wrong app is too easy – you need to be precise and not move around while doing it. Siri integration seems snappy. My son asked Siri “how far is the moon” and she responded promptly. I had to ask Cortana on my Band the same question and the response was obtained in about the same time. Voice quality seems to have been nailed. It’s very Dick Tracy, but I had to explain to my son what that means. I could hear my son’s voice quite clearly, and he could hear my voice just fine through it. It was tested in quiet room in ideal conditions.

What Apple is doing with tactile feedback is interesting. Vibration patterns can be used for different things. For example, my son has CarPlay. When using navigation his Apple Watch starting vibrating with one pattern when he had to turn left, and another vibration pattern when he had to turn right.  My son enjoys the one less step he needs to do to buy things via Apple Pay on the Apple Watch. Not sure I like that it is even easier to spend money. Basically you just need to move the watch to the NFC reader and boom – you have been charged. The idea here is that since the device is strapped to your arm, it remains authenticated after supplying the passcode once per day. If the Apple Watch is taken off, it would need to be authenticated again before it could be used. Another interesting feature is a use of a meta language to communicate to your friends. Of course they will also need an Apple Watch. Through this digital touch, you can send vibration patterns to a friend. This could even be your heart beat. Also, you could send simple sketches to each other. He doesn’t know anyone yet with an Apple Watch so it remains to be seen how well used this feature will be.

So these are the lenses of my son. He loves new gadgets and loves the Apple ecosystem. Vanity is important to him. At $400, the device feels worth it to my son. But at $800, he says it doesn’t. The vanity effect seems to balance out the feeling that it is not worth it. It will be interesting to ask him the same questions he has worn the device for a couple of months. Perhaps he can sell it this week for 4x what he paid for it. Anyone?


Grace Under Pressure

I just wanted to give a shout out to the IK IMSA development team. Lots of unplanned long hours for gotchas related to how we are pushing iOS, Android, and Windows with Visual Studio, Xamarin, and Azure. Just on a phone call early this morning with the Technical Lead Mike Beverley and Project Manager Jaimie Hogan, and with everything coming to a head on our last push to stabilize the multiple platforms and get the builds into app stores, you can hear their calm confidence in their voices as they continue to triage issues and make amazing progress. True grace under pressure. Thank you to the entire IK IMSA development team, everyone at IMSA, and Microsoft for the continued hard work to make this a success.

Generational Recruiting – What Microsoft and Other Companies Can Learn From Facebook

My son is attending Art Center College of Design for Interaction Design, and was recently accepted for a summer internship at Facebook. I had a front row seat to the whole process. As someone directly involved in recruiting top talent for InterKnowlogy, watching how Facebook recruits was fascinating. What Facebook has embraced from head to stern is that they are only as good as their people. Many companies give this lip service, and they might actually believe it, but their actions don’t hold up. Facebook is continuously bringing in the best young minds into the fold through internships and putting their stripes on them as quickly as possible. Having name cache does not hurt – what college student wouldn’t want to work at Facebook? But that only helps to generate a long list of potential candidates. It’s the rest of what my son underwent that was most impressive.

At Facebook, potential interns are interviewed by qualified people. These are senior employees that are skilled in the domain of the intern, and not someone who has been delegated a task to interview a “lowly intern”. Qualified people can quickly evaluate and bond with potential interns, and it only serves to excite the students. At every step, Facebook was demonstrating how invested they are in the success of the interns. They have the foresight to prioritize the time of the right employees to engage with the potential interns. By doing this, Facebook can maximize their chances of having a highly qualified crop of new talent. My son felt that they took the time to get to know him and his interests, including the type of work he wants to do to best align his internship into one of the many Facebook teams. They never made him believe that they were doing him a favor by just talking to him – they made him feel special and valued.

After successfully completing this process, my son was given an internship offer beyond his expectations. Facebook is showering him with hardware and software (all the tools he needs to design apps). Their handling all aspects of the internship including how he will be travelling to Menlo Park, managing his lodging for three months, and even will be feeding him. Facebook, at every turn, is a sending a message that they value him and his time, and want him to value his experience. This investment has already produced a sense of loyalty in my son. By spending the money and taking the time and putting forth the energy to engage the right people, Facebook is setting themselves up to be able to hire top talent right from the best colleges year after year, generation after generation.

Want to make sure the best and brightest are continuously flowing into your company? See the above.


IMSA Mobile Apps – 4 – Xamarin Insights

Instrumentation – not exactly the first thing you want to think about when building cool mobile apps. But when you are testing an app on an Android, iOS, and/or Windows mobile device, and it blows up, wouldn’t it be nice if you had the forensics on the exception to be able to track it down? Regardless of the device being used? In a centralized web portal? That’s what Xamarin Insights is about. Although still in preview, it seems to have finally stabilized to allow us to start using it. This capability is similar to what Microsoft is offering with Application Insights but for cross mobile OS scenarios.

I spent some time with the lead developer on the IK IMSA team, Mike Beverley, chatting about Xamarin Insights, in this brief video:

Motion Energy (ME) – Microsoft Kinect in Healthcare

My mother passed away from cancer in 2011. During her final days, she was comatose and needed to be moved frequently. The movement was not enough to prevent bed sores, otherwise known as pressure ulcers. Pressure ulcers are caused by pressure that limits blood flow to skin and surrounding tissues. This experience is burned into my memory, and as a designer and technologist I wanted to explore ways to prevent others from suffering as my mother did. I was shocked to find that, in the United States alone, more than $10 billion is spent annually on the treatment of pressure ulcers. Could technology be used to help? Yes it can.

I led an internal R&D effort to explore the use of Microsoft Kinect camera technology to monitor the amount of motion over a period of time. The goal was to identify and measure how much motion was happening, and then report when a lack of motion occurred. We call the amount of motion detected Motion Energy, or ME for short. If a lack of ME is detected over a period of time, a nurse would be automatically notified with an ME event. As the exploration continued, I found studies related to patients falling out of their beds and hurting themselves. I began to wonder if a lack of motion could be monitored, could too much motion also be monitored? The answer is also yes.

ME_LackOfMotion

The studies related to patient falls are reporting that more than $20 billion is spent annually on the treatment related to injuries suffered from the fall. The same ME technology used for monitoring a lack of motion can detect when a motion spike is occurring. A nurse can be immediately notified about the event and an attempt can be made to prevent the patient from falling or further injuring themselves after a fall by providing immediate assistance.

ME_TooMuchMotion

This video shows an example of the core ME technology:

We envision a system that can be placed into a patient’s room that would passively monitor them. When too little or too much motion is detected, a ME event would be generated by the system. This ME event would be sent to a Cloud backed system to log the event. The on call nurse would be notified on a mobile device immediately of the ME event, and an action can be taken. From the mobile device, a video feed from the patient’s room could be viewed when the ME event occurred (e.g. for a patient’s fall). The video event can also be stored in the cloud with the ME event.

We believe a system like this could not only prevent suffering but improve patient satisfaction. Dollars saved from prevention would more than offset the cost of the system within the first year of use. If you would like more information about Motion Energy, or discuss other potential uses for Motion Energy, please contact us at curious@interknowlogy.com.