Crossing The Finish Line With IMSA

This slideshow requires JavaScript.


As Tim called out the other day, we recently went live with a brand new mobile experience for IMSA, and I had the privilege of leading the engineering team on the project here at IK. The scope and timeframe of the project were both ambitious: we delivered a brand-new, content-driven mobile app with live streaming audio and video, realtime in-race scoring results, custom push notifications and more, across all major platforms (iOS, Android, and Windows), with custom interfaces for both phone and tablet form factors – all in a development cycle of about twelve weeks. It goes without saying that the fantastic team of engineers here at IK are all rockstars, but without some cutting-edge development tools and great partnerships, this would have been impossible to get across the finish line on time:

  • Xamarin allowed our team to utilize a shared codebase in a single language (C#, which we happen to love) across all of our target platforms, enabling massive code reuse and rapid development of (effectively) six different apps all at once.
  • Working closely with the team at Xamarin enabled us to leverage Xamarin.Forms to unlock even further code-sharing than would have been otherwise possible, building whole sections of the presentation layer in a single, cross-platform XAML-based UI framework.
  • On the server side, our partners at Microsoft’s continued world-class work on Azure made utilizing Mobile App Service (née Azure Mobile Services) a no-brainer. The ability to scale smoothly with live race-day traffic, the persistent uptime in the face of tens of thousands of concurrent users making millions of calls per day, and the ease of implementation across all three platforms, all combined to save us countless hours of development time versus a conventional DIY approach to the server layer.
  • Last but not least, being able to leverage Visual Studio’s best-of-breed suite of developer tools was essential to the truly heroic amounts of productivity and output of our engineering team at crunch time. And Visual Studio Online enabled the Project Management team and myself to organize features and tasks, track bugs, and keep tabs on our progress throughout the hectic pace of a fast development cycle.

The final result of this marriage between cutting-edge cross-platform technology and an awesome team of developers is a brand new app experience that’s available on every major platform, phone or tablet, and this is just the beginning – we have lots of great new features in store for IMSA fans worldwide. I’ll be following up with a couple more technical posts about particular challenges we faced and how we overcame them, and be sure to check out the next IMSA event in Detroit the weekend of May 29-30; I know I’ll be streaming the live coverage from my phone!

IMSA MOBILE APPS – 3 – SCALING WITH AZURE

When taking your presence to mobile, there is always a scalability conversation that quickly occurs. This is especially true when the systems you need to access are on-premise. Your on-premise systems may have never been designed for the user load you would add with mobile apps. Additionally, your on-premise systems may not even be exposed to the internet, introducing a whole set of security complexities that need to be solved. In the case of IMSA, we are relying on services already exposed to the internet, so one less set of issues to manage.

Through our build experiences with Azure projects such as CNN, we knew several considerations would apply. The services referenced below are those supplied directly from IMSA:

  • How many users would concurrently be accessing the services through the mobile apps?
  • What is the latency for the service calls?
  • How much effort is it for the service to generate the data?
  • How often are the services taken down for maintenance? For how long?
  • Will the services change over time as backend systems change?

These are relatively simple questions, but they serve to shape the approach you take to scale. To provide the best possible mobile experience, we envisioned a brokering capability to be served by Azure. All mobile apps across iOS, Android, and Universal Apps would access this brokering layer for data access. This brokering layer is caching data from IMSA services for fast access.

There is immense flexibility in how you shape solutions in Azure for scale, particularly around caching. Ultimately the purpose of data caching is to minimize the number of trips to the backend services. There can be instances where the backend services are so expensive in time and resources to call that the architecture must do everything possible to minimize the user paying the price of waiting for that call to complete. In this case, Azure can be setup to actively keep its cache fresh and minimize the amount of calls to the backend services. Mobile apps would then always have a fast and fluid experience and never feel slow, and a company would not have to worry about putting a massive amount of resources for scaling up their backend services.

Fortunately, this was not the case for us and the IMSA backend services. The backend services are responsive and data is small per service call. Also, it is not expensive for the backend services to produce the data. Even in this case, there is benefit to leveraging Azure. IMSA race events are at key moments in time, and traffic heavily spikes around each event. It is not beneficial to have hardware laying around mostly idle 90%+ of the time waiting for the spike in usage. Additionally, the IMSA services could be taken down briefly for maintenance. Using Azure for brokering calls still has merit because capability can be scaled up and down around the IMSA events. There will be minimal additional load put on the backend services because Azure is doing most of the work of serving data to the mobile apps.

The approach we took for IMSA relied on a combination of HTTP output caching (via ETag) and Azure Redis Cache all within Azure Mobile Services. Basically, when a mobile app makes a request from an Azure service for the first time, no ETag is present because our services did not already generate it. However, we have the URL and parameters passed in, which forms a unique key to the requested data. Redis cache is checked to see if the data is present. If the data is present and is not expired, then the cached data from Redis is returned. If the data is not present or is expired in Redis, then Azure makes the request into the backend IMSA services, puts the response into the cache, and returns it to the calling mobile app. An ETag is generated with each response, so if the mobile app requests the same data again that ETag is supplied. This is informing our Azure services that the calling mobile app has data already, but is not sure if the data is still valid. The benefit of supplying the ETag is that we can check whether or not the ETag has expired, meaning the related data in cache has expired. If it has not expired, an HTTP 304 is returned which is much lighter weight response than if the cached data was returned.

There is a downside to this approach. When simultaneous requests are made for the exact same data (based on the URL and the parameters passed in) at the exact same moment, each request could do the full trip to the backend IMSA services. If IMSA had millions of users during each event, we would prevent this by doing data locking within Redis, but they do not so the extra engineering to prevent this is not warranted.

Through this technique, we have set ourselves up to be prepared for tens of thousands of new users at each event without bringing the IMSA services to their knees.

IMSA Mobile Apps – 2 – Planning For Maximizing Code Re-Use Across iOS, Android, and Universal Apps

While we were busy thinking through the interaction design elements of new IMSA mobile apps, we knew we were going to have to build six apps (iOS, Android, Universal Apps for phone and tablet). The app architecture we chose to follow for this is Model-View-ViewModel (or MVVM). Our mission was to maximize the amount of code sharing across all implementations of the app. The more code sharing we could do, the less code we would have to develop for each platform. Less code means less time to develop and less to test, making the aggressive schedule more achievable.

The Model layer contains the business logic and data that will drive the IMSA app. Data would be served through a scalable cloud infrastructure being constructed for the mobile apps. Regardless of mobile OS, the business logic and data will remain the same. How we access the data in the cloud and retrieve would also remain the same. These layers are devoid of any user interface elements are a logical candidate for re-use across all the mobile operating systems. Perfect – one layer to write one. But we want more.

We were suspicious that the View layer would be so unique across mobile operating systems that the ViewModel layer would not be re-usable. The ViewModel layer is responsible for binding the Model layer (the business logic) to a View (the user interface). Remember we are talking about code sharing across iOS, Android, and Universal Apps – this have to be do so different that writing a consistent and shareable ViewModel layer would not be possible – right? Wrong! After some initial prototyping we were pleasantly surprised. The path we have chosen is going to allow us to use the same code in the ViewModel layer across all operating systems.

From our early calculations, thanks to Visual Studio and Xamarin, we are predicting to see about 75% code re-use (of the non-generated code) across all the implementations! Excellent news for the developers and project manager. We’ll dive into code examples in an upcoming blog, but next we’ll discuss our approach with Azure.  Also, this video has additional information for code re-use with Xamarin.

IMSA Mobile Apps – 1 – Architecture Design Session

The IMSA Mobile Apps project is currently in flight and we are actively working on building this cross platform/cross OS solution. This article is the first in a series of blogs discussing the details of the project, and we’ll be actively trying to catch up to the current day as we are busy building towards the Laguna Seca race.

Back in the first week of December 2014 we flew out to Florida to visit the IMSA team with Microsoft in Daytona Beach. Microsoft was hosting an Architecture Design Session, or ADS for short, to flesh out features of the solution. It became quickly apparent that the solution was layered and complex. Many features discussed have become part of longer product roadmap as IMSA is committed to providing the best experience possible to their fans. Also, it should be noted as in all ideation sessions, some ideas discussed are put deep down in the feature backlog.

I am certain that some would ask why IMSA involved Microsoft. This is a mobile app – what does Microsoft know about building mobile apps across iOS and Android? Well, it turns out quite a lot. From past projects, we already knew the tooling we get with Visual Studio and Xamarin allows us to build amazing mobile apps across all platforms and OS’s. The other side of the coin is the plumbing we get to integrate into cloud infrastructure. This app needed to scale across the huge IMSA fan base during live events. From past projects we knew how effective we could be building scalable mobile apps with Azure. So to IMSA and to us, involving Microsoft made perfect sense.

In the ADS, some of the interesting features starting popping up:

The app would need to change shape depending on whether or not a race is live or not. We thought treating the app almost like the NFL Now app would be interesting. There could be something interesting always to watch on our app, regardless if an event is live or not.

IMSA radio is a live audio stream. The app would need to deliver this feed just like other integrated audio content on your device. So turning on IMSA radio, putting your headphones on, and then your device in your pocket should be as natural as playing music.

Using the device’s GPS, if the race fan is at the event the app should respond differently than if the person were elsewhere. When you are at an event, what you are interested in is different than when you are not.

Telemetry information from the cars. It would be just awesome to watch your favorite car at the event or at home and see all the g-forces they are pulling when they are flying around the corners.

IMSA services to content and structured information are not scalability to a mobile play. A cloud infrastructure would need to be placed in front of the IMSA services so content could be cached and served more quickly.

 

After the ADS we went home and decomposed all the features while looking at the schedule. We needed to pick a race event to target deployment. We had a lot of homework to determine our approach. In this next blog we will be discussing how we were planning on maximizing code re-use across all platforms and OS’s.

How to use PhoneGap API + iOS, Android

Last blog post I showed how to use PhoneGap’s API with Windows Phone 7. For this blog post, I will use the same web files (HTML and Javascript files) and create an iOS and Android application.

How to setup dev environment?
First thing to get started is visit PhoneGap’s website where they have instructions on how to setup your environment to create an iOS and Android app with PhoneGap. Just select your platform and the instructions will update update on the page for that environment.

After you have your iOS and Android environment setup using Xcode and Eclipse, copy the web files into the “www” folder for each project. Below is a screen shot of the project navigator or package explorer inside Xcode and Eclipse.

Package Explorer in Eclipse for Android Project Navigator in Xcode for iOS

Since I’m reusing the web files, all the code is exactly the same across the Android, iOS and Windows Phone 7 app. So, I’ll just show a few of the pages/screens in the iPhone simulator and Android simulator.

Device Screen (Android) Device Screen (iPhone)
Notification Screen (Android) Notification Screen (iPhone)

Click here to download the iOS example project.
Click here to download the Android example project.

How to use PhoneGap to port quickly your web app to a native iOS and Android device

This blog post will show how to take a web site written only in HTML5, CSS3 and JavaScript and quickly develop an iOS and Android mobile application. In order to do this, I will use Adobe PhoneGap, a mobile web frame to build cross-platform mobile applications. As mentioned in my previous blog post, PhoneGap is an HTML5 app platform that allows you to author native applications with web technologies. Even though my blog post will only show how to develop an iOS and Android application, PhoneGap also supports developing BlackBerry, webOS, Windows Phone 7, Palm devices, etc. In a nutshell, you’ll be using a PhoneGap wrapper that contains your web-based HTML, CSS and JavaScript code. Also, you will gain access to many of the device’s native features such as the compass, camera, the contacts list, accelerometer, etc.

PhoneGap introduced last year PhoneGap Build which offers a cloud-based service that takes your app written in HTML5, CSSe and Javascript and sends you back app-store ready apps for iOS, Android, Palm, Symbian and the other mobile platforms. Check out pricing and other details on their web site.

Below is a general process involved in getting setup using PhoneGap and developing an application:

1. PhoneGap has a great Getting Started tutorial for all the mobile platforms. Download the PhoneGap tools and the specific set of tools for the platforms you’re working with. For example, download Xcode and iOS SDK if you plan on porting your web site to an iOS application. Download the Android SDK and Eclipse if you plan on making an Android application.

2. Install all the platform tools and PhoneGap to your existing environment.

3. Start developing your web application and test with a web browser of your choice. I normally use Visual Studio to write my web application and test it using Safari, Chrome or any webkit supported browser. Since these are the types of browsers on smartphones. I would take advantage of Visual Studio’s IDE in debugging my javascript code or Web Inspector in Safari or the dev tools found in FireFox or Chrome.

4. Once you’re ready to port it to a mobile application, you will put all your web files within the www folder. For Xcode, all you need to do is create a new project using the PhoneGap template. Once the project is created, immediately compile the project. Once the project is done building, a www folder will be created in your project folder. Add the www folder into the project navigator. This is where you will house all your web files (HTML5, CSS3 and Javascript files). Run your project and you should see your web page displayed in the iPhone or iPad simulator.

5. For Android, install Android SDK, Eclipse and then the Android plug-in for Eclipse. Create an Android project in Eclipse and follow these steps to modify the project to support PhoneGap. Then run the project and select an Android Virtual Device (AVD) to use as the emulator to display your application.

6. Steps 4 and 5 can be done for you using PhoneGap Build. Just upload your web application files using their Cloud service and they will send you back app-store ready packaged files for the mobile platforms they support.

7. A noteworthy caveat is you will have to tweak some of your web language code depending on which mobile platform you choose. Not all mobile devices are similar in hardware but PhoneGap does an excellent job in pointing out issues through their API Docs section on their web site.

For step 5, here are some links to great tutorials on how to Get Started in developing for Android:
1. Getting Started With Android Development Using Eclipse
2. Getting Started with Android on a Mac
3. Running Android SDK Examples Using Eclipse

So below is a web page example written only in HTML5, CSS3 and Javascript. The HTML consists of an input element and a button. Using JavaScript, it dynamically shows in an unordered list the input values you submit. If you click any of the list items in the unordered list, it will make a call using the Twitter Search API and return the latest tweet with that keyword as an input parameter.

HTML elements:

- (void) prepareForSegue:(UIStoryboardSegue *)segue sender:(id)sender
{
    <body>
	<div id="main">
        <h1>Search Twitter</h1>
        <form id="foodForm">
            <input type="text" id="foodName" placeholder="keyword" />
            <button id="submitFood">Tell Us!</button><br>
        </form>
        <ul id="foodList">
        </ul>
        </div>
   </body>
}

JavaScript code event handler when DOM is loaded:

- (void) prepareForSegue:(UIStoryboardSegue *)segue sender:(id)sender
{
    document.addEventListener("DOMContentLoaded", function () {
            //navigator.notification.alert("I think today will be a great day.", function() {}, "Great Day");
            // uncomment below code to clear items in LocalStorage
            //window.localStorage.clear();
            // <ul> instance variable
            var foodList = document.getElementById('foodList');
            // textbox instance variable
            var foodField = document.getElementById('foodName');

            // length of food items in localStorage
            var l = window.localStorage.length;
            var i = 0;
            var storedFoodName;


            // add food item to <li> child
            function addNewFoodItem(foodName) {
                var newFoodItem = document.createElement('li');
                newFoodItem.innerHTML = foodName;
                foodList.appendChild(newFoodItem);
            }

            // loops through localStorage for food items and calls addNewFoodItem function
            for (i; i < l; i++) {
                storedFoodName = window.localStorage.key(i);
                if (storedFoodName.match(/^food[.]/))
                    addNewFoodItem(window.localStorage.getItem(storedFoodName));
            }
}

Javascript code that gets handled when the button is clicked:

document.getElementById('foodForm').addEventListener('submit', function (evt) {
                evt.preventDefault();
                var newFood = foodField.value;

                // creates a foodKey value with food.[length]
                var foodKey = "food." + (window.localStorage.length + 1);
                // calls addNewFoodItem function
                addNewFoodItem(newFood);
                // saves foodKey and input food to localStorage
                window.localStorage.setItem(foodKey, newFood);

                // clears foodFied variable
                foodField.value = "";
                return false;

            }, false);
        });

Javascript code that calls the Twitter Search API remote service using the XMLHttpRequest object and parses the JSON data returned. It displays the latest tweet in a notification alert after you click on the ListItem in the unordered list.

// callback function to get the latest tweet
        function getLatestResult(JSONstring) {
            var twitterPayload = JSON.parse(JSONstring);
            var latestResult = twitterPayload.results[0];

            return latestResult;
        }

        // performs a get request for url
        // passes the response text to callback
        function getXHR(url, callback) {
            var req = new XMLHttpRequest();
            req.onreadystatechange = function () {
                if (this.readyState == 4) {
                    if (this.status == 200 || this.status == 0) {
                        callback(this.responseText);
                    } else {
                        console.log('something went wrong');
                    }
                }
            }
            req.open('GET', url, true);
            req.send();
        }

        // event handler for the clicks on <li> elements using event delegation to catch all of these events.
        // We listen on the document element, and then, if the target of the event matches one of our list items, 
        // fires the event handler
        document.addEventListener("click", function (evt) {
            if (evt.target.tagName == 'LI') {
                // gets the selected <li> element
                var foodSubject = evt.target.innerHTML;
                var foodSearch = encodeURIComponent(foodSubject);
                // generates twitterURL for Twitter Search API
                var twitterUrl = 'http://search.twitter.com/search.json?q=' + foodSearch;

                // calls getXHR function to get latest tweet of selected <li> element 
                // then calls callback function to pop-up an alert msg with latest tweet
                getXHR(twitterUrl, function (response) {
                    var latestTweet = getLatestResult(response);
                    var msg = 'Latest Tweet about ' + foodSubject + ' from ' + latestTweet.from_user + ': ' + latestTweet.text;
                    //navigator.notification.alert("PhoneGap is working")
                    alert(msg);
                })
            }

        }, false);

Project Navigator in Xcode

Project Navigator in Xcode for PhoneGap project

iPhone Simulator of Search Twitter

iPhone emulator of PhoneGap application

Project Explorer in Eclipse

Project Explorer of Android project in Eclipse. Notice the /assets/www folder and /libs folder with PhoneGap related web files

Android Virtual Device

Android Virtual Device running the web page

PhoneGap is good at helping developers leverage their experience at building web applications using web standards, HTML, CSS and JavaScript. If you know web standards, you’ll experience few problems while working with PhoneGap. All you need to learn is how to use the PhoneGap API. Once you become familiar with the PhoneGap API, you can quickly take advantage of accessing the device’s camera, pull up the contacts or work with the accelerometer or compass.

As the web site example above shows, if you need to connect your application with a remote web service, you can easily bring in tools like jQuery to create powerful Ajax handlers. Or as I demonstrate in how to use XmlHttpRequest objects.

As I pointed out in step 7 above, just because you code a web application using PhoneGap and it works on an iPhone device, it doesn’t automatically mean that it will work on other devices. You will have to test and tweak for the other devices that are supported.

If you want to port to multiple devices, you will need separate environments for each wrapper. For example, you won’t be able to maintain your Android PhoneGap wrapper with Xcode. I used my Mac and have installed Xcode and Eclipse but I make sure I keep them in separate environments.