My aim is to load a good clean database of South African cities linked to suburbs (I’ll write another post about that later) something a lot of people seem to want but no one seems to be doing very well, and those that have very clean data don’t open it up. I also wanted a relational table with SQL Server Geography datatypes, simplegeo or openstreetmap aren’t close to a perfect dataset. Ideally I wanted to load the geographic polygons that define the cities and the suburbs, I also want to load the bounding boxes, centroids and radius that encapsulate the suburbs.

I know there is some open street map data out there but Shaun Trennery, who I trust implicitly and is the tech brain behind safindit.com pointed me at YQL. Yahoo’s data seemed the cleanest and I decided to use their data to load the suburbs. (By the way, YQL is amazing: you can perform crazy queries like find all sushi restaurants in San Francisco “select * from local.search where query=”sushi” and location=”san francisco, ca” and Rating.AverageRating=”4.5”” awesome hey?) I have a JSON file of WOEIDs (Where on Earth ID) that define South Africa, this can be retrieved with a simple YQL query or download it here: LocationsZA.json. I wrote some code to parse these and load them into a SQL Server table that I’d designed (again – that post to follow).

I was having a problem loading Yahoo WOEID polygons as Microsoft.SqlServer.Types.SqlGeography objects in SqlServer2008. This fails on insert for certain polygons with “24200: The specified input does not represent a valid geography instance” exception which, typical of Microsoft, will take you here, and well, that doesn’t tell you anything you don’t’ already know (a side note, System.Data.SqlClient does not support the Geography type so you have to make a reference to Microsoft.SqlServer.Types).

I wanted to investigate if these polygons were in fact polygons. I also want to visualise the other elements of the WOEID (centroid, bounding box, radius etc) so I wrote the following ugly web form (let me reiterate I hate web forms and NML only build ASP.NET MVC but this was a quick prototype to see what the polygons looked like on google maps mostly taken from this post).

Figure 1 as you can see only slightly adapted from the codeproject. (click to enlarge)

The following image illustrates Yahoo’s idea of the Constantia WOEID Polygon.

As you can see it’s pretty obvious why the polygons won’t load, they aren’t true polygons. So that’s my warning don’t use yahoo polygons for suburbs rather uses bounding boxes where you can be sure that your geospatial queries will return trustworthy results. I’ve also noticed that in some instances Yahoo doesn’t even store the polygon.

So I had another chat to Shaun Trennery and we are keen to open this data up, and possibly build a geographic wiki app that allows anyone to modify the polygons, the administrators will have the final say. This will allow us to add suburbs yahoo isn’t aware of like those being defined by the Name Your Hood guys (if you know how much I initially disliked the Name Your Hood project you’ll find this ironic) and also define parent suburbs, for example Constantia is the parent suburb Bel Ombre, Nova Constantia, Constania Hills, Klein Constantia , Rust en Vrede and Alphen for example.

I’d be interested to get feedback from the GIS gurus on the best sources of Country/Region/City/Suburb/ information. Please also let me know if you are keen to collaborate on this mini project.

Build something you need and others will need it too, right? I must lose thousands to the tax man because I’m too lazy to keep a log book of all my business trips. So, the juniors at New Media Labs have built an iPhone application that logs your business trips to a web site (http://triplogger.info) via a web service. At the end of the financial year you can download a CSV of all your business trips and submit to SARS.

We would love you to try the app out. Hopefully you find it useful too. Feedback would also be highly appreciated.

Steps:

1) Download the “Business Trip Logger” from http://itunes.apple.com/za/app/business-trip-logger/id373066501?mt=8 or search for “Business Trip Logger” in the app store and install the app.

2)      The app requires you to register on the website. This is easiest done directly from the phone – a browser will be launched from the app. All that is required is your email address and a password (the phone id is already sent through to the website). Registration is necessary so that we know which incoming messages (trips) are yours and so that you can download your CSV from the website to submit to SARS at the end of the year. Also, local storage on the phone was not considered an option because the system allows for more than one phone per account. Also if you lost your phone, and we were using local storage, you would lose all your trips too.

3)      Start logging all your trips. SARS require a client name, reason for trip and start odometer and end odometer reading.

4)      All client names and trip reasons are stored so you can select those from the select boxes on similar or repeat trips. You can also delete and edit these in the traditional iPhone fashion.

5)      The end odometer reading must obviously be greater than the start odometer reading.

6)      If you have started the trip the app will expect you to close (end) the trip.

7)      You can also add and edit trips on the website.

Since we are focusing on iPhone, Blackberry, Android, Symbian and Windows Mobile (6.1 and 6.5 and 7) expect to see this app built for a number of different phone operating systems (good practice for our juniors (Chris Goosen who finished the iPhone app and Ashraf Vanker who built the website and the web services).

Why didn’t we store the whole trip’s GPS points?

At the time when we started building the app, multi-tasking wasn’t possible so we figured we’d end up with a whole lot of broken trips that would leave questionable results.

There are lots of trip loggers on the app store, why did you make another?

Practice for our juniors. This app is specifically designed for submission to SARS.

Last thoughts: we have noticed some improvements we could make to the usability.

]]> http://www.newmedialabs.co.za/custom-development/business-trip-logger/feed 1 http://www.newmedialabs.co.za/silverlight/creating-the-adora-crm-with-global-vision http://www.newmedialabs.co.za/silverlight/creating-the-adora-crm-with-global-vision#comments Tue, 01 Dec 2009 08:46:22 +0000 Richard Schroder http://www.newmedialabs.co.za/?p=855 Adora is a marketing CMS platform that is currently being developed by Global Vision and is being built to replace their highly successful current offering, Eureka.  The driving force behind Adora is communication, not just marketing, through all effective communications channels.  Using Adora, marketers will be able to communicate effectively with consumers and be able to maintain a complete picture of consumer relationships to their respective brands.

As Global Vision made the decision to develop the Adora front-end using Microsoft Silverlight (at the time a very brave move – Silverlight 1.1 was VERY limited compared to Silverlight 3!), New Media Labs was asked to provide some assistance in the development process.  With full-time in-house User Experience (UX) designers and interface designers from Stonewall+, this project has really brought to light the power and flexibility of the Silverlight platform.

Design of Adora

The application is built along a classic Model-View-ViewModel (M-V-VM) architecture and utilises a custom-built framework for this that supports all the required elements.  We had a number of technical challenges to overcome along the way, mainly related to data storage and using the same Domain model on both the Client- and Server-side.  Most of these issues are posts in their own right and will be covered in the near future.

Adora consists of many layers, each with its own set of technologies.  Starting from the client and working our way to the backend:

• Client
  Silverlight 3, utilising the Silverlight Toolkit and numerous custom-built controls.
• Services
  The application services are provided by WCF services that are hosted on the server.  These are communicated with using a custom-built data-contract serialiser.
• Data storage
  Data storage is provided by SQL Server 2008 and data goes through many stages of filtering and matching to ensure maximum data integrity.
• Messaging
  A messaging and scheduling engine resides on the server side to handle all email and SMS communications with consumers.  Messages received are linked to the consumer’s profile.

The Development Process

The goal is to deliver a product that is not only scalable, flexible and fast but is also pleasant to use with minimal or no user training.  With a team consisting of UX designers, developers, graphic designers and QA testers, we are continuously and iteratively tweaking the design.  Using an Agile approach has allowed us to make fast-paced design changes whilst ensuring that the end result not only looks good, feels good and performs well but also matches the expectations of the Product Owner.

We currently employ a mix of various established practices – test driven development, peer reviews, daily stand-ups and whiteboard sessions are some of the things that we do every day.

Interface Design

Adora’s interface borrows heavily from concepts used by Windows Explorer to allow users to work in a familiar environment.  For example, “folders” represent brands and “files” represent elements such as email lists, creatives or even communication schedules. 

Adora Explorer interface

Wireframe designs are created by our UX designers and the initial development work to implement functionality is done by the developers and testers.  Whilst this is taking place, the interface design is UAT-tested by the UX designers – the interface is then sent to Stonewall+ for “polishing” and styling.  Due to Silverlight’s dynamic binding model, it is simple for the developers to drop the finished interface on top of the functional elements built.

All work with consumer populations is graphical – that is, instead of reams of confusing numbers that may not appear to have any significant proportions or relationships, Adora presents interactive graphs.  A user with little or no SQL knowledge is able to build up complex queries very quickly with this system and it provides a “quick-glance” overview of the consumers that the user is currently communicating with. 

Marketing List builder interface design

Adora Query Language

Central to Adora’s success is our own query language that we have developed: Adora Query Language, or AQL.  With a syntax that is similar to SQL, AQL allows developers to retrieve consumer related information using queries that are business oriented.  If that doesn’t excite you, how about this:

SELECT firstname, lastname, age FROM brandname WHERE gender=’male’ AND age>25

The result of this query is a list of consumers within the “brandhouse” brand that are male and over 25 years of age.  A complete subset of SQL is supported, including ORDER BY, GROUP BY, INTERSECT, UNION and many more.  On top of this, we have our own extensions to the language that allow us to apply pagination to data results or to even group data into ranges.  The result?  Our developers can focus on working with data, not on retrieving data.  It is envisioned that future versions of Adora will allow 3^rd party vendors to create add-ins to the Adora system using AQL.

What’s next?

Still to come – details on the technical challenges faced and more screenshots!  Beta 1 is scheduled for release in January 2010. 

Aurecon is developing a whole suite of new Incident Management/Emergency Response Software for clients in areas such as Emergency Services, Disaster Management and Policing.

The Problem

New Media Labs came on board specifically to design, develop and implement a system that:

• can capture all the necessary information needed by an emergency response unit in order to respond to an incident more quickly and efficiently.
• has an easy to understand and operatable Graphical User Interface.  The more “Minority Report” it looked the better, and as the client once put it, “if we had to put Homer Simpson in front of it, he’d know how to operate it.“
• is GIS enabled, so that operators can pinpoint locations easily and relay co-ordinates to responding units.
• is easily deployed across various control stations across the country, and ultimately the world.

It just so happened that one of Aurecon’s existing clients operates a large CCTV control centre in the City of Cape Town and was in the market for such an application.

The Solution

Designed for a call centre/control station environment, our system presents the user with a graphically rich, simple and easy to use User Interface.  Aurecon was adamant that not only should the system work well, but it should also look great, as this would be a large selling point over competing systems.

We implemented a container based UI where functionality is broken up and grouped into several independent, movable, expandable containers/panels.  These panels allow the user to view and work on multiple aspects at once, as well as the ability to drill into one specific section and focus all their attention on that one task.

One of the key aspects to the system is a map - the map gives the user the ability to quickly navigate to a street, intersection, and popular landmark or CCTV camera location.  Designed specifically for a CCTV control station, the system contains a list of hundreds of operational CCTV cameras in and around the City of Cape Town region.  If an operator identifies something on one of their camera monitors, they then pinpoint the location on the map and capture necessary information about the incident so units can be dispatched accordingly.  Ultimately, the information captured in the system is also reported on.  Existing incidents are indicated by a coloured dot on the map depending on the priority of the incident.  CCTV cameras are marked by camera icons.

How did we do it?

In a nut shell, Start Lite, as the system is called, has a 3 tier architecture comprising of:

• a SQL Server 2008 backend
• WCF SOA middle tier
• Silverlight UI

Working closely with a newly established Aurecon Development Team, we were on site full time for the duration of the project.  We introduced their team to the SCRUM methodology and went from initially running all morning stand ups and sprints to SCRUM being fully integrated into the Aurecon staff and their project management processes.  With SCRUM, New Media Labs were able to keep up with the demanding deadlines of the project and finish Start Light on time and successfully.

Database and Web services

At New Media Labs, we believe that applying tried and tested patterns and practices is key when building software.  Not only does this accelerate design and development, but it reduces potential risk by ensuring that you’re applying industry standards that other software developers out there are doing as well.

On this particular project we decided to use 2 Microsoft Patterns and Practices: Repository Factory and Web Service Software Factory Modeling Edition.

The Repository Factory is a guidance package that automates creation of entity classes that map to database tables and repository classes to read and write those entity classes.  With it we were able to quickly build our data access and business logic layers.  The Repository Factory creates Business Entity classes from your database schema, generates CRUD (Create, Read, Update and Delete) stored procedures, and ultimately links the two together into ode that is easy to understand and maintain.

With the Web Services Software Factory Modeling Edition, we were able to sit with the project owner and business analysts and graphically model the web services in full view with the use of a projector.  These sessions added incredible value as the whole team (not only developers) was involved and we could brainstorm and discuss more freely.  Once Modeled, the service contracts, message contracts and data contracts are all generated into code that is easily understandable.

With the Repository Factory and Web Service Software Factory, New Media Labs were able to maintain a high project velocity whilst simultaneously getting the Aurecon Development Team up to speed with the architecture, patterns and practices.  Using the tools is trivial and the resulting code clear and understandable.

User Interface

Why Silverlight?

• Silverlight is a cross-browser, cross-platform and cross-device web technology, perfect for Aurecon’s need for a system that is quickly, easily and cheaply deployed.
• Deployed on an IIS 7 web server - all the client needs to do is download the Silverlight plugin and start using the system.  It’s that simple!
• With Silverlight, we were able to rapidly develop a visually rich user interface and experience.

Yes, ok but why not Adobe Flash?

• Aurecon are already heavily Microsoft with various systems and backend processes already built on Microsoft platforms.  With Silverlight leveraging the .NET framework, we were able to create a system that can wield a rich user experience as well as support and easily integrate with other Microsoft back ends.

As for the Mapping functionality, we used the ESRI ARC GIS API for Silverlight component on top of Aurecon’s very own ESRI Arc GIS Server.  To avoid taking credit where it’s not due, Aurecon own and manage their own GIS data, New Media Labs just got the map to do all the funky things with the data we wanted it to do on screen.

Worth Mentioning

- Login with ASP.NET Authentication Services

• ASP.NET already comes with a huge array of out-the-box Membership, Role and Profile functionality, so why recreate the wheel?  Start Lite leverages this Membership and Role Functionality.  One can simply expose the ASP.NET Authentication system as a WCF web service with a few configuration settings.

- Blacklight

Multiple Panels Open

One Panel Open

• For draggable collapsable panelscontainers we utilized the open source Codeplex Blacklight Control suite and customized some of their controls to suit our needs better.

- ViewBox Control

• We decided to use the ViewBox Control which comes with the Silverlight Toolkit to handle the resizing of our various panel and container contents.  Manually resizing each UI element within a panel’s content when the panel is either maximized or minimized would have been cumbersome to do, and this handy control saved us a lot of time and effort.  It works wonders!

- Announcement and Notifications Engine

Announcement Functionality

• It wouldn’t be New Media Labs style without a social media element to it - we custom built an announcements and notifications engine.  Announcements let a user post important messages for others to see, whilst notifications get fired when important things happen within the system such as the creation of an incident.

- AutoCompleteBox Control

• In Start Lite, we tried to guide the user as much as possible when entering information. The AutoCompleteBox control is another cool control that comes with the Silverlight Toolkit (http://silverlight.codeplex.com/).  For Example, when searching for an address, the AutoCompleteBox keeps presenting the user with options based on what they’ve typed in thus far.

- Embedding PDF in Silverlight

• In Start Lite we implemented some magic to embed a PDF document within the Silverlight.  What’s actually happening though is with java script, we’ve actually placed a div tag on top of our Silverlight object and that div tag contains the PDF viewer control.

What is next?

As they often say, there is no rest for the wicked, and at New Media Labs, building “wicked” software means once we’re finished with one, we’re onto another!

Utilizing elements and components from Start Lite, we helped Aurecon develop a Back Office Incident Logging System, to be utilized by patrol officers and units, who while on patrol and by their own intervention identify or become involved in emergencies and incidents and ultimately resolve the issue then and there.  Upon their return to the station after duty, they can “back” capture such incidents for reference and reporting.

As with all systems, there is always a phase 2 and with Start Lite we’d like to focus more on modularity and being able to plug in or unplug functionality based purely on a particular clients requirement.  Silverlight is always evolving and with the release of Silverlight 3 there are some awesome new technology advancements we’ll be taking into account, specifically:

- MVVM

• In the case of Slverlight, you run the risk of ending up with code behind mixed with XAML in varying degrees and ultimately a solution which is difficult to maintain, refactor and test.  As with any large application, it is always a good idea to structure ones User Interface into its own subset of layers.  MVVM (Model-View-View Model) is a Silverlight specific pattern much like MVC or MVP which separates data (the Model) from the UI (the View), and presents the data in a bindable way for the UI (the View Model).

- RIA

• Microsoft .NET RIA Services simplifies the traditional n-tier application pattern by bringing together the ASP.NET and Silverlight platforms.  RIA Services provides a pattern to write application logic that runs on the mid-tier and controls access to data for queries, changes and custom operations.  In a nutshell, with RIA Services, we wouldn’t need to build a WCF layer.

- PRISM

• Ever heard of a Composite Application Block, well that is what PRISM is, a Composite Application Guidance Block specifically for Silverlight and WPF which allows one to modularize the UI and build it in such a way so that one can easily load or unload modules at choice.  Ideal for large, ever changing and ever growing applications.

We chose to use the ASP.Net MVC 1.0 Framework because it satisfied most of our requirements, both for the CMS and the site.  The look and feel for the website was designed and created by Stonewall+ and this HTML was seamlessly plugged into our MVC view as soon as it was supplied.  We used SQL Server and Linq for the data access.

Navigation

The navigation for the site was driven by a SiteMapProvider with all the nodes being managed in the CMS.  There were certain rules around the nodes i.e. Certain nodes cannot be removed or unpublished; nodes can only be published if their parent is published etc.  The sidebar and breadcrumb navigation on the site is dynamically generated from the SiteMapProvider.

CMS Item Workflow

The following content types:

• Generic Content
• Person
• Fund
• Media

incorporate the following features:

• Publishing workflow
  • Item Creation
  • Item Approval
  • Item Publishing
  • Item Archiving
• Email notifications
• Versioning
• Featured Item

Only certain user roles can perform the different workflow steps, with an over-seeing administrator role which can perform any of the tasks.  Once the item moves to the next status, an email is sent to the relevant parties to inform them and provide a direct link for them to action.

Each item has an Effective From and Effective To date, so you can easily set an item to only appear on the live site from a certain date, or until a certain date.

A versioning system is in place which allows a currently published item to be edited, without affecting the actual live item, until the new version is approved and published, at which stage the “old” version will fall away.

Each of these content types also allows for linking to media items such as newsletter PDFs, promotional video clips, fund fact sheets, etc.  Our link selector tool allows you to create an association to already-uploaded media (done via the Media Manager)

You can also feature an item in the CMS (Fund or Media) and that item will immediately appear on the home page of the website in the featured item boxes.

CMS User Management

Creating a new user

A user can either be an Administrator, or any of the other options or combinations thereof.  More roles can obviously be added depending on the business requirement.

CMS Media Manager

Media Manager

Administrators can upload files via the Media Manager, and these files can then be linked to the content items.The administrator can add and remove folders to allow for easier file storage management.

Website Screenshots

Investment Principles

Executive Committee Listing

Fund Detail

News

Things we learned

After the site went live we discovered an interesting Linq bug.  Turns out the Load Testing slipped through our QA’s net!  We learned very quickly to make sure this doesn’t happen again…

Next time we’ll also make sure we implement the latest and greatest version of the SiteMapProvider for MVC.  The version we used didn’t seem to refresh the nodes properly and so we spent far too much time on it than it deserved!

Repeat orders from existing clients, proving once again that NML is more than a supplier they are a partner that delivers.

Microsoft is now working in collaboration with New Media Labs to generate a case study out of this project. The project makes a good case study for Microsoft’s new Rich Internet Application technology known as Silverlight. This project is being billed as the largest Silverlight implementation in Africa.

Domain Driven Design was utilised extensively to model the specific banking domain. The Service Layer Data Contract and Service Contract models were modeled using the web service software factory.

The site has now moved into user acceptance phase and should be launched shortly.

Our base layer CMS was used for the development. Often when you tell clients you own a CMS they are skeptical – “why would you want to own or build a CMS when there are so many out there”. The reality is, a site like this is not only managing textual content it is managing the business content too. In other words you have to manage the data for each investment fund, the allocations of the fund and price performance over time. These are the kinds of systems NML has become experts in developing.

NML do not claim to own a packaged CMS or CMS framework. Packaged and Framework are words we’ve become skeptical of. We have a base layer of CMS code , that has been designed with integration in mind, to tie the company’s business model into the content management system. Our CMS allows for the usual features such as adding pages, changing navigation and publishing workflow.

The trustees are Alan Knott-Graig (MD IBurst), Tracey Lee Cohen (ex-Marketing Head of Blue Label Telecommunications), Gordon Douglas Stuart (Sentinel International Advisory), Dr Andile Ngcaba (Council of the University of South Africa (UNISA) and was an advisor to the Digital Inclusion Programme at Harvard University Law School), Jonathan Ackerman (Marketing Director Pick n Pay).

NML have integrated with the MyGate payment gateway and also MiMoney and UKash payment methods. We also integrated EFT back into the payment assurance processes.

NML have developed and commisioned this project at reduced rate as part of our corporate social responsibility.

Who is The Trust?

The Trust is a web portal that aims to make it easy, affordable and risk free for individuals and corporates to find and contribute to charities throughout South Africa and other African countries. The Trust provides a network for reputable and accountable South African charities who have been qualified through ongoing due diligence – irrespective of size, purpose or geographical location.

The Trusts’ web portal is your helping hand. The Trust aims to streamline the act of giving by creating a single access point for a range of causes. The idea is to boost the fundraising ability of charities by consolidating marketing efforts under one umbrella. By focusing on international celebrity endorsement through its many high-level contacts, The Trust is able to marshal significant support around several causes, from animals in distress to women and child abuse.

Charities affiliated to The Trust are expected to adhere to strict reporting and accountability guidelines in order to continue receiving support from us. Every cent of every rand donated to our causes is tracked, accounted for and distributed to charities with maximum efficiency and transparency. In addition, to all of the above, The Trust has the active support and buy-in of many of South Africa’s leading businesspeople. This means we are able to benefit from literally decades of cumulative business experience and the result is a charity run on business priorities for maximum effect.

New Media Labs will provide architecture and development expertise on Global Vision’s Adora product. Adora is a marketing automation and CRM tool that will supercede the product built for Diageo (the consumer goods companies with portfolio of world-famous drinks brands).

The application will be built as a Rich Internet Application which will utilize the Silverlight presentation layer technology developed by Microsoft.

Adora is a marketing automation software providing:
1) Multichannel Campaign Management (Distribution)
2) Enterprise Feedback Management (Collection)
3) Business Intelligence (Analysis)

Adora’s goals are:
To become the leading marketing technology provider to CPG / FMCG businesses globally and to become a recognized leader in the marketing technology space as defined by number and caliber of customers, global footprint, industry awards, and industry analyst opinion.

1) Short term: make ADORA one of the top 5 Marketing software products in the Business-to-Consumer space worldwide within the next 3 years.

2) Medium term: make ADORA the accepted leading software product for businesses wishing to extend the marketing capabilities of MS CRM within the next 5 years.

3) Long term: make ADORA one of the top 5 marketing software products in the market worldwide in the next 8 years.