Soul Solutions

Aligning a Kinect Skeleton with the Video or Depth images

Sun, 18 Dec 2011 05:44:40 GMT

The most asked for snippet of code at our recent workshop that I didn’t have was how to properly align a Kinect skeleton to either the Video image or the depth image. It turns out this is non trivial and most samples that just scale all three items to fit the screen are wrong. I took the time this weekend to make a good sample for others to follow. If you just want the two snippets scroll to the very bottom of the article otherwise continue reading for a full walkthrough, it starts with Kinect Beta2 clean WPF code.

Download the full code here.

The two key methods we are going to use are:

Microsoft.Research.Kinect.Nui.SkeletonEngine.SkeletonToDepthImage(Microsoft.Research.Kinect.Nui.Vector, out float, out float)
Microsoft.Research.Kinect.Nui.Camera.GetColorPixelCoordinatesFromDepthPixel(Microsoft.Research.Kinect.Nui.ImageResolution, Microsoft.Research.Kinect.Nui.ImageViewArea, int, int, short, out int, out int)

To make this super easy for you to drop into your project I’m going to put this scaling into my skeleton control and let you choose by setting a mode. In order to scale to the depth image or the video image you need to do up to 3 things:

Set the ScaleMode property to ToFill, ToDepth, ToVideo
Set reference to the Kinect Runtime being used, set property Runtime.
(Video only) We need the Depth PlanerImage, pass into method DepthImage.

For example let use WPF binding to do this and switch using a radio button. Here is the XAML:

<Window x:Class="VideoSkeleton.MainWindow"
        xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
        xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml" 
        xmlns:Skeleton="clr-namespace:SoulSolutions.Kinect.Controls.Skeleton;assembly=SoulSolutions.Kinect.Controls.Skeleton" 
        xmlns:VideoSkeleton="clr-namespace:VideoSkeleton" 
        Title="Kinect Nui Video Skeletons" WindowStartupLocation="CenterScreen" SizeToContent="WidthAndHeight">
    <Window.Resources>
        <VideoSkeleton:EnumConverter x:Key="EnumConverter" />
    Window.Resources>
    <Grid x:Name="layoutRoot" Width="1024" Height="768">
        <Image x:Name="screenImage" />
        <Skeleton:SkeletonControl x:Name="skeleton" ScaleMode="{Binding ScaleMode, Mode=TwoWay}" Runtime="{Binding Nui}" BoneFill="Blue" />
        <StackPanel Orientation="Vertical" HorizontalAlignment="Right" VerticalAlignment="Bottom" Background="White" Margin="8,8,8,8">
            <RadioButton Content="Fill" IsChecked="{Binding ElementName=skeleton, Path=ScaleMode, Converter={StaticResource EnumConverter}, Mode=TwoWay, ConverterParameter=ToFill}" />
            <RadioButton Content="Depth" IsChecked="{Binding ElementName=skeleton, Path=ScaleMode, Converter={StaticResource EnumConverter}, Mode=TwoWay, ConverterParameter=ToDepth}" />
            <RadioButton Content="Video" IsChecked="{Binding ElementName=skeleton, Path=ScaleMode, Converter={StaticResource EnumConverter}, Mode=TwoWay, ConverterParameter=ToVideo}" />
        StackPanel>
    Grid>
Window>

The “screenImage” we will set in code to be the depth or vide image, the “skeleton” is the control, more on how it works below, and the radio buttons use element binding to set the ScaleMode (a simple EnumConverter makes the binding magic work).

The code behind (XAML.cs) looks like this:

using System;
using System.ComponentModel;
using System.Linq;
using System.Windows;
using Coding4Fun.Kinect.Wpf;
using Microsoft.Research.Kinect.Nui;
using SoulSolutions.Kinect.Controls.Skeleton;

namespace VideoSkeleton
{
    /// 
    /// Interaction logic for MainWindow.xaml
    /// 
    public partial class MainWindow : Window, INotifyPropertyChanged 
    {
        private InteropBitmapHelper imageHelper;

        private Runtime nui;
        public Runtime Nui
        {
            get { return nui; }
            set
            {
                if (nui != null)
                {
                    //dispose all events and unintialise
                    nui.VideoFrameReady -= NuiVideoFrameReady;
                    nui.DepthFrameReady -= NuiDepthFrameReady;
                    nui.SkeletonFrameReady -= NuiSkeletonFrameReady;

                    nui.Uninitialize();
                }
                nui = value;

                if (nui != null)
                {
                    //wire up events and initialise

                    //step 1 events
                    nui.VideoFrameReady += NuiVideoFrameReady;
                    nui.DepthFrameReady += NuiDepthFrameReady;
                    nui.SkeletonFrameReady += NuiSkeletonFrameReady;

                    //step 2 initialise
                    nui.Initialize(RuntimeOptions.UseColor | RuntimeOptions.UseDepth | RuntimeOptions.UseSkeletalTracking);

                    //step 3 open streams / settings
                    nui.VideoStream.Open(ImageStreamType.Video, 2, ImageResolution.Resolution640x480, ImageType.Color);
                    nui.DepthStream.Open(ImageStreamType.Depth, 2, ImageResolution.Resolution640x480, ImageType.Depth);

                    //Must set to true and set after call to Initialize
                    nui.SkeletonEngine.TransformSmooth = true;
                }
                OnPropertyChanged("Nui");
            }
        }

        private SkeletonScaleMode scaleMode;
        public SkeletonScaleMode ScaleMode
        {
            get { return scaleMode; }
            set
            {
                scaleMode = value;
                //clean up background images
                screenImage.Source = null;
                imageHelper = null;
                OnPropertyChanged("ScaleMode");
            }
        }

        private void NuiDepthFrameReady(object sender, ImageFrameReadyEventArgs e)
        {
            skeleton.DepthImage = e.ImageFrame.Image;
            if (ScaleMode != SkeletonScaleMode.ToDepth) return;

            //Use Coding4Fun extension method on ImageFrame class for Depth
            screenImage.Source = e.ImageFrame.ToBitmapSource();
        }

        void NuiSkeletonFrameReady(object sender, SkeletonFrameReadyEventArgs e)
        {
            if (e.SkeletonFrame == null) return;
           
            SkeletonFrame allSkeletons = e.SkeletonFrame;
            
            //get the first tracked skeleton
            var skeletonData = (from s in allSkeletons.Skeletons
                                where s.TrackingState == SkeletonTrackingState.Tracked
                                select s).FirstOrDefault();

            if (skeletonData == null) return;

            skeleton.SkeletonData = skeletonData;
        }

        void NuiVideoFrameReady(object sender, ImageFrameReadyEventArgs e)
        {
            if (ScaleMode != SkeletonScaleMode.ToVideo) return;

            PlanarImage planarImage = e.ImageFrame.Image;

            //An interopBitmap is a WPF construct that enables resetting the Bits of the image.
            //This is more efficient than doing a BitmapSource.Create call every frame.
            if (imageHelper == null)
            {
                imageHelper = new InteropBitmapHelper(planarImage.Width, planarImage.Height, planarImage.Bits);
                screenImage.Source = imageHelper.InteropBitmap;
            }
            else
            {
                imageHelper.UpdateBits(planarImage.Bits);
            }
        }

        public MainWindow()
        {
            InitializeComponent();
            Closed += WindowClosed;
            Loaded += WindowLoaded;

            layoutRoot.DataContext = this;
        }

        private void WindowLoaded(object sender, RoutedEventArgs e)
        {
            Runtime.Kinects.StatusChanged += KinectsStatusChanged;
            Nui = Runtime.Kinects.Where(k => k.Status == KinectStatus.Connected).FirstOrDefault();
        }

        void KinectsStatusChanged(object sender, StatusChangedEventArgs e)
        {
            //ignore the case where a 2nd kinect is connected while we are using the 1st kinect.
            if (Nui != null && Nui.InstanceName != e.KinectRuntime.InstanceName) return;

            Nui = Runtime.Kinects.Where(k => k.Status == KinectStatus.Connected).FirstOrDefault();
        }

        void WindowClosed(object sender, EventArgs e)
        {
            Runtime.Kinects.StatusChanged -= KinectsStatusChanged;
            Nui = null;
        }

        public event PropertyChangedEventHandler PropertyChanged;

        private void OnPropertyChanged(string propertyName)
        {
            if (PropertyChanged != null)
            {
                PropertyChanged(this, new PropertyChangedEventArgs(propertyName));
            }
        }
    }
}

If you have read the Kinect Beta2 clean WPF code post this will look very familiar. The changes to make this work are:

INotifyPropertyChanged

Really simple trick to allow us to bind in XAML to properties in the code behind is to implement INotifyPropertyChanged and raise PropertyChanged events, this lets us wire up the Kinect Runtime and the ScaleMode properties with ease.

ScaleMode

This property is bound to the skeleton control’s property of the same name, we will use it to know what background image to show, nothing, video or depth.

NuiDepthFrameReady

Here we do two new things, first we pass the depth data to the skeleton control, it needs this to align the depth values to the colour vide image. Second we render the depth as the background only if we are in the Scale ToDepth mode.

NuiVideoFrameReady

Similar to the depth event handler we only render the video image if we are in the Scale ToVideo mode.

Download the full code here.

Changes made to the Skeleton Control to support this?

Things get a little more tricky here but with this good example I sure you’ll follow and be enhancing this for your project in no time. Firstly we’ve added three new properties:

Runtime, Here we need the Kinect Runtime so we can do the conversions.

ScaleMode, new enum this lets the user set what type of scaling they want.

DepthImage, you need to pass in the most recent Depth planer image available to let us convert depth to video.

All the magic happens in the old SetUiPosition method, a big switch statement applied the correct logic for each mode, let’s review each in turn:

ToFill

Scale the skeleton to best fit the size of the control. Use the Coding4Fun helper as usual

if (ActualWidth == 0 || ActualHeight == 0) return;

var scaledJoint = joint.ScaleTo((int) ActualWidth, (int) ActualHeight, 1f, 1f);

//center in control
Canvas.SetLeft(ui, scaledJoint.Position.X - ui.ActualWidth/2);
Canvas.SetTop(ui, scaledJoint.Position.Y - ui.ActualHeight/2);

ToDepth

Convert the vector to the corresponding depth image pixel then scale to fit the control.

if (Runtime == null || DepthImage == null) return; 

Runtime.SkeletonEngine.SkeletonToDepthImage(joint.Position, out depthX, out depthY);
depthPixelX = (int) Math.Max(0, Math.Min(depthX * DepthImage.Value.Width, DepthImage.Value.Width-1));
depthPixelY = (int) Math.Max(0, Math.Min(depthY * DepthImage.Value.Height, DepthImage.Value.Height-1));

//scale to actual size of control
Canvas.SetLeft(ui, depthPixelX * (ActualWidth / DepthImage.Value.Width));
Canvas.SetTop(ui, depthPixelY * (ActualHeight / DepthImage.Value.Height));

ToVideo

Convert the vector to the corresponding depth image pixel, then convert to the colour pixel, then scale to fit the control

if (Runtime == null || Runtime.VideoStream == null || DepthImage == null) return;

Runtime.SkeletonEngine.SkeletonToDepthImage(joint.Position, out depthX, out depthY);
depthPixelX = (int) Math.Max(0, Math.Min(depthX * DepthImage.Value.Width, DepthImage.Value.Width-1));
depthPixelY = (int) Math.Max(0, Math.Min(depthY * DepthImage.Value.Height, DepthImage.Value.Height-1));
int colourX;
int colourY;
var viewArea = new ImageViewArea
{
    CenterX = 0,
    CenterY = 0,
    Zoom = ImageDigitalZoom.Zoom1x
};

int index = (depthPixelY * DepthImage.Value.Width + depthPixelX) * 2;
var depthValue = (short)(DepthImage.Value.Bits[index] | (DepthImage.Value.Bits[index + 1] << 8));
Runtime.NuiCamera.GetColorPixelCoordinatesFromDepthPixel(Runtime.VideoStream.Resolution, viewArea, depthPixelX, depthPixelY, depthValue, out colourX, out colourY);

//bug in Kinect SDK? Seems to assume depth is 320x240, half of colour image
if (DepthImage.Value.Width == 640)
{
    colourX = colourX / 2;
    colourY = colourY / 2;
}

//scale to actual size of control
Canvas.SetLeft(ui, colourX * (ActualWidth / Runtime.VideoStream.Width));
Canvas.SetTop(ui, colourY * (ActualHeight / Runtime.VideoStream.Height));

Comments

As you can see it is not as simple or as clean as you like, a simple extension method to the Joint would be ideal, instead we need access to the Runtime itself to get the depth pixel and a depth image frame to get the colour pixel. Lastly during testing I found that 640x480 video with 320x240 depth worked fine, but up the depth to 640x480 and the whole thing was off, hence the fix, weird. You may have noticed this all works with/without the playerindex being captured in the depth stream but even with that off it fails for colour image about 640x480. I don’t see why this shouldn’t work with 1280x1024 image but it doesn’t. Let’s hope this all gets addressed for the next beta but until then feel free to use the code from this article in any form.

Download the full code here.

Question / Comment? Catch me on http://twitter.com/soulsolutions

Make your room larger for playing Kinect with Nyko Zoom

Fri, 16 Dec 2011 01:25:28 GMT

We have a decent sized play area for our Kinect but do find with 2 players we need to shift a bit of furniture and can get a bit dangerous when the person you’re playing against is 6’4” and has a really long arm span in some of the games. When we’re playing with the KinectSDK we often want to get a bit closer to the kinect for some of our tests than what it allows at the moment.

Decided to buy a Nyko Zoom and see what effect it had. I’d seen the ads for them and have been keen to see how much putting a lense over the Kinect messes with the optics/sizes etc. EBGames are selling them in store at the moment for $40. It’s really easy to mount to the Kinect and even comes with a protective bit of plastic if you’re planning on taking the lense on and off a bit. It slots on well and easy to tell if you’ve put it in the right spot.

Zoom with XBox and Kinect

First I tried it on our Kinect we use on our XBox with the classic “Kinect Adventures”. I found at the borders of the 2 zone space I got about 30cm extra I could come forward and to each side. While it’s not a massive improvement, it does mean we’re well clear of the furniture when we need to stretch from front to back and should give me an extra bit of room when I play any of the games against someone tall.

In the one player zone space I found I could get quite a bit further forward but I then ran into trouble with the projector and the top of my head casting shadows over the screen. So the 1 player zone should work quite well on smaller children who seem to like to get really close to the screen. Think we might give it a good test over Christmas

I think it could be really handy if you don’t quite have enough space for your Kinect with your XBox at home.

Zoom with PC and Kinect SDK

Next up…what does it do for us when we move it to the pc with the Kinect SDK. At the closest point, the depth camera now allowed me to get about 30-40 cm closer to the Kinect before it couldn’t see me anymore. At the furthest point I lose about 1.2 metres in depth. Not surprising, but it also distorts all the measurements. It picks up all the depth and skeleton information just fine, but instead of thinking I’m 180cm tall it thinks I’m about 140.

I think this could be really handy if you need to be able to use the sensor at closer ranges as long as you don’t need to use the height etc. data straight from the Kinect.

Technorati Tags: Kinect,Nyko Zoom

Kinect beta2 clean wpf code and getting started

Tue, 06 Dec 2011 13:34:14 GMT

Since the update to Beta2 of the Kinect for Windows SDK all our Kinect samples are out of date so I thought I’d update them to the latest and greatest and post them all here.

To begin with I want a nice clean framework that is both easy for the first time Kinect coder to understand but also makes use of the new abilities of the Kinect SDK. What I’ve come up with achieves the following 3 goals:

Support all combinations of Kinects be attached/detached/powered off/multiple.
Exposes a single Kinect Runtime as a property called Nui (It is null if there isn’t a Kinect attached).
Obvious location to wire up event model (and remove events if Kinect is removed)

This is meant to be a good place to start with simple examples with all the code clear in the one file.

    public partial class MainWindow : Window
    {
        private Runtime nui;
        public Runtime Nui
        {
            get { return nui; }
            set
            {
                if (nui != null)
                {
                    //dispose all events and unintialise

                    nui.Uninitialize();
                }
                nui = value;

                if (nui != null)
                {
                    //wire up events and initialise

                }

            }
        }

        public MainWindow()
        {
            InitializeComponent();
            Closed += WindowClosed;
            Loaded += WindowLoaded;
        }

        private void WindowLoaded(object sender, RoutedEventArgs e)
        {
            Runtime.Kinects.StatusChanged += KinectsStatusChanged;
            Nui = Runtime.Kinects.Where(k => k.Status == KinectStatus.Connected).FirstOrDefault();
        }

        void KinectsStatusChanged(object sender, StatusChangedEventArgs e)
        {
            //ignore the case where a 2nd kinect is connected while we are using the 1st kinect.
            if (Nui != null && Nui.InstanceName != e.KinectRuntime.InstanceName) return;

            Nui = Runtime.Kinects.Where(k => k.Status == KinectStatus.Connected).FirstOrDefault();
        }

        void WindowClosed(object sender, EventArgs e)
        {
            Runtime.Kinects.StatusChanged -= KinectsStatusChanged;
            Nui = null;
        }

    }

Here is our classic first demo from our talks where we wire up the video, tilt slider, depth and skeleton using this new framework.

WPF XAML:

<Window x:Class="NuiExample1.MainWindow"
        xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
        xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml" 
        xmlns:Skeleton="clr-namespace:SoulSolutions.Kinect.Controls.Skeleton;assembly=SoulSolutions.Kinect.Controls.Skeleton" 
        Title="Kinect Nui Example 1" WindowStartupLocation="CenterScreen" SizeToContent="WidthAndHeight">
    <Grid>
        <Grid.RowDefinitions>
            <RowDefinition />
            <RowDefinition />
        Grid.RowDefinitions>
        <Grid.ColumnDefinitions>
            <ColumnDefinition />
            <ColumnDefinition />
        Grid.ColumnDefinitions>
        <Image x:Name="colourImage" />
        <Image x:Name="depthImage" Grid.Column="1" />
        <Skeleton:SkeletonControl x:Name="skeleton" Grid.Row="1" />
        <StackPanel Grid.Column="1" Grid.Row="1" Orientation="Horizontal">
            <Slider x:Name="tiltSlider" Orientation="Vertical" Margin="40" Maximum="27" Minimum="-27" SmallChange="1" TickPlacement="TopLeft" Interval="1" IsSnapToTickEnabled="True" />
            <Button x:Name="tiltButton" Content="Set Tilt" Height="30" Width="200" Click="TiltButtonClick" />
        StackPanel>
    Grid>
Window>

C#:

using System;
using System.Linq;
using System.Windows;
using Coding4Fun.Kinect.Wpf;
using Microsoft.Research.Kinect.Nui;

namespace NuiExample1
{
    /// 
    /// Interaction logic for MainWindow.xaml
    /// 
    public partial class MainWindow : Window
    {
        private InteropBitmapHelper imageHelper;

        private Runtime nui;
        public Runtime Nui
        {
            get { return nui; }
            set
            {
                if (nui != null)
                {
                    //dispose all events and unintialise
                    nui.VideoFrameReady -= NuiVideoFrameReady;
                    nui.DepthFrameReady -= NuiDepthFrameReady;
                    nui.SkeletonFrameReady -= NuiSkeletonFrameReady;

                    nui.Uninitialize();
                }
                nui = value;

                if (nui != null)
                {
                    //wire up events and initialise

                    //step 1 events
                    nui.VideoFrameReady += NuiVideoFrameReady;
                    nui.DepthFrameReady += NuiDepthFrameReady;
                    nui.SkeletonFrameReady += NuiSkeletonFrameReady;

                    //step 2 initialise
                    nui.Initialize(RuntimeOptions.UseColor | RuntimeOptions.UseDepth | RuntimeOptions.UseSkeletalTracking);

                    //step 3 open streams / settings
                    nui.VideoStream.Open(ImageStreamType.Video, 2, ImageResolution.Resolution1280x1024, ImageType.Color);
                    nui.DepthStream.Open(ImageStreamType.Depth, 2, ImageResolution.Resolution640x480, ImageType.Depth);

                    //Must set to true and set after call to Initialize
                    nui.SkeletonEngine.TransformSmooth = true;

                    //Use to transform and reduce jitter
                    var parameters = new TransformSmoothParameters
                                         {
                                             Smoothing = 0.75f,
                                             Correction = 0.0f,
                                             Prediction = 0.0f,
                                             JitterRadius = 0.05f,
                                             MaxDeviationRadius = 0.04f
                                         };

                    nui.SkeletonEngine.SmoothParameters = parameters;

                }

            }
        }

        void NuiSkeletonFrameReady(object sender, SkeletonFrameReadyEventArgs e)
        {
            SkeletonFrame allSkeletons = e.SkeletonFrame;

            if (e.SkeletonFrame == null) return;

            //get the first tracked skeleton
            var skeletonData = (from s in allSkeletons.Skeletons
                                where s.TrackingState == SkeletonTrackingState.Tracked
                                select s).FirstOrDefault();

            if (skeletonData == null) return;

            skeleton.SkeletonData = skeletonData;

        }

        void NuiDepthFrameReady(object sender, ImageFrameReadyEventArgs e)
        {
            //Use Coding4Fun extension method on ImageFrame class for Depth
            depthImage.Source = e.ImageFrame.ToBitmapSource();
        }

        void NuiVideoFrameReady(object sender, ImageFrameReadyEventArgs e)
        {
            PlanarImage planarImage = e.ImageFrame.Image;

            //An interopBitmap is a WPF construct that enables resetting the Bits of the image.
            //This is more efficient than doing a BitmapSource.Create call every frame.
            if (imageHelper == null)
            {
                imageHelper = new InteropBitmapHelper(planarImage.Width, planarImage.Height, planarImage.Bits);
                colourImage.Source = imageHelper.InteropBitmap;
            }
            else
            {
                imageHelper.UpdateBits(planarImage.Bits);
            }
        }

        public MainWindow()
        {
            InitializeComponent();
            Closed += WindowClosed;
            Loaded += WindowLoaded;
        }

        private void WindowLoaded(object sender, RoutedEventArgs e)
        {
            Runtime.Kinects.StatusChanged += KinectsStatusChanged;
            Nui = Runtime.Kinects.Where(k => k.Status == KinectStatus.Connected).FirstOrDefault();
        }

        void KinectsStatusChanged(object sender, StatusChangedEventArgs e)
        {
            //ignore the case where a 2nd kinect is connected while we are using the 1st kinect.
            if (Nui != null && Nui.InstanceName != e.KinectRuntime.InstanceName) return;

            Nui = Runtime.Kinects.Where(k => k.Status == KinectStatus.Connected).FirstOrDefault();
        }

        void WindowClosed(object sender, EventArgs e)
        {
            Runtime.Kinects.StatusChanged -= KinectsStatusChanged;
            Nui = null;
        }

        private void TiltButtonClick(object sender, RoutedEventArgs e)
        {
            //Set angle to slider1 value
            Nui.NuiCamera.ElevationAngle = (int) tiltSlider.Value;
        }
    }
}

Download this full sample here.

What do you think? What kinect code are you looking for? You can catch me on http://twitter.com/soulsolutions

Kinect SDK Fun Day at ANU HCI Summer School

Tue, 06 Dec 2011 02:12:12 GMT

John and I were invited to help out with the HCI Summer School at ANU to get the students started with the KinectSDK for Windows. In the morning Henry Gardner opened the Summer School quickly followed by a look at some of the Kinect Innovations that have been happening by one of the staff from CSIRO.

We put together what we titled a KinectSDK “Fun Day”. John and I gave a couple of talks to give the students an introduction to the Kinect SDK. The “naked Kinect”, my gesture demo and John’s depth camera demonstrations proved popular with the audience again. Just before lunch we helped break up the group into smaller groups of 3 or 4 so they could share around the 12 kinects in the labs. KinectSDK is great for collaboration between team members as they have to literally get out of their seats to test their ideas so is ideal for working in groups.

The groups of students (including students from high school and universities from interstate) got stuck right into the SDK and fleshing out their ideas for the afternoon.

We took a break around 4pm and went around the room and shared the ideas and vision the students had come up with for the day. There was lots of variety across the 12 groups, using all different parts of the SDK. The Skeleton tracking and Audio proved to be very popular. They had a fun day with the Kinect and many of them will get to spend the afternoons of the rest of the week to continue working on their ideas and of course they can take the code home with them and continue with their ideas. We had lots of fun and look forward to hearing how they went through the week.

Thanks to Henry and Di from ANU for having us!

Technorati Tags: Kinect SDK,ANU,HCI Summer School

DDD Brisbane wrap-up

Mon, 28 Nov 2011 01:57:00 GMT

A little late in coming I know…but on 26th December Brisbane was host to the first DDD Brisbane! We’d sold out quite a lot before the event and had 40+ people on the waitlist so it can only be a good sign that people are keen for these types of events. We were there early to setup to hand out lanyards (which was the secret password for free coffee) and the all important swag.

Damian gave the keynote and all important logistical information around rooms, food and how to win free stuff!

For lunch we brought in 35 pizzas, 6 platters of subway, 4 platters of sushi. I thought we had way too much food but in the end, all the pizza and sushi went and we were only left with 2 platters of subway. Lunchtime saw a few people trying on various swag like the bucket hats and the very popular ladies Geekette shirts from Telerik.

A bunch of great sessions by the presenters and then everyone gathered at the end for their chance to win a bunch of prizes supplied by the sponsors.

Overall I think the day went quite well and I hope we’ll get to see a bigger and better DDD Brisbane next year.

Technorati Tags: DDDBrisbane

Speaking at Imagine Cup Smackdown at QUT

Thu, 17 Nov 2011 03:09:15 GMT

After being fortunate enough to be a Software Design Finalist Judge for Image Cup for the last 2 years, I’ve been invited to be part of the keynote and give a breakout session at the QUT Image Cup Smackdown Event 23rd November. If you aren’t aware this years worldwide finals are going to be held in Australia! If you can’t make the event and want to more about the competition itself, check out the main website http://www.imaginecup.com/.

Imagine Cup is the premier student technology competition in the world challenging students to create solutions for the worlds toughest problems.

Imagine Cup Smackdown brings students a prime opportunity to learn about Imagine Cup and learn from experts on what will assist you with solving those tough problems using technology!

Taking on the concept from the highly successful Smackdown 2010 event held at QUT last year, Imagine Cup Smackdown will showcase some of the latest technology and amazing demo's which will blow attendees away.

Imagine Cup Smackdown will not only hold a keynote event but will provide attendees the opportunity to deep dive into tracks including Software & Technology, Business and Design & UI.

See below for the confirmed awesome speakers of Imagine Cup Smackdown!

Lawrence Crumpton - Microsoft
Bronwen Zande - Soul Solutions
Ben Place - Microsoft
Danny Rowe - Microsoft

So when is it? Wednesday 23rd of November kicking off at 9am sharp

and where is it? QUT Gardens Point S Block Room 403

and how much? FREE

Technorati Tags: Imagine Cup 2012,Imagine Cup Smackdown,QUT

Free tickets to YOW! 2011 Brisbane

Mon, 14 Nov 2011 23:29:49 GMT

The folks over at YOW! 2011 are putting up a couple of free tickets to the Brisbane conference and offering a discount to the Girl Geek Dinner ladies to attend. For those that aren’t familiar, YOW! is part of the GOTO global conferences which includes QCON. If you have never been to a YOW! event be ready for some great talks and training.

YOW! is known for breadth and depth. Checkout these links for your interest:

Free Ticket 1 – Anyone attending the SOLD OUT DDD Brisbane conference on 26th November will get a chance to win a free ticket to the Brisbane event on 5/6 December.

Free Ticket 2 – Girl Geek Dinners Brisbane members – a free ticket to the Brisbane event 5/6 December will be randomly selected from those who publicly tweet or post on the Girl Geek Dinners Facebook page before 3pm (Brisbane time) 25th November – Which of the YOW! 2011 Conference Speakers they’d invite to a Girl Geek Dinner. The tweet or facebook post must contain #YOW_OZ and #GGDBNE and a link to http://www.yowconference.com.au

Plus YOW could like to offer members of Girl Geek Dinners a 10% discount to attend the Brisbane or Melbourne Conferences. Please email bronwen@soulsolutions.com.au to take advantage of the 10% discount.

Technorati Tags: YOW,DDDBrisbane,Girl Geek Dinners

KinectSDK Beta 2 Released–a few changes to be aware of

Mon, 07 Nov 2011 01:22:59 GMT

While we were sleeping on the weekend the latest version of the Kinect SDK for Windows was released – Beta 2. It also mean a new web address for everything KinectSDK here: http://kinectforwindows.org/.

There’s been a bunch of changes to the install aswell as the actual SDK. Here’s a few things I found when I upgraded to the newest version.

Installation

If you already have a previous version of the SDK installed you must uninstall it first. If you try to install Beta 2 over the top of a previous version you will receive the following error:

Install location - If you’re having trouble finding the dll locations the samples and documentation now have a new home under C:\Program Files\Microsoft SDKs\Kinect\v1.0 Beta2 – so you’ll have to update all the references in your existing projects to point to the correct directory in here.

RunTime change – The var runtime = new Runtime(); syntax is now obsolete. You can now access the this using Runtime.Kinects – which returns a KinectDeviceCollection. This also has a StatusChanged event where we can now get access to the state of each Kinect connected to the computer and its current state in the KinectStatus enum : Connected, Disconnected, Not Powered, Not Ready and Error.

Speech

Download the 64bit options – Make sure you have the correct versions of the speech rutime, sdk and language pack and can now build 64 bit applications!

Microsoft Speech Platform Runtime, version 10.2 – select 32-bit if you are running 32-bit Windows. If you have 64-bit Windows, we suggest that you install both the 32-bit and 64-bit runtime.
Microsoft Speech Platform - Software Development Kit, version 10.2 – select 32-bit or 64-bit according to your Windows installation
Kinect for Windows Runtime Language Pack, version 0.9

KinectAudioSource on the UI thread – For audio with WPF we no longer need to run the audio on a separate thread.

KinectAudioDemo – new audio demo added to the samples to checkout.

Other Documented changes:

Significant improvements to skeletal tracking:

Accuracy has been improved overall, resulting in more precise tracking.
Skeletal Frame delivery is faster, resulting in reduced latency.
Skeletal Tracking is now multi-threaded and can take advantage of multiple CPU cores.
When using 2 Kinects, developers can now specify which one is used for skeletal tracking.

The driver, runtime, and SDK work correctly on the Windows 8 Developer Preview for desktop applications.
NuiImageBuffer has been replaced by INuiFrameTexture, defined in MSR_NuiImageCamera.h. It is no longer necessary to include the file NuiImageBuffer.h in your project.

Sample code changes:
The samples have been updated. Also the C# samples use helpers, KinectWpfViewers, that may be useful in your apps.
Driver and runtime stability and performance improvements, especially for the managed API layer.

Technorati Tags: KinectSDK

Mixing things up by adding some colour to your retrospectives

Mon, 31 Oct 2011 00:40:58 GMT

I’m a big fan of SCRUM / AGILE / insert other name that helps me actually get on doing my job and be able to get it done more efficiently without being tied up in titles, roles and rank. I’ve always like the retrospectives in the sprints I’ve done. I find it a great way for everyone to reflect on the awesome stuff they did, what things they tried that didn’t work so great and to really decide as a team what you want to improve/change for next time. It’s a great way to air the dirty laundry and start with a clean slate. The problem I’ve often encountered is these meetings can get quite boring after awhile if you do them the same way and ask the exact same questions each time. Inevitably, people fall into roles of being quiet while the more dominant get more airtime.

Changing up the retrospective to have different formats can be a great way to bring out the voice of different types of people by tapping into different talents and comfort zones. I also find it’s a great way to get people talking about what happened in a different way and makes them think of things that they wouldn’t have come up with with the tried and true method.

I’d previously taught the team how to run the “emotional” kind of retro, where they plot their happy/sadness on a graph over the duration from the sprint. We got some good results from that but had a few comments about how talking about their feelings was just weird and was waiting for the group hug at the end

I’ve wanted to mix things up a bit last week, so decided to bring some colour into the retrospective by running an artistic retro. I’ve done this type before starting with a blank piece of paper and usually find it really hard (unless the team is quite artistic and in to the whole idea) to get going and get good results from it. I think I get much better participation and results if you give the team a theme/background picture to start with. I’ve seen a few different ideas such as a speedboat, a race track etc. but I think a good one to start with is the sail boat. The abstraction of putting things on the anchors for things that slowed you down, things on the sails that made you go quicker, obstacles in the water for things you had to overcome, canons etc on the bow for battles you had to fight really help people start with the drawing process.

Above is the result of my experiment on the team on our latest retrospective. At first I took the black pen away to force some colour but they quickly got right into it so I quickly gave it back as it wasn’t an issue. I’ve taken off all the postits that described what each of part of the drawing represented and how we thought about fixing or changing things but I think you can still see the variety of ideas that came through.

Love to hear any ideas on base images to start the artistic retro out with if you have them – leave me a comment below.

Technorati Tags: SCRUM,Artistic Retrospective

Why Open Data Sharing Is Important

Fri, 14 Oct 2011 03:01:05 GMT

This week we were able to expand TransHub to 7 cities – now including Canberra. We were able to do this because the dataset for Canberra became available on data.gov.au.

Just over 5 years ago we saw the first campains to “Free Our Data” in the UK. Governments in every country collect and hold a massive amount of data, much of it very useful in our everyday lives, and quite often data you have to pay to get hold of and not necessarily in a particularly useful format. The idea behind this campaign was the Governments should collect the best quality data possible and then make it available for free to all.

In recent years there has been a push for Australian government agencies to make more data available to the public. Since then Data.Gov.Au has been created.

Data.gov.au provides an easy way to find, access and reuse public datasets from the Australian Government and state and territory governments. The main purpose of the site is to encourage public access to and reuse of government data by providing it in useful formats and under open licences.

The great thing about putting up the data so the public can use it, is that developers, businesses and hobbyists can use the data and mash it up in ways that were never before thought of and make useful sites, applications and creations for others to use.

How we’ve used Data.gov.au for TransHub Canberra

To show how powerful publishing data in an open format is I’ll take you through the process we went through of creating TransHub Canberra.

When we created TransHub Sydney, we had thought of expanding and reusing the same idea with other cities. Some minor code changes were made from the original competition entry to allow us to apply to different modes of transport in Australia (an other metric countries). This allowed us to easily apply other datasets that we had at our disposal such as Adelaide, Auckland, Cairns, Perth and Townsville.

In a little over a week we were able to add a whole new set of data through the process:

We noticed the Journey Planner Data for the ACT had been made available on Wed night 5/10.
By Thursday 6/10 we had made the necessary adjustments and internal testing to prepare TransHub Canberra for Beta Testing.
Friday 6/10 we started the Beta and put out a call for Beta testers
Friday 14/10 TransHub Canberra is live on the marketplace

How we’ve used Data.gov.au previously

We’ve been a fan of data.gov.au for awhile. This year we were also involved in the Library Hack competition. We placed 2 entries:

QLD Mosaic

The images are from the State Library of Queensland’s out of copyright photographs from their photograph collection “People and places from across Queensland across time” . Data source is here: http://data.gov.au/dataset/picture-queensland/

The image for the mosaic is NASA’s Blue Marble Imagery cropped to the political boundary of Queensland. The mosaic was created using AndreaMosaic’s 64bit professional version, Photoshop CS5 and DeepZoom tools. It was processed on a Dell dual 6 core Xeon X5680 T5500 Workstation with 24GB Ram. It took just under 7 days to compile the images, mosaic, process, tile and upload to our website. The energy used to power the hardware was offset by our 6KW/h Solar System and the awesome Queensland Sun.

Historical Real Estate Maps

Soul Solutions has taken the State Library of Queensland's collection of digitized estate maps, advertising new housing estates in Queensland from the early to mid 20th Century. 165 of these have been digitized in the collection that can be found here: http://data.gov.au/dataset/real-estate-maps/

The maps are predominantly from Brisbane but also cover some regional areas of Queensland such as the Gold and Sunshine Coasts.

We have chosen to visualise these using an enhanced version of the Microsoft PivotViewer control, from LobsterPot Solutions, that allows us to presents 100’s of things at once and visualise them in a way that can add value by allowing multiple ways to filter and sort the collection and view details and metadata while showing the estate map image in a “zoomable” format.

Here’s a quick video demonstrating the application or you can watch it on YouTube here :http://www.youtube.com/watch?v=5RpauKnxp2w&feature=channel_video_title

What is this GTFS thing anyway?

Many of you have asked us what format we need the data in to be able to use it. There’s a few that are supported by we highly recommend GTFS – General Transit Feed Specification.

GTFS defines a common format for public transport schedules and associated geographic information. For more information see http://en.wikipedia.org/wiki/GTFS

It’s now a format accepted through many apis around the world for public transport data including Bing Maps and Google Transit.

Why don’t we have TransHub Brisbane and TransHub Melbourne?

Our top 2 feature requests for TransHub is to create a version for Brisbane and Melbourne. Many of you have asked why we haven’t done it and why can’t we? Currently neither Brisbane nor Melbourne publish their data publically for application developers to use. If we can’t get to the data we can’t make an application.

As many of you have pointed out we could screen-scrape or crowd source the timetable and station information we really don’t want to go down this path. We’d prefer to encourage the other cities to get on board with the open formats and publish their data so everyone can benefit and do it properly!

Our only thoughts are that public pressure may encourage them to change their minds and follow what the other cities in Australia and the world have done. So keep voting for the feature!

Why is Open Data Sharing important?

In short, let them do what they do best and let us developers develop the best apps possible on their data.

If we just look at phone application developers – there are many different smart phones out there, and we all like different features and functions in applications – which is why there’s about 66 different “torch” apps just on the WP7 marketplace. The government is good at collecting this data and running it’s services, not necessarily great at writing the applications we all want to use. The cost for them to develop a fully featured transit application on every smartphone platform, to keep it current, evaluate the features that people want would be enormous, not to mention the time to market for across all of the devices would be substantial.

If they just publish the data to a known standard, then application developers can make their own applications, with reliable data and bring it to market using their specialist skills much quicker. I’d love to see 10 different Brisbane Transit applications on WP7 and I’m sure many of you would love to see it on your favourite devices.

For us, it would mean we can take the investment we made in TransHub Sydney, and apply it to the Brisbane data and make an equivalent application in under 2 weeks – I don’t think any government department could match that time to market. As we have customers using each city version of the application we can address, and add feature requests for a specific city and apply them across the range of cities with not much additional effort.

Technorati Tags: Open Data,Data.gov.au,TransHub,Canberra,LibraryHack2011