Thursday, August 27, 2015

Getting Lazy with LINQ, Json.Net and WebAPI

I've been working on a project that uses a WebAPI hosted via OWIN inside a Win32 client application. Yes, inside a client application. Forgive me. Let's not quibble about it.

My Controller Actions generally return an IEnumerable<T>, with T varying depending on the Action. Most of my Actions return the (raw) result of a LINQ query, which is convertible to IEnumerable<T>. A contrived example;

public IEnumerable<ApiCustomer> GetAllCustomers()
return (from customer
in _CustomerRepository.GetAll
where c.IsPublic
select GetApiCustomerFromFullCustomer(customer));

All was well until I needed to return an IEnumerable<BaseType> with the elements as different derived types. To enable this, you must tell the serialisation formatter to include type information. This allows the client to determine what type to create and populate from the serialised data. Luckily, the awesome Json.Net solves this problem. Just configure your default serialiser to include the type information;

config.Formatters.JsonFormatter.SerializerSettings.TypeNameHandling = TypeNameHandling.Auto;

TypeNameHandling.Object should also work. The difference is Auto is only adds the type information where it detects a difference in the types used. Object always includes the type information.

Unfortunately, this is where I came unstuck. I have declared a return type of IEnumerable<T>, but I'm not returning one. Sort of. IEnumerable<T> is an interface, so you can't actually return an instance of it. You can, and I do, return something that is castable/convertible to it.

The result of a LINQ statement is actually a generic type, created by the compiler at runtime. If you check the type at runtime you'll see a name like "WhereSelectIterator`2". This object instance is an enumerator (IEnumerator<T> or IEnumerator) that allows lazy enumeration of the query. It is also convertible to IEnumerable<T>, so without type information (de)serialises fine.

Json.Net serialises the LINQ result with type information of "WhereSelectIterator`2" (or similar). Presumably because "WhereSelectIterator`2" isn’t exactly IEnumerable<T>. When the client deserialises the data it can't find "WhereSelectIterator`2". That type only existed at runtime on the server, it's not defined in any assembly on the client. This results in an exception being thrown.

Json.Net already knows how to serialise some types when returned as IEnumerable<T>. Lists and arrays work (serialised as a Json array), and maybe other collection types too. Because of that behaviour, I had expected anything convertible to IEnumerable<T> would do the same. Unfortunately the enumerators from LINQ queries aren't treated the same way. I'm not sure if this is a bug in Json.Net. I haven't checked the Json.Net code, so I don't know what it actually does in detail. I'm also not sure if there are use cases I'm unaware of that might depend on this behaviour. It’s possible this behaviour is by design. Either way, I had a problem.

I needed a solution. The obvious one, often given on the interwebs, is to "just add ToList or ToArray" to the LINQ queries, so they return a type that exists on the client. This does work, but it didn't suit me.

I have a lot of Controllers and Actions using this pattern and I don't want to change them all. Calling ToList/ToArray also has a cost in terms of memory allocation and CPU usage. These may be minor and we shouldn't "prematurely optimise", but ignoring these issues felt sloppy. There's also the environment to consider. The API is running in a long lived instance of a client application, on low spec hardware and often in use by a local interactive user simultaneously. It wasn't designed up front to host the API, or to serve more than one user at once.  Memory fragmentation, garbage collections and excessive CPU usage would all be cause for alarm. In a perfect world we'd have a different design. At the moment time to market and available resources are choosing our path. Another concern is every developer who ever works on this has to avoid the same issue. This isn't obvious. The code compiles and runs fine on the server. It's only when you try to deserialise it in a remote process that you'll find a problem. Even then, you might not understand the problem or how to solve it. That moves the "pit of success" out of the common path.

I could have written a WebAPI filter which automatically called ToArray/ToList when appropriate. This would have solved some of the problems above, but not the efficiency issues.

There's always a question of "did you look hard enough?" but I only found one other published solution. This is to change the Action return type to IList<T>, List<T>, or Array<T> or something similar that is a 'more concrete' type. This has pretty much all the same issues as the other solution, perhaps more. Some people insisted this was more correct architecture anyway, which I question. It's possible Array<T> is more 'correct', but even then I think it's questionable. List/IList definitely doesn't seem better to me.

So without further ado, I present my solution: A JsonConverter for generic enumerators. This class plugs into the Json.Net serialiser, detects when the object to serialise is an IEnumerator<T> and writes out a Json array. Deserialisation is left to Json.Net to handle with its default logic. With this class and one line of code to use it, all my Controller Action methods now work as I expect.

Consider this code released under the MIT license, and with the caveat that it "works for me". I'm not aware of any immediate short comings, but I didn't code it for anyone else. There's no error handling, no unit tests. I haven't checked if it copes with nulls. I didn't worry about the naming. I haven't optimised the code for maximum performance. I didn’t try it with other generic types. It’s provided “as is” and I do not want to set any expectations of quality. YMMV.

You can configure the default serialiser to use it like this;

config.Formatters.JsonFormatter.SerializerSettings.Converters.Add(new LinqToIEnumerableJsonConverter());

Here's the class itself (also available as a gist);

internal class LinqToIEnumerableJsonConverter : JsonConverter
public override bool CanConvert(Type objectType)
if (!objectType.IsGenericType) return false;
var genericType = objectType.GenericTypeArguments.First();

var enumeratorType = typeof(IEnumerator<>).MakeGenericType(genericType);

return (objectType.GetInterface(enumeratorType.Name) != null);

public override bool CanRead
return false;

public override bool CanWrite
return true;

public override object ReadJson(JsonReader reader, Type objectType, object existingValue, JsonSerializer serializer)
return null;

public override void WriteJson(JsonWriter writer, object value, JsonSerializer serializer)
foreach (var item in (IEnumerable)value)
serializer.Serialize(writer, item);

Now go forth and serialise!

Monday, July 27, 2015

Compose Tasks, UWP & Crash Workarounds

If you’ve worked on WP Silverlight applications you’re probably familiar with the ‘compose tasks’. These present consistent UI for composing email, sms messages or making a phone call.

Recently I needed similar behaviour in a UWP application where the API is different. The good news is this functionality still exists. It's just not always be obvious where to look.

The replacement for PhoneCallTask is Windows.ApplicationModel.Calls.PhoneCallManager.ShowPhoneCallUI.

The replacement for EmailComposeTask is the Windows.ApplicationModel.Email.EmailManager.ShowComposeNewEmailAsync method and the Windows.ApplicationModel.Email.EmailMessage class.

The ComposeSmsTask also has a replacement API. There is a significant  issue using this on the desktop versions of Windows 10.

The replacement for ComposeSmsTask is the Windows.ApplicationModel.Chat.ChatMessageManager. ShowComposeSmsMessageAsync method and Windows.ApplicationModel.Chat.ChatMessage class.

This is a universal API and thus is available on desktop and mobile platforms. In the pre-release bits for UWP, this works fine on mobile but will crash the app on desktop. Whether this will be fixed before the official release is unknown. Note I really mean crash. There is no exception to catch and the unhandled exception event doesn't get raised. The process just dies in a nasty way. ApiInformation.IsTypePresent and such don't help because the API is intentionally available on both platforms.  

This leaves you with no (good) way to support the feature and avoid the crash. Thankfully @aruntalkstech has the answer. You can use ChatMessageManager.GetTransportsAsync to determine if the current device supports any messaging protocols. If it does, it’s safe to call the compose API, if not then you should avoid it. For more information sample code, see Arun’s blog post.

Sunday, July 26, 2015

Mobile Barcode Scanning With Zxing.Net

A LOB mobile application I’m working on requires the ability to scan barcodes. A proper hardware solution would be best but sourcing something suitable has proven challenging. As a result we’re trying to scan barcodes using the device’s camera.

When we started the app was a Xamarin project so we tried Zxing.Net.Mobile. Unfortunately performance was abysmal. Many barcodes wouldn’t scan at all and the ones that did required many seconds to get a read. To be fair, we may have configured the settings badly. We found little guidance on the right settings. Even on iOS, enabling Apple's native API’s for decoding barcodes only made marginal difference.

We then looked at commercial solutions. We're not against paying for quality. Most of these SDK's provided a great experience. Yet we found the licenses over the top and difficult to understand. One company wanted tens of thousands of dollars because we will distribute the app via the store. This is despite our app requiring a specialised back end, limiting the audience to a few thousand people. Several others wanted per user per symbology, or per user per platform licenses.  These quickly rack up into the tens of thousands as well. The licenses were all yearly or monthly renewals too. Tracking the exact number of users is problematic. Our devices may not connect to the internet, or connect only periodically.  These devices aren’t personal the same way most mobile phones are. We will have several different users using the same device at different times.  There isn’t an accurate way to track users per device, or total users. No one talked about these issues, or how to report active users for licensing purposes. Another problem is most of these SDK’s only support one or two platforms not all three.

When we changed to a UWP application the situation was even worse. There are few SDK’s for the Windows platform and none that currently target UWP. There are some for WinRT that work in a UWP project, but they have separate binaries for x86/x64 and arm. None have packaged these binaries in a way a UWP project can reference and remain 'universal'. I believe this can be solved by repackaging the binaries ourselves but we never tried.

So we returned to Zxing.Net-, but this time only for decoding the barcodes. We wrote our own camera integration, based on the CameraGetPreviewFrame sample from Microsoft. We also configured Zxing differently. Now we have something with ‘acceptable’ performance for camera based barcode scanning. Unscientific tests show it out performs some implementations our competitors have used on iOS. What follows are tips based on our experience for getting the best out of Zxing. Your mileage may vary. Hardware, environmental conditions and use cases can have an impact. These tips should serve as a good starting point though.

Speed is King

The key take away is this: ensure decoding the barcode is fast. Slow decoding of the barcode is death to the user experience. This might seem counter-intuitive because accuracy would seem just as important. The trick is, with camera based scanning you have a lot of ‘dead’ frames. Frames where the barcode isn't present, isn't in focus or isn't lit correctly. The longer it takes to process each dead frame, the longer it will take to get a read.

  • Use a low to medium resolution. In my testing resolutions between 640x480 and 1024x768 work best. A higher resolution might seem better, as it should improve the barcode definition. In reality it slows down the decoding process and doesn’t help. Most mobile devices pick the highest resolution by default. They assume you’re trying to take video for the sake of video. You need to choose a sensible resolution for barcode scanning.
  • Don’t queue frames. I never actually tried this, but I did think about it. Later I found someone who had tried it, and abandoned it as a bad idea. Most queued frames will be ‘dead’ frames. Processing dead frames means getting a valid read will take longer. Just get a frame, attempt a decode, then get another frame. Don’t worry about the frames that get dropped.
  • Configure Zxing for best performance. Your use cases determine the best settings for you. I recommend these settings as a starting point. 
    • AutoRotate = false. Turning this on means decodes take about three times as long. Note you may need to enable this if you're scanning 2D barcodes. For 1D barcodes it is generally not helpful as orientation doesn't matter anyway. 
    • TryHarder = false. This setting isn’t intended for mobile scanning. It’s for large documents (say A4 document scans) that may contain barcodes anywhere. It makes a big impact to speed. Turn this off. You don’t need it. Trust me.
    • TryInverted = false. I haven't found this ever helped. 
    • Only add barcode formats you need to decode. As a general rule every barcode symbology you add increases the decode time. Get rid of the ones you don’t need. In our app we either enable all 1D barcodes and QR codes, or just all 1D barcodes. Which set is based on the context we are scanning in.
  • If you need to rotate the preview frames, do so efficiently. The Microsoft sample referenced above shows the most efficient way. There is a more discoverable, simpler mechanism but it is significantly slower. Use the one from the sample.

Using these settings we get a decode rate of 80-110ms per frame on a Lumia 636. We average about 10 decoded frames a second. While this might not sound great, we get a pretty good experience.

Auto Focus is Overrated

My last tip concerns focus. There’s a lot written about the need for ‘auto focus’ to get fast accurate reads. There’s also ‘continuous focus’ supported on some phones which sounds like a great idea. We have abandoned both of these. Continuous focus was never good enough to get a barcode read, in my experience. The frames looked ok to my eyes but Zxing.Net never found a barcode.

Auto focus is a bit more complex. It does work. We do use it in our app. We will perform an auto-focus if the user taps on the frame preview control on screen. This helps if the user is having difficulty or is scanning small or large barcodes. We don't perform auto-focus by default or automatically though.

Our default focus when scanning starts is a manual focus. We ask the camera to focus, once, at it’s smallest focal length. Depending on your hardware you may want to use some other (manual) value, but this works well for us.

The manual focus eliminates the camera seeking back and forth through the focal range. This improves the read speed. A user familiar with the system will get an instant read by holding the barcode the right distance away. For those not so familiar the read is still quick. If the barcode isn’t in focus immediately, the user can move the barcode or the phone slightly closer or further away. Scan times under a second are common.

Give manual focus try and see how it works for you.

Obviously none of this matches the performance of dedicated barcode scanning hardware. Yet it’s the best we’ve been able to achieve with what we have. It’s also better than where we started, and so far it’s been free.

One Last Thing

We haven’t tried this. It's just supposition.

It’s possible recompiling Zxing as a UWP library using .Net Native may improve things further. This should provide better performance for the decoding process.

Saturday, July 25, 2015

PointerDownThemeAnimation, PointerUpThemeAnimation & Null Reference Exceptions

WinRT and UWP projects have access to PointerDownThemeAnimation and PointerUpThemeAnimation. These animations provide a consistent visual feedback experience with the OS and other apps. Unfortunately these animations have some specific implementation details and don’t always behave as expected. This is most often true when using them from code behind instead of declaring them in XAML. This usually happens when you’re building a templated control, behaviour or similar reusable component.

The most common issue that occurs is an unexpected null reference exception. To avoid this;

  • Ensure the target element (the one to be animated) has a name. The easiest way is to assign one in XAML at design time. If no name is present at runtime you need to assign one.
  • Ensure you set the TargetName property on the animation. Setting the Target property won’t work. You must set the TargetName property, and that’s why you need a name set on the target.
  • You must add the animation into the logical tree for the target. My suggestion is to add it to the resources collection for the target. If you don’t do this the system has no context to search in so it still won’t find the target. This will also result in a null reference exception.

It's a shame there isn’t a better error message. The inability to assign a target object without a name is also disappointing.

Another issue is the duration property for these animations. You can set it without receiving an error, but the duration won’t change. This is fine as you should be seeking consistency with other apps. It's just sad that this is a silent failure. It is noted on MSDN though.

Below is a draft XAML Behaviour to apply these animations to any visual element. Feel free to use it (Apache 2.0 license). I have Windows 10 UWP code based on this and it should work in WinRT for Windows 8.1 too. In both cases it requires the Behaviours SDK for WinRT

This custom behaviour adds both the animation and a command binding. If specified the bound command will execute after the animation completes. If the command reports as not executable, neither the animation nor the command executes.

   1:      public class BehaviourBase<T> : DependencyObject, IBehavior where T : DependencyObject
   2:      {
   4:          private T _AssociatedObject;
   6:          public DependencyObject AssociatedObject
   7:          {
   8:              get
   9:              {
  10:                  return _AssociatedObject;
  11:          }
  12:          }
  14:          public T TypedAssociatedObject
  15:          {
  16:              get { return _AssociatedObject; }
  17:          }
  19:          public void Attach(DependencyObject associatedObject)
  20:          {
  21:              if (associatedObject == null) throw new ArgumentNullException("associatedObject");
  23:              _AssociatedObject = (T)associatedObject;
  24:              Attached(_AssociatedObject);
  25:          }
  27:          public virtual void Attached(T associatedObject)
  28:          {
  29:          }
  31:          public void Detach()
  32:          {
  33:              if (_AssociatedObject != null)
  34:              {
  35:                  Detatching();
  36:                  _AssociatedObject = null;
  37:              }
  38:          }
  40:          public virtual void Detatching()
  41:          {
  42:          }
  44:      }
  46:      public class VisualTapBehavior : BehaviourBase<UIElement>
  47:      {
  49:          public static readonly DependencyProperty CommandProperty = DependencyProperty.Register("Command", typeof(ICommand), typeof(VisualTapBehavior), new PropertyMetadata(null));
  50:          public static readonly DependencyProperty CommandParameterProperty = DependencyProperty.Register("CommandParameter", typeof(object), typeof(VisualTapBehavior), new PropertyMetadata(null));
  52:          private Storyboard _PointerDownStoryboard;
  53:          private Storyboard _PointerUpStoryboard;
  55:          public Storyboard Storyboard { get; internal set; }
  57:          public ICommand Command
  58:          {
  59:              get { return (ICommand)this.GetValue(CommandProperty); }
  60:              set
  61:              {
  62:                  this.SetValue(CommandProperty, value);
  63:              }
  64:          }
  66:          public object CommandParameter
  67:          {
  68:              get { return this.GetValue(CommandParameterProperty); }
  69:              set
  70:              {
  71:                  this.SetValue(CommandParameterProperty, value);
  72:              }
  73:          }
  75:          public override void Attached(UIElement associatedObject)
  76:          {
  77:              base.Attached(associatedObject);
  79:              //Stupidly, these theme animations require the object to have a name :(
  80:              var fe = ((FrameworkElement)associatedObject);
  81:              var name = fe.Name;
  82:              if (String.IsNullOrEmpty(name))
  83:              {
  84:                  name = Guid.NewGuid().ToString();
  85:                  fe.Name = name;
  86:              }
  88:              if (fe.Resources.ContainsKey("VisualTapDownAnimation"))
  89:                  _PointerDownStoryboard = (Storyboard)fe.Resources["VisualTapDownAnimation"];
  90:              else
  91:              {
  92:                  var pointerDownStoryboard = new Storyboard();
  93:                  var downAnimation = new PointerDownThemeAnimation();
  94:                  Storyboard.SetTargetName(downAnimation, name);
  95:                  pointerDownStoryboard.Children.Add(downAnimation);
  96:                  _PointerDownStoryboard = pointerDownStoryboard;
  97:                  fe.Resources.Add(new KeyValuePair<object, object>("VisualTapDownAnimation", pointerDownStoryboard));
  98:              }
 100:              if (fe.Resources.ContainsKey("VisualTapUpAnimation"))
 101:                  _PointerUpStoryboard = (Storyboard)fe.Resources["VisualTapUpAnimation"];
 102:              else
 103:              {
 104:                  var pointerUpStoryboard = new Storyboard();
 105:                  var upAnimation = new PointerUpThemeAnimation();
 106:                  Storyboard.SetTargetName(upAnimation, name);
 107:                  pointerUpStoryboard.Children.Add(upAnimation);
 108:                  pointerUpStoryboard.Completed += PointerUpStoryboard_Completed;
 109:                  _PointerUpStoryboard = pointerUpStoryboard;
 110:                  ((FrameworkElement)associatedObject).Resources.Add(new KeyValuePair<object, object>("VisualTapUpAnimation", pointerUpStoryboard));
 111:              }
 113:              associatedObject.PointerPressed += AssociatedObject_PointerPressed;
 114:              associatedObject.PointerReleased += AssociatedObject_PointerReleased;
 115:          }
 117:          private async void PointerUpStoryboard_Completed(object sender, object e)
 118:          {
 119:              await Dispatcher.RunAsync(Windows.UI.Core.CoreDispatcherPriority.Normal,
 120:                  () =>
 121:                  {
 122:                      var command = this.Command;
 123:                      var commandParameter = this.CommandParameter;
 124:                      if (command != null && command.CanExecute(commandParameter))
 125:                          command.Execute(commandParameter);
 126:                  });
 127:          }
 129:          public override void Detatching()
 130:          {
 131:              base.Detatching();
 133:              TypedAssociatedObject.PointerPressed -= this.AssociatedObject_PointerPressed;
 134:              TypedAssociatedObject.PointerReleased -= this.AssociatedObject_PointerReleased;
 136:              _PointerDownStoryboard = null;
 137:          }
 139:          private void AssociatedObject_PointerPressed(object sender, Windows.UI.Xaml.Input.PointerRoutedEventArgs e)
 140:          {
 141:              ((FrameworkElement)sender).CapturePointer(e.Pointer);
 143:              var command = this.Command;
 144:              var commandParameter = this.CommandParameter;
 145:              if (command == null || command.CanExecute(CommandParameter))
 146:                  RunStoryboardIfNotNull(_PointerDownStoryboard);
 147:          }
 149:          private void AssociatedObject_PointerReleased(object sender, Windows.UI.Xaml.Input.PointerRoutedEventArgs e)
 150:          {
 151:              ((FrameworkElement)sender).ReleasePointerCapture(e.Pointer);
 152:              RunStoryboardIfNotNull(_PointerUpStoryboard);
 153:          }
 155:          private void RunStoryboardIfNotNull(Storyboard storyboard)
 156:          {
 157:              if (storyboard != null)
 158:              {
 159:                  storyboard.Stop();
 160:                  storyboard.Begin();
 161:              }
 162:          }
 163:      }