Updated Code Sample: Windows 8.1 Input: Ink sample in C#/Xaml
Original Code Sample: Windows 8 Input: Ink sample in C#/Xaml
In the previous posts (Drawing Api I, Drawing Api II and Drawing Api III), we talked about the fundamentals of Windows Ink Api. The methods revised were related to the rendering of the ink strokes, saving/loading, selecting the ink strokes on the canvas and also dealing with the clipboard.
In this post, we will be mainly talking about the multi-touch implementation of the canvas and dealing with multiple pointers (i.e. fingers or toes if you want/can 🙂 ) firing multiple events.
Introduction to Multiple Pointers
As you might have noticed, each pointer event includes the PointerPoint which we were using to retrieve the location of the pointer gesture. When it comes to multiple touch enabled devices, another property of this class becomes important – PointerId. Essentially what happens is, every time a “pointer” comes in contact with the control that is firing the events, the pointer gets a new id assigned. So for instance, if I was trying to draw with two fingers on different coordinates on a canvas, each of the press, move, release events would be fired independently associated with a different pointer instance with a separate id.
As an example, we can use the simulator and select the Pinch/Zoom Touch mode. In this case we have two pointers. If we were to add the following line to the pointer pressed event handler, we can easily see how the touch points are handled.
CurrentPointerDevice = args.Pointer.PointerDeviceType; // Get information about the pointer location. PointerPoint pt = args.GetCurrentPoint(m_Canvas); System.Diagnostics.Debug.WriteLine("Pointer Pressed\r\nType:{0}\r\nPointerId:{1}\r\nPosition:{2}\r\n", CurrentPointerDevice.ToString(), args.Pointer.PointerId, pt.Position.ToString());
The simulator screen combined with the debug output looks similar to the screenshot below.
Revising the Canvas Manager
Considering the repercussions of Canvas Manager trying to micromanage each pointer event from different pointers itself, we should let it delegate a little. (yes, unresolved issues from work 🙂 )
For this purpose, we can create a handler class that will deal with the pointer events until the pointer is released. (e.g. draw the temporary lines until the canvas manager takes over, or move the selected ink strokes with the selection zone until the user releases the pointer)
public abstract class PointerHandler : IDisposable { protected readonly CanvasManager m_Parent; /// <summary> /// Abstract PointerHandler class, if implemented, deals with pointer move events /// </summary> /// <param name="parent">CanvasManager that created and owns this pointer handler</param> public PointerHandler(CanvasManager parent) { m_Parent = parent; } /// <summary> /// Pointer Id defines the specific pointer used used for this touch /// <remarks> /// In multi touch enabled devices each pointer gets a unique id. /// </remarks> /// </summary> public uint PointerId { get; set; } /// <summary> /// Handle the new point /// </summary> /// <param name="pt"></param> public abstract void HandlePointerMove(PointerPoint pt); /// <summary> /// Dispose is supposed to be used when the pointer handler is not needed anymore /// <example>When the ink pointer handler is disposed, it is suppose to clear all its lines drawn</example> /// </summary> public abstract void Dispose(); }
So if we were to implement a pointer handler for the drawing mode (e.g. InkPointerHandler), it would be responsible for creating a number of line geometries drawn the same pointer until the pointer is released and temporary lines cleaned up so we can use the Bezier segments from the ink manager. In this pattern, the canvas manager would be only responsible for creating the handler (i.e. on pointer pressed event) and passing on the pointer move events.
private void OnCanvasPointerPressed(object sender, PointerRoutedEventArgs args) { CurrentPointerDevice = args.Pointer.PointerDeviceType; // Get information about the pointer location. PointerPoint pt = args.GetCurrentPoint(m_Canvas); System.Diagnostics.Debug.WriteLine("Pointer Pressed\r\nType:{0}\r\nPointerId:{1}\r\nPosition:{2}\r\n", CurrentPointerDevice.ToString(), args.Pointer.PointerId, pt.Position.ToString()); switch (m_Settings.CurrentMode) { case CanvasMode.Select: // TODO: break; case CanvasMode.Erase: // TODO: break; default: // We create the appropriate pointer handler and add it to the hash table keyed with the pointer id m_Handers.Add(pt.PointerId, new InkPointerHandler(pt, this)); // We still need to pass the pointer down event to the inkmanager so it can create the bezier approximation. CurrentManager.ProcessPointerDown(pt); break; } args.Handled = true; }
The pointer moved event handler would be responsible for passing on the pointer moved event to the InkManager and the ink handler.
private void OnCanvasPointerMoved(object sender, PointerRoutedEventArgs args) { PointerPoint pt = args.GetCurrentPoint(m_Canvas); if (m_Settings.CurrentMode == CanvasMode.Select) { #region Select // TODO: #endregion } else if (m_Handers.ContainsKey(args.Pointer.PointerId)) { m_Handers[pt.PointerId].HandlePointerMove(pt); CurrentManager.ProcessPointerUpdate(pt); } else if (m_Settings.CurrentMode == CanvasMode.Erase || (pt.Properties.IsEraser && args.Pointer.PointerDeviceType == PointerDeviceType.Pen)) { #region Erase // TODO: #endregion } }
Implementing the Ink PointerHandler
InkPointerHandler would be a quick implementation of the PointerHandler. We already have the basic code in CanvasManager that deals with drawing the temporary lines until the ink manager takes over. We can move this code to the derived class and expose the Canvas member of the CanvasManager to the child handlers.
public override void HandlePointerMove(PointerPoint pt) { m_CurrentContactPt = pt.Position; var color = m_Parent.Settings.CurrentMode == CanvasMode.Ink ? m_Parent.Settings.CurrentDrawingColor : m_Parent.Settings.CurrentHighlightColor; var size = m_Parent.Settings.CurrentMode == CanvasMode.Ink ? m_Parent.Settings.CurrentDrawingSize : m_Parent.Settings.CurrentHighlightSize; if (CheckPointerDrawingDelta(m_CurrentContactPt)) { var line = new Line { X1 = m_PreviousContactPt.X, Y1 = m_PreviousContactPt.Y, X2 = m_CurrentContactPt.X, Y2 = m_CurrentContactPt.Y, StrokeThickness = size.Size, Stroke = new SolidColorBrush(color.Color) }; if (m_Parent.Settings.CurrentMode == CanvasMode.Highlight) line.Opacity = 0.4; m_PreviousContactPt = m_CurrentContactPt; // Draw the line on the canvas by adding the Line object as // a child of the Canvas object. m_Parent.Canvas.Children.Add(line); // Adding the lines to a local container so canvas manager can reuse these // i.e. one pointer is released but the other one is still drawing, remove the lines for // the released handler and add the lines from the current one. m_Lines.Add(line); } }
As you might have noticed, the handler code is almost completely copied from the previous CanvasManager implementation.
And the whole class structure looks like:
public class InkPointerHandler : PointerHandler { private Point m_PreviousContactPt; private Point m_CurrentContactPt; private List<Line> m_Lines = new List<Line>(); private double m_DrawingDelta = 3.0; /// <summary> /// Checks if the new point from the pointer move event meets the drawing threshold /// </summary> /// <param name="point">The new point reference received from pointer event</param> /// <returns><code>true</code> if the it meets the threshold</returns> private bool CheckPointerDrawingDelta(Point point) { // Calculating the distance between the previous contact and the new point double distance = Math.Sqrt( Math.Pow((point.X - m_PreviousContactPt.X), 2) + Math.Pow((point.Y - m_PreviousContactPt.Y), 2)); return (distance > m_DrawingDelta); } /// <summary> /// Temporary Lines drawn with simple geometry (i.e. line segments) /// </summary> internal List<Line> Lines { get { return m_Lines; } } public InkPointerHandler(PointerPoint initialContact, CanvasManager parent) : base(parent) { m_PreviousContactPt = initialContact.Position; PointerId = initialContact.PointerId; } public override void HandlePointerMove(PointerPoint pt) { //... See Previous Snippet } public override void Dispose() { throw new NotImplementedException(); } }
Wrapping Up
Finally when the pointer is released the canvas manager is responsible for clearing the canvas and redrawing the geometries using the ink manager Bezier segments. While doing this, we have to keep in mind that some of the pointer handlers might still be drawing their temporary lines.
private void OnCanvasPointerReleased(object sender, PointerRoutedEventArgs args) { if (m_Settings.CurrentMode == CanvasMode.Select && m_Handers.Values.OfType<SelectionPointerHandler>().Count() == 1) { #region Selecting // TODO: #endregion } else if (m_Handers.ContainsKey(args.Pointer.PointerId) && m_Handers[args.Pointer.PointerId] is InkPointerHandler) { PointerPoint pt = args.GetCurrentPoint(m_Canvas); // Pass the pointer information to the InkManager. CurrentManager.ProcessPointerUp(pt); m_Handers.Remove(pt.PointerId); } // Call an application-defined function to render the ink strokes. RefreshCanvas(); args.Handled = true; }
And the refresh canvas method implementation looks like:
/// <summary> /// Renders the canvas with the strokes and any other additional elements (selection box) /// </summary> /// <remarks>If we are in the selection mode, we check for any strokes that intersect with the Selection Rectangle</remarks> private void RefreshCanvas() { System.Diagnostics.Debug.WriteLine("Refresh Canvas"); m_Canvas.Children.Clear(); // Renders the ink strokes from the ink managers (i.e. ink and highlight) RenderStrokes(); // // Getting all the temporary lines from currently active ink handlers foreach (var line in (m_Handers.Values.OfType<InkPointerHandler>().SelectMany(handler => handler.Lines))) { m_Canvas.Children.Add(line); } if (m_Settings.CurrentMode == CanvasMode.Select) { m_Handers.Values.OfType<SelectionPointerHandler>().First().DrawSelectionBounds(); } }
That’s about it for now. With these changes the current implementation can handle multi-touch enabled devices.
In the next post, I will try upload the universal app sample I have been working on and describe the implementation approach I took to compensate for the missing Windows.UI.Input.Inking namespace on windows phone runtime applications.
Happy coding everyone…