Sunday, September 28, 2003

For non english speakers the Divider object makes the whole Ink Analysis thing very important , and while the obvious benefit of analysis is in building recognizers for other "languages" , shape and other non character recognition promises all manner of ink based applications. see the tablet uml application and then think all types of graphical design and modelingtools
simply all Tablet PC devices have a digitizer underneath the screen that accepts pen input . if you can divide the SDK into subsystems , you would have the following subsystems

  • Collection API
  • Pen or Ink API
  • Reco API

Jarret and Su divide the pertinent material in their book into :

  • Tablet Input
  • Ink data management
  • Ink Recognition

The cover other relevant areas in Pen Computing as implemented by the SDK API, but those are main parts of the technology according to them. Thats pretty accurate.
The SDK documentation does a similar subsystem identification with Ink Collection,Ink data and Recognition. The SDK also brings in the idea of Ink analysis, but thats the same as Ink data as far as am concerned

Friday, September 26, 2003

this was about what the pen does to the collection and overlay, so back to that, Modes , modes and more modes, will talk about modes when i can,
11 methods and 14 properties of the PenInputPanel provide the rest of all the capability that is available to program the panel . Personally i think the panel should be a really last solution. That is if there is nothing else you can do to re design your app so that it can take adavantage of ink and related technologies.

Though am sure there are genuine time when firing up that component on a tablet makes sense like during the login secure desktop session.
and also there is what the Pen does to the Overlay or Collector. Remember that the collector and overlay are the main ways that the pen acutally lays out ink to Windows, or put another way all the ink flows to Windows through this two components -yes the input panel also picks up ink--

yes the Input Panel a hangover from Windows qwerty computing
The PenInputPanel has 3 constructors, a no args constructor and of course one that attaches to a Window and another that attaches to a control
public PenInputPanel();
public PenInputPanel(IntPtr AttachHandle);
public PenInputPanel(System.Windows.Forms.Control AttachControl);

also has a panel type enum that stores what the input panel is at a specific time. They are in order Default = last input panel type used , Inactive = just what it says, Handwriting = defaults to handwriting (current input language is used), Keyboard = like it says, the keyboard.

4 simple events are handy for doing all kinds of things from checking input failures too checking if the panel is visible

InkCollector and InkOverlay objects keep a set of ink rendering properties around named default drawing attributes. A kind of ink state. This properties are the what the ink that this components are able to produce at any given instance -- The State of the Ink so to speak

These are characteristics such as color, thickness, and pen tip style that are part of a class named Drawing­Attributes. InkCollector/Overlay have a property of type DrawingAttributes to maintain the default drawing attributes, named DefaultDrawingAttributes.

snippets galore: changing the color of ink using the standard color dialog

// Create and display the common color dialog, using the current ink color as its initial selection
ColorDialog dlgColor = new ColorDialog();
dlgColor.Color = inkOverlay.DefaultDrawingAttributes.Color;
if (dlgColor.ShowDialog(this) == DialogResult.OK)
{
// Set the current ink color to the selection chosen in
// the dialog
inkOverlay.DefaultDrawingAttributes.Color = dlgColor.Color;
}

snippet does the following things -

  • creates a ColorDialog,
  • shows the dialog and
  • sets the inkOverlay.DefaultDrawingAttributes.Color to selected dlgColor.Color;


Most of my snippets will use InkOverlay

Thursday, September 25, 2003

where doth that ink come from is the important question. Where are we going to get that ink from. Remember InkOverLay and the baby InkCollector who can only collect (thats why they named them InkCollector). Thats where all the ink comes from..

Ink Collection
(and not on your desk blotter, on the Microsoft Windows Tablet PC Edition)
Ink collection begins with the digitizer. A user places a pen on the digitizer and starts to write. the collection of ink data "flows" from the pen. You then use the InkCollector object to get the data that comes from the digitizer.
InkCollector Object
The InkCollector object captures ink input from available Tablet devices. The InkCollector object only collects ink and gestures that are input into a specific window. A very efficient event sink renders this input in real time. The InkCollector object captures the input and directs it into an Ink object.
-if you use two pens the ink may or may not flow those that really depends on the implementation of the entire input system, down to the driver level

Ink Collector mechanics
The InkCollector object attaches itself to a known application window. It then enables users to employ any available Tablet PC device (including the mouse) to lay down ink in real time on that window. The ink strokes that it collects are stored in an associated Ink object. These strokes can then be manipulated or sent to a recognizer for recognition. The InkCollector object also notifies the application when a cursor comes into range of any of the Tablet PC devices that are being used.
There are of course more details but on a very high level thats what happens.

InkOverlay Object (The Mother of Ink creating objects.)
The InkCollector object, discussed above, is useful for applications to provide their own model for selecting, erasing, and other user interaction. The InkOverlay object is a superset of the InkCollector object that provides editing support. This is useful for applications to integrate ink drawing and editing into their own document canvas by using a set of standard ink selection models that the object provides.

Both the InkCollector object and the InkOverlay object (as well as the InkPicture control) use common constructs, such as the Ink object and the InkDrawingAttributes collection, so that the basic way to interact with ink is the same for all Objects.

InkOverlay is a COM object that is useful for annotation scenarios in which users are not concerned with performing recognition on ink but, instead, are interested in the size, shape, color, and position of the ink. It is well suited for taking notes and basic scribbling. The default user interface is a transparent rectangle with opaque ink.
InkOverlay extends the InkCollector class in three ways:


  • It raises events for begin-stroke, end-stroke, and ink attribute changes.
  • It enables users to select, erase, and resize ink.
  • It supports Cut, Copy, and Paste commands.


InkOverlay tactical mechanics:
3 basic steps are required to get an InkOverlay doing its thing - commonly called the CAT procedure for create, attache, true-it(set its Enabled property to true) . this is the same for InkCollector
CAT procedure

  • Instantiate an InkOverlay object.
  • Attach the hWnd property of an InkOverlay object to that of another window.
  • Set the InkOverlay Enabled property to True.

The InkOverlay object includes basic printing support, but you must implement print preview or other advanced printing capabilities.
InkOverlay persists ink in Ink Serialized Format (ISF) format.

other interesting things about InkOverlay are the EditingMode property that determines what the pen does when it gets to the digitizer, the events (such as InkAdded, InkDeleted, and Stroke)
Hard to talk about this super inkers without talkingabout the
Drawing Attributes object
A DrawingAttributes object describes how any known set of ink is to be drawn. A DrawingAttributes object includes basic properties such as Color, Width, and PenTip. It can also encompass advanced parameters, such as variable transparency and Bezier smoothing, that can provide interesting effects or improve ink readability.

PenInputPanel Object
The PenInputPanel object allows you to easily add in-place pen input to your applications. The PenInputPanel is available as an attachable object that allows you to add Tablet PC Input Panel functionality to existing controls. The user interface is largely mandated by the current input language. You have the option of choosing the default input method for the pen input panel, either handwriting or keyboard. The end user can switch between input methods using buttons on the user interface.

and thus we now have the INK.....all the ink you see , any ink you can make and every ink there ever will be in in SDK 1.0 and 1.5 will have entered its existence from the above ways. The most versatile you might realize is the Overlay.

*note that Overlay is one word so the class name is Overlay not OverLay,

maybe i have been looking at this stuff for too long but it sounds like something that happens at a truck stop or something bad and unexpected like waylay, maybe thats what this thing is doing to linux


whats an overlay anyway
from dictionary.com
8 entries found for overlay
o·ver·lay1 ( P ) Pronunciation Key (vr-l)
tr.v. o·ver·laid, (-ld) o·ver·lay·ing, o·ver·lays
To lay or spread over or on.

To cover the surface of with a decorative layer or design: overlay wood with silver.
To embellish superficially: a simple tune that was overlaid with ornate harmonies.
Printing. To put an overlay on.
To create (a new area code) having the same boundaries as another: In Maryland, area code 240 was overlaid on top of area code

am not sure , the API designer needs to tell us why the fuck he called it Overlay, i guess it covers a window?
once more
an Ink object is a container
an Ink object is a container for stroke objects
an Ink object has a strokes collection that references its strokes objects


of course its a container its objects, but more important its single meaning in life is to contain stroke objects.
alright just a bit of an explanation for the random behavior, this is my blog, and i can say whatever i want, almost whatever, but thats my take on that

an Ink object is a container
an Ink object is a container for stroke objects
an Ink object has a strokes collection that references its strokes objects
And yes a Strokes collection is only existent in the context of an Ink Object.

A Strokes collection is created by an Ink object and can refer only to Stroke objects owned by that same Ink object.


simply to think of it in OO terms, encapsulation and all that ...

an Ink object is a container for Stroke objects, and a Strokes collection references Stroke objects.

ok we are going to overdo it a bit
an Ink object is a container for Stroke objects, and a Strokes collection references Stroke objects.
an Ink object is a container for Stroke objects, and a Strokes collection references Stroke objects.
an Ink object is a container for Stroke objects, and a Strokes collection references Stroke objects.
an Ink object is a container for Stroke objects, and a Strokes collection references Stroke objects.

an Ink object is a container
an Ink object is a container for Stroke objects,
an Ink object has a Strokes collection that references the Stroke objects.


an Ink Object has a collection of stroke objects that are stored in a Strokes Collection -- that's it....

INK is simply a bunch of strokes...How elegant.....

public class Ink : ICloneable

that's what that's about...Who is complaining.

A list of all the properties of the Ink Class reads like below, notice that the rest of the stuff is basically clueless, meaningless and really not such a big and could have been implement at the stroke level or wherever.

Property Description
CustomStrokes Returns the collection of custom strokes to be persisted with the ink.
Dirty Returns or sets the value that specifies whether the strokes of an Ink object have been modified since the last time the ink was saved.
ExtendedProperties Returns the collection of application-defined data that is stored in the Ink object.
InkSerializedFormat Returns a string that contains the name that should be used to query DataObject on the Clipboard to see if it contains that particular format
and what its all about
Strokes Returns the collection of all of the strokes in the Ink object.

"Strokes Collection
Contains the collection of Stroke objects that represent a single ink stroke.

Strokes collections are references to ink data and are not the actual data itself.

Caution In managed code, to avoid a memory leak you must explicitly call the Dispose() method on any Strokes collection to which an event handler has been attached before the collection goes out of scope. "
from Microsoft Tablet PC SDK definition for Strokes Collecction

this abstraction represents a single ink stroke, it implements ICollection and IEnumerable , has 28 methods and 2 events(StrokesAdded and StrokesRemoved ) and 7 properties that take care of things like how many Stroke objects are stored in the collection. In addition to the standard collection methods that take care of adding ,removing, moving stroke objects there are methods to scale and rotate stroke objects (remember that from PacketProperties that stroke objects are really just a fancy polyline - you have all the information that is required to do all kinds of vector type transformations on strokes) It would have been easy for the designers of the API to keep it at the bitmap level but the it might not have been the same magical ink that we have today. Requires more data to be stored but gives real powerful ink.
PacketProperties of course are very exciting to think about but thats too much paper for folks that just recently (70 centruries ago) figured out how to abstract on a surface. So we will still talk abit about Strokes. That power is coming shortly.

A Stroke Object is contained by an instance of an Ink Object. Ink Objects define the scope of a Stroke Object. an Ink Object owns zero or more Stroke objects. Yes am going to capitalize all the class names i reference. Thats because the SDK did such a good job at OO that its literally possible to talk about this and it would sound like a normal english conversation, talk about Object abstraction.

liteally the ink is made up of strokes and the strokes are made up of packets described by PacketProperties.
its hard to talk about ink persistence without talking about ink itself. We have all heard how ink is a first class citizen in the the Tablet PC SDK object model.
Another way of looking at it is that the Ink class is the outermost entry point into the Ink Data API, and it is analogous to a document class , much like text or files are the basis of keyboard based computing.

User input occuring on InkCollector and InkOverlay create Stroke objects.
Stroke objects are the characters or basic building blocks of an ink document . Each Stroke objects represents a stroke of digital ink, the same way each handritten stroke represents some type of human intention, a doodle and scribble, a straight line, a letter, a number, a stroke of the pen. Each time you actually take a pen to paper , you are really create strokes of some type. Thats what is abstracted by Stroke objects. A stroke.
-Movement of the pen on a surface is essentially what is captured though more information is collected than the movement - see the next post, its scary to imagine how powerful this strokes are compared to ordinary pen and paper strokes. Since this blog is about digital ink, i will from now on refer to ordinary paper and pen as rotm paper or rotm pen, rotm pronounced "rom" is for run-of-the-mill or ordinary paper.
Rotm paper and pen interaction of course has a lot of expressiveness but once you have a computer underneath , you can the capability to store things like the time the stroke was done, a serial number for each stroke, even the angle the pen was making with the digitizer....thats why rotm paper is at the end of its time, and of course there 's the whole "save a tree thing"

Stroke objects are are a collection of packets and each packet contains an x,y location and other packet properties. Next is the almighty PacketProperty Object.
Your stroke is represented as a collection of PacketProperties

Tuesday, September 23, 2003

on saving ink, see previous post

Sunday, September 14, 2003

Is JNT really a proprietary binary format?

Ok , couple of things first. Ink is ink and i would rather it stayed ink. That probably means i would rather have the actual serialized object . Thats where ISF shows up.
Ink Serialized Format (ISF) , the most compact persistent representation of ink is extensible with custom attributes (identified by a GUID) and allows you to store any kind of data or metadata as an attribute in an ISF stream.

Alright , and .....ISF is a format specific to Windows XP Tablet PC Edition, and the only way you can get at ISF is to use the Ink object's Load and Save methods.....

everything else starts after ISF.
For best performance and total control of ink ISF is the way to go with whatever enhancements you need to work with ink

For heterogeneous content, use fortified GIFs to encode ink into HTML documents. Browsers and html's hyper linking are your friend.
are required.

For Images you could move bitmap and metafile that are got from inks conversion to GIF

And we can alsoe encode ink in the RTF as OLE objects. This is very useful for pasting into an OLE container, such as Microsoft Word.

and of course the most useful format is XML .

the supported PersistenceFormat are and though it essentially is 2 formats , you can go anywhere from this. Actually go any where and come back ...

enum PersistenceFormat {
InkSerializedFormat = 0,
Base64InkSerializedFormat = 1,
Gif = 2,
Base64Gif = 3
};
based on the work am doing for WriteFree "the open source journal" , i think that the SDK persitence model is pretty well done, at least from a usability perspective. The frustrations that most users have with Windows Journal i believe are related to not being able to get at all of Journal programatically. We want access to the documents before they are created, after they are created and yes we want access to that Print Driver. We want this things and we want them now ;)). I think the SDK does a very good job at making it possible to built your own ink creation and consumption framework.
Journal design considerations probably sent it down the "proprietary" document format that it has. I am not sure its very proprietary.

Saturday, September 13, 2003

i have met some people that think handwriting recognition is not a big deal.
I am not sure when most input will be done by non keyboard mechanisms, but i know that it will happen. Maybe thats why all those guys in Star Trek do not use qwerty keyboards