UI automation with accessibility object and win 32 API

In my earlier blogs (Fun with UIAutomation and calc, Fun with UIAutomation and calc 2 (event handling)) I have been talking about how to user uiautomation in .NET. Now it time to dig down bellow uiautomation and go directly to the source WIN 32 API. I have tried to translate the first calculator project to use win 32 API and accessibility object instead. The first thing that we need to-do is to create DllImport statement for user32.dll used for handling more or less all windows UI and oleacc.dll used for accessing the accessibility object. You will find my complete solution in the end of the blog.

An example of DllImport is EnumChildWindows that gives use the possibility to get all child windows to a specific window.

[DllImport("user32.dll", SetLastError = true, CharSet = CharSet.Unicode)]
[return: MarshalAs(UnmanagedType.Bool)]
public static extern bool EnumChildWindows(IntPtr parentHandle, Win32Callback callback, IntPtr lParam);

To understand what a window is in this sense I will use calculator as an example. When you start the calculator you will see some buttons a text box and so on. The first window for the calculator is the other window containing everything. The main window then has a bunch of children. A main window normaly has title window child and content window child. The title window has the close button, minimize and maximize button. The content window has some other window as a pane containing buttons for example. To get all child windows for a specific window I did create this method. Control is base class for all types of control as window, button, textbox, and so on.

public Collection Childs(TreeSearchScope scope)
{
  var control = new Collection();
  User32WindowsMethods.EnumChildWindows(this.handle, delegate(IntPtr ptr, IntPtr parameter)
  { 
    var tmp = new Control(ptr); 
    if (scope == TreeSearchScope.Children)
    { 
       if (tmp.Parent.Handle == this.handle) 
       { 
         control.Add(tmp); 
       } 
   }
 
   if (scope == TreeSearchScope.Descendant)
   { 
      control.Add(tmp); 
   } 
   return true; 
 }, IntPtr.Zero); 
 
 return control; 
}

By help of the above method it’s easy for example to find a button.

private static Button FindButton(Window parentWindow, string label)
{
    Button button = null;
    foreach (Control control in parentWindow.Childs(TreeSearchScope.Descendant))
    {
        if (control.Name.Contains(label) && control.ClassName.Contains("Button"))
        {
            button = new Button(control.Handle);
        }
    }
 
    return button;
}

When you want to click on a button you will use the PostMessage method from user32.dll that will send any message directly to any specific control. For the button we will first send a message that the mouse button has been pressed on him followed by the mouse button have been released. This will simulate a normal user interaction without using a real mouse. The same thing can be done for a keyboard key and so on.

public bool Invoke()
{
    int result = User32WindowsMethods.PostMessage(this.Handle, (uint)WindowsMessages.WM_LBUTTONDOWN, IntPtr.Zero, IntPtr.Zero);
    if (result == 1)
    {
        result = User32WindowsMethods.PostMessage(this.Handle, (uint) WindowsMessages.WM_LBUTTONUP, IntPtr.Zero, IntPtr.Zero);
    }
 
    if(result == 1)
    {
        return true;
    }
 
    return false;
}

The last thing that I would like to point out is the usage of the accessibility object to get the content of a text box.

try
{
    var accessibleObj = (IAccessible)oleacc.AccessibleObjectFromWindow(this.Handle, (uint)oleacc.OBJID.CLIENT, oleacc.IID_IAccessible);
    return accessibleObj.get_accValue(0);
}
catch (System.Runtime.InteropServices.COMException)
{
    return string.Empty;
}

The easiest way to understand how it works is probably to take my example solution and play around with it. Modify it use a other program then the calculator

The download for the VS 2008 solution is found here.
Another good source is the pinvoke site