Accessing Additional Element Properties in OpenRPA

When retrieving an element in OpenRPA, it displays a set of properties. These properties can also be viewed using the Windows Inspect application. However, OpenRPA seems to show fewer properties compared to Windows Inspect. Specifically, I want to access properties like the LastChild property, which is visible in Inspect. Is it possible to reach and utilize these additional properties in OpenRPA?

Yes and no …
The short answer is “no.”
The long answer is …
For Windows, there is a “Raw” property that gives you access to the FlaUI object used to get the object. By accessing this, you can get access to everything exposed by Windows. But doing a quick search in the FlaUI GitHub, I don’t see any “NextChild” or “FirstChild” pattern …

I’m not sure here, so feel free to tell me I’m wrong. I suspect those come from traversing the tree and are part of the search query. And those you cannot access in OpenRPA since those are defined and controlled by the selector.

1 Like

There are ways to do it, at least as long as the undelying type is fla-based (if it’s windows automation, I give up :smiley: ).


Code transcript:

' myElement is a standard OpenRPA element found by GetElement
If (Typeof myElement.RawElement is FlaUI.Core.AutomationElements.AutomationElement)
Then
    FlaUI.Core.AutomationElements.AutomationElement flaElement = CType(myElement.RawElement, FlaUI.Core.AutomationElements.AutomationElement)
End If

Once you have the flaElement, you can access standard FlaUI stuff on it:
var firstChild = flaElement.FindFirstChild()
var lastChild = flaElement.FindAllChildren().Last() ← expensive to run
var lastChild2 = flaElement.CachedChildren.Last() ← uses cache (might miss on some pages if elements are dynamically inserted)

Now to get things like siblings, we’d need to get to a TreeWalker which is I think outside the scope of a single post ;), or iterate (by index preferably, so we can go back and forth if needed) over the children.

What I do not know though, is how to get back from a FlaUI AutomationElement back to the OpenRPA UIElement (didn’t go super hard looking for it).

Btw - when is the RawElement not FlaUI based? Just wondering just how cautious one needs to be on the CType.

Sidenote - I’ve done many automations (especially on old legacy apps, or on some customers in-house systems) where being able to linq over the children/descendants was the only way to get things done as the selectors were never useful there, so it’s great to know that we can get to the underlying Fla object (TIL :slight_smile: ).

EDIT:
Actually, navigating siblings could be done something like that (not tested):

// flaElement taken from before
var siblingsArray = flaElement.CachedParent.CachedChildren.ToArray();
var maxIndex = siblingsArray.Length - 1;
var currentIndex = Array.IndexOf(siblingsArray, flaElement);
// now +1 -1 over siblingsArray as much as you want as long as you're in range of 0...maxIndex
3 Likes

I love it when you do post’s like that … :smiley:
and “spot on”

to answer a few of the questions:

  • Getting an UIElement back from the Fla element is as simple as just doing new UIElement(yourFlaElement);
    The problem is, this will traverse the element to get off a lot of properties, if ANYTHING hangs, it will hang the entire robot, so be careful with that.
  • All XXElement implements IElement who has an “RawElement” property of type object, so we can store anything. For UIElement this will always be an FLAUI automationelement.
  • I don’t think using CachedParent will work, this require fla ui to be running inside a caching scope, but there are functions to do all that was talked about above, directly on the automation element, like FindAllChildren , FindFirstChild
1 Like

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.