Best way to get the value of a selected element?

[Tavis_Craik]

Hi there,

I am wondering what people would recommend as the best way to get the value of a selected element in an application is.

Currently building a bot to navigate a software, enter and extract data. The software has a limit on the number of times I can complete the copy function, which makes this significantly harder. Additionally, the software also has pretty bad naming conventions causing the selector tool to have quite a bit of difficulty getting the on screen element I want, so navigating it using TAB etc is the most efficient way.

Is there an alternative to copying (ctrl c) such as ‘Get Focused Element’ / ‘Get Focused Value’ that I could use?

Any assistance would be appreciated. Let me know if you would like any additional info from me.

[Allan_Zimmermann]

Sounds like a really old application or an application built to try and make RPA harder?
It’s hard to give clear advice here. Is this a Windows application? Can you find elements with Get Element or are you using keyboard shortcuts to navigate everything?
OpenRPA gives access to an older version of OpenCV. It can do basic OCR on the screen, but reliability is not great. It is meant to be used for finding labels, not “actual” data, but that could be a fallback plan.

[Tavis_Craik]

I’m able to get some objects using the Windows Get Element, however it gets complicated in some parts of the application as each object is named the same ‘Pane’. When running the bot, it always selects the first ‘Pane’ rather than the specific one that I am looking for. Do you know how I can select a specific ‘Pane’ in the list and then get the Name out of it? I’ve attached a screenshot with my reference.

Alternatively I have been using keyboard inputs to do some of the navigation, but since Ctrl + C sometimes (at random intervals) can throw an error in the software, I don’t really have a reliable way to pull data out. If Ctrl + C is my only option, do you know how I can clear the clipboard via OpenRPA?

Screenshot:
Using the selector tool, I can choose the ‘Due Date’ field, but upon running the bot, it ends up selecting the ‘Invoice Date’ instead seemingly because it is the first Pane.
Is there a way that I can modify the text below the screenshot to reliably get the Pane and value that I am looking for?

[
{
“filename”: “%ProgramFiles(x86)%\M1\M1.exe”,
“processname”: “M1”,
“arguments”: null,
“Selector”: “Windows”,
“search_descendants”: “True”,
“mouse_over_search”: “False”
},
{
“ClassName”: “WindowsForms10.Window.8.app.0.13965fa_r8_ad1”,
“Name”: “314032 - AP Invoice Entry - AP Invoice - HiltonTest System”,
“ControlType”: “Window”,
“AutomationId”: “EntryForm”,
“FrameworkId”: “WinForm”
},
{
“ClassName”: “AfxOleControl42”,
“Name”: “//__”,
“ControlType”: “Pane”,
“FrameworkId”: “Win32”
}
]

[Allan_Zimmermann]

Try and diisable short selectors, and select the element again. then at the pane that gets the wrong hit, add a property called indexinparent, and experiment with different numbers to see if you can make it find the right panel

[Tavis_Craik]

I’ve been able to modify it according to your recommendation. It almost works, but strangely, as soon as I add an ‘IndexInParent’ property to the last group in the example below, it results in the selector taking minutes longer to actually locate a field. Additionally, it can still only find the first pane in the last group, despite the new property working as expected higher up in the tree. I have tested with a range of numbers 1,2,3,4,5,6 with and without wrapping quotes.

[
{
“filename”: “%ProgramFiles(x86)%\M1\M1.exe”,
“processname”: “M1”,
“arguments”: null,
“Selector”: “Windows”,
“search_descendants”: “True”,
“mouse_over_search”: “False”
},
{
“Name”: “*AP Invoice Entry - AP Invoice - *”,
“ControlType”: “Window”,
“AutomationId”: “EntryForm”,
“FrameworkId”: “WinForm”
},
{
“ControlType”: “Pane”,
“AutomationId”: “EntryForm_Fill_Panel”,
“FrameworkId”: “WinForm”
},
{
“ControlType”: “Pane”,
“AutomationId”: “splitContainer1”,
“FrameworkId”: “WinForm”
},
{
“ControlType”: “Pane”,
“IndexInParent”:“1”,
“FrameworkId”: “WinForm”
},
{
“ControlType”: “Pane”,
“AutomationId”: “splitContainerRight”,
“FrameworkId”: “WinForm”
},
{
“ControlType”: “Pane”,
“IndexInParent”: “0”,
“FrameworkId”: “WinForm”
},
{
“ControlType”: “Pane”,
“AutomationId”: “uxViewLoader”,
“FrameworkId”: “WinForm”
},
{
“ClassName”: “ThunderRT6UserControlDC”,
“ControlType”: “Pane”,
“IndexInParent”:“0”,
“FrameworkId”: “Win32”
},
{
“ClassName”: “ThunderRT6UserControlDC”,
“ControlType”: “Pane”,
“IndexInParent”:“0”,
“FrameworkId”: “Win32”
}
]

[Allan_Zimmermann]

If IndexInParent is 0, there is no need to add it …
IndexInParent is very expensive, so only use it for the segment where it fails to find the element.

First, make the complete selector.
Then remove everything except the first 3 sections, make sure it works, then add the next one, and so on… When you find one that fails, play around with IndexInParent, and once it works, start adding more sections.

[Tavis_Craik]

Thanks for your assistance. I found that the windows getelement wasn’t quite able to get the value of elements in this software (not OpenRPA’s fault). Ended up getting an image version to work just how I need it.