JavaFx and OpenRPA

I’d like to control a JavaFx-Application which has dynamic Elements.
I tried with OpenRPA.Windows.Get Element-component.
It yields e.g.
{
“ClassName”: “GlassWndClass-GlassWindowClass-3”,
“Name”: “T2med”,
“ControlType”: “Window”,
“AutomationId”: “JavaFX1”,
“FrameworkId”: “Win32”
},
{
“ControlType”: “Button”,
“AutomationId”: “JavaFX5506”
}

But the AutomationId is different on each start.

I also tried with OpenRPA.Java.Get Element but this doesn’t select anything.

So I could remove the AutomationId from the selector and would get all the buttons. There are a few buttons in that application.
I found out that I cannot set MaxResults to 100 (or at least Highlight only highlights a max up to 20 items)
And most of the Buttons dont have Text associated with them (Image-Buttons :man_shrugging:)
But some of them do have Text.
Can I access the text of the Buttons? Is that feasible?
In that regard I liked the function ImageSearch of Autohotkey.
Is that possible in OpenRPA? To provide a screenshot of a button and find that image on the screen to get the coordinates and click the button?
As you can see … I’m a bit lost.
Thanks for any hints or tipps

I have no experience with javafx, does it have a javabridge like normal java ?

As you found out, if there is no way to make selectors unique, there is not much to do. Falling back to image recognition might be a solution.
OpenRPA already support image recognition, but yes, you can also use autohotkey for that ( but then you need to manage the images )

Hi Allan,
yes I saw this video and the Image Recognition works … but I don’t know how to get the coordinates of the Image-Recognition to put them in the Click-Component X-Offset and Y-Offset.

If you record with image recognition ( but that will most likely not be an option for you, since windows components also work ) it will do this for you.
If you manually added a get element/click element you can use select image to define an area and grab an image, but you will need to manually add offsets in click element ( i normally just do 20, 20 or something like that to move it 20 pixels down and right to where the image starts ) … you could use move mouse instead and play around with the numbers until it moves the mouse to the correct place.

Thanks, Allan.
I didn’t think of the record-functionality … I will try that.