Hey Allan,
We are currently working with IFrames and are dealing with problems with the load times of the elements. While searching for solutions i realized some weird behavior with OpenRPA.Windows Get Element.
The Min Max Settings of Get Element do not chance this behavior.
There are a few things wrong here, and I’m not sure how you ended up there.
All selectors must start with a definition section (the first element in the JSON array). For Chrome, that would normally be NM (short for native messaging, the protocol that Chrome uses to allow extensions to talk to an outside application). There is an exception to the above: if you have a Get Element inside another Get Element, it will take the item from the outer Get Element and put it inside the “from” property. This way, you can then “extend” the selector for the first Get Element to the next. But you did not state that this is what you did here. Since we can see you got the desktop back, I’m pretty sure that is not what you are trying (and if you are, you are missing “item” inside the “from” property).
Your selector seems to be a Windows selector. Chrome should be using NM (so a selector with an XPath or CSS selector). It is true, that if you enable accessibility for Chrome, you CAN use Windows, but generally, that is not what you want and is not really supported by us. So again, it should be NM.
Per the known issues page, iframes are officially not supported. The only reason we do not support them is that our extension does not know how to calculate the physical location of elements on the screen. Therefore, it cannot do highlighting and use physical clicks. But since the default is to always use virtual clicks and assign values using “item.value”, iframes will work in most cases anyway.
I would recommend you try using the recorder button first and then try working your way back if you need to manually create selectors or have some special use case I’m not seeing here. Do you need to use Windows over NM? If yes, why? If you tried the recorder and it did not work, did you remember to install the Chrome extension? If using the recorder on an iframe, beware that the highlighter will be “off,” but doing the clicks should still work and create the correct selector.
To 1) Outside the IFrame removing the other parts of the Selector works fine, so i am confused as to why i need the beginning with a windows selector, for the NM one i understand that its needed. Will try to leave the beginning parts in tomorrow but i don’t expect another result, since from my understanding that’s just a hierarchy and only needed if the lower levels are ambiguous, the selector works fine if i wait till everything is loaded so i don’t see a problem in the selector itself. The Only problem is that it returns an element even so it shouldn’t be finding one with the given selector.
To 2 and 3) I am using a windows Selector because the NM selector wont help here, since all the stuff is inside an Iframe which is unreadable by the NM selector. I tried using it anyways but it didn’t work even so i defined the xpath correctly.
Will report back tomorrow if adding the rest of the windows selector helps.
I have never tested OpenRPA with a selector that does not have the first “settings” part, and I am kind of surprised it’s not just giving an error.
It’s designed to “parse” that, to know what plugin to ask, to work with the rest of the select. But the default plugin is Windows, so if everything else fails, it falls back to Windows, that is why Windows is the one returning something.
The way the Windows plugin works with selectors is, at first, it tries finding the first element (so second in the selector) without searching children. This element should always be a window, or the taskbar, or start menu. Then, if “search children” is enabled, it will do a search in the tree under that element. If not, it will only search the direct children, and so on.
Since there is no element to search for, I guess that’s why it’s returning the desktop instead(?). Not sure though… either way, it is not designed to work without at least one element to search for.
2,3) Okay, if that works, there is nothing wrong with using the Windows plugin over nm if you have success finding the element you need.
hmm, I think I misread a part of your reply.
So you say it works, so you don’t understand why you need change it.
Well, to be honest, if it works, there is no need to do that, except 1 thing. It was not intended to work, so it might break during some update in the future (not likely, but still).
And the reason it is not meant to work is that searching the ENTIRE UI tree is very expensive and will be very slow if there are multiple applications open with many elements in them, so by forcing the first element to a window, we at least cut out some of those expensive queries.
I’m open to a good argument to not enforce that (?)
Hey Allan,
you are completely right, as soon as i add the definition part the behavior is as we would expect. Is there any way you can implement a warning for that? Like throw an error when it cant find a definition? Because being able to use it like this and then getting unexpected behavior in some cases not all of them will confuse people.
If I do it at runtime (when you run a workflow), I don’t think that should be too hard. But for the UI, I think it will be a lot more work since we need to check if the selector is part of an activity that sits inside a nested get element. I will definitely do it for runtime (either throw an error or warning).
