I am trying to automate navigation through a web user interface of SAP BusinessObjects. I am able to locate elements with OpenRPA.NM Get Element and able to read values from Elements. Frustratingly I am not able to click on elements. I guess this is because all of them are located inside an iframe-Tag.
So officially, OpenRPA does not support iframes, but that is mostly due to me not being able to figure out reliably where an element is within an iframe. I know this can be solved - most other RPA tools have - but I have just not been able to make it work (my wife wrote most of the code that is finding location’s in chrome right now).
The reason it’s not really a big deal is OpenRPA can find anything, also in iframes, and it uses virtual clicks by default (for anything, not just Chrome). In that case, we don’t care where the element is on the screen, so besides “highlight” looking weird, it normally does not matter. But for those rare cases where a virtual click does not work, we need to know the exact location of the element, and that is broken for most iframes and framesets.
If a virtual click or using keyboard shortcuts (using item.sendkeys or typetext) is not an option, then no, I’m really sorry; there is no solution with OpenRPA - directly.
But you can run Tagui/Robot Framework inside OpenRPA, so if you can find the element in one of those, you can just invoke a Python script with Invoke Code from inside OpenRPA.
Alternatively, If you already know how to do RPA in Robocorp and/or Robot Framework, you can use OpenFlow agents to orchestrate those. And you can also run those “outside” OpenFlow soon with the Electron app once that is completed in a few weeks. (haven’t found a good name for that yet).
I tried to invoke a python-script similar to this:
from selenium import webdriver
from selenium.webdriver.common.by import By
# start the browser
driver = webdriver.Chrome()
# identify the iframe element
iframe = driver.find_element(By.ID, "contentFrame")
# switch to the iframe
driver.switch_to.frame(iframe)
# identify the element within the iframe
element = driver.find_element_by_xpath("//a[text()=\'Ordner\']")
# click on the element
element.click()
I then got the following error:
# ModuleNotFoundError : No module named 'selenium'
I checked the installed version of Python on my machine (3.11.2) and the location of selenium (located inside Python/Python311/Lib/site-packages).
What is the reason for the script not finding the module selenium? Is OpenRPA using another version of Python and i need to install selenium for that version too?
OpenRPA uses embedded python. It currently only supports python 3.7 so it has one builtin. You can install packages using the pip install activity in openrpa, or you can install it your self from a command line at C:\Users\%USERNAME%\AppData\Local\python-3.7.3-embed-amd64
If you go to settings → recorder extensions → scripting you can disable/enable using the builtin, or the one you have in your %PATH% environment variable. But if you switch to your locally installed one, it MUST be version 3.7 ( this can be needed if you need to use UI libraries, since TK is missing from the embedded version )
Sorry for the amount of questions, but I can’t seem to figure it out… feeling so stupid rn.
I tried using the pip install activity like I would if i installed it from the CLI:
pip install selenium
Therefore i just put this inside the Pip Install Activity:
Modules: {"selenium"}
I then tried to invoke the same script as before and got the same error:
# ModuleNotFoundError : No module named 'selenium'```
Maybe the pip install failed but was not detected by openrpa.
What happens if you do it from commandline ? ( i must admit i only tested using the rpa package, i ever tried with the selenium package, but i assume that should work also )
I tried to clarify, how I used the Pip Install Activity, as I was not sure how to use it. Therefore I tried to explain It with the CLI-command.
I used the Pip Install like I described above and then I used the invoke code part I mentioned before:
So what error do you get if you try to install from the command line ?
something like c:\Users\%USERNAME%\AppData\Local\python-3.7.3-embed-amd64\python.exe -m pip install selenium
