OS file operations from applications

Hi,

I would like to attach files from OS storage to the web Form.
I manage to open desired functionality (Add button), but I stuck on the File window (typical “open”) handling/use. I would like to set path to file storage area and search file name, so that choose it by one “click”.

I expected problems with {Enter}, but even “Assign” activity did not set values, as for “Get Element” (NM) did.

What went wrong?


Kind regards,
Andy

PS. “add attachement” is also “table” which is recognized by “Recording” as “get table” instead of “get element” and “click”.

Yea, that is a little tricy. The openfile dialog is a windows component inside chrome. So the chrome part of openrpa cannot see it and the recorder does not support finding windows elements inside chrome.
Here is a post about that thing, including a video I made showing how do handle uploads

here is direct link to video

Hi Allan,

I figured it out, that for windows object the “Get (Windows) Element” will be better.
Also I like “selector” more and more :wink: as it shows parent - child processes.

But I fell into the trap of habit and tried again force “a robot” to work as me…
I would like to use “path field” and “search” as I did in other automation tool.

So far I have not been able to find right property of the “toolbar”, which will change its content (value is null). Maybe it is not right component, but only detected by selector.

Full path is a quick win solution and thank you for this “lifebuoy” :wink:
But I not always would know full file name, scans are in many formats.


Kind regards,
Andy

“path field” - ToolBar
{
“ClassName”: “ToolbarWindow32”,
“Name”: “Adres: C:\Roboczy\workplace\sam\uiVision\tess\Assets\faktury”,
“ControlType”: “ToolBar”,
“AutomationId”: “1001”,
“FrameworkId”: “Win32”
}

“search field” - Edit
{
“ClassName”: “ModernSearchBox”,
“Name”: “SearchEditBox”,
“ControlType”: “Edit”,
“FrameworkId”: “Win32”
}

Hi Allan,

As far as I understand the problem, it is related to the dynamic nature of the windowsUI.
I observe, that depending on object recognition method, the results are different.

“Selector” recognise only static shape of object, which can be not relevant to the actual its state. “Recorder” is able to unveil desired object in sequence of activities. When I highlighted an interesting field, first “click” inform “recorder” to “write down” pointed object. But this “click” also change the type of the field. As I am interested in this second face (class?) of the field, I have to click second time to tell “recorder” - “write it down”… And now I got a typical for edit fields behaviour, assign dialogue.

Unfortunately, I see some issues in this workflow. As I work mainly with FF and “open file” OS window, I expected that, as “selector” do, “recorder” will recognise child process. Even “recorder” correctly highlighted “search” field, first click results in NM.Get_Element (FF), second on the same highlight creates Windows.Get_Element with second face of the field.

I tried to compound both recognition and make a sequence “first face” - “click” - “second face” - “assignment”, but the “click” seems does not change the field face, even not virtual and with focus or the item is not properly passed between sequences.

Workflow ends with “Cannot locate element” message.
How to deal with this case?
I needs “search” field, as I know only name of file (unique id), but types can be various.


Kind regards,
Andy

first “face” of “search”
{
“ClassName”: “ModernSearchBox”,
“Name”: “SearchEditBox”,
“ControlType”: “Edit”,
“FrameworkId”: “Win32”
}

second “face” of “search”
{
“ClassName”: “RichEditBox”,
“Name”: “Pole wyszukiwania”,
“ControlType”: “Edit”,
“AutomationId”: “SearchTextBox”,
“FrameworkId”: “XAML”
}

Hi Allan,

That works!!! :star_struck:

The cloue is the right sequence organization.
Please find attached test case.

Just open FF, “open file” window and run “twofaces”. I expected some language problems, but the idea is in the test case.


Kind regards,
Andy

twoFaces.json (14.2 KB)

Hi Allan,

It works, but for the win10 only.

I also made a trial on the win7 and that behaviour does not exists. ModernSearchBox UI element has editable value and doesn’t require such strange approach.

Unfortunately, not always effective :\

Moreover, win7 UI has an events, which simplify the robotics tasks, as are automatically fired (search). In win10, after setting of the field value, it change its view and inside appear blue square with arrow. This element is not recognised by both “selector” and “recorder” as UI component (to dynamic?).

obraz

It seems that UI model of different OS versions are more separate as I expected (I am not frontend dev). Maybe recognition tools of the openRPA needs to be revised?

For blue square I tried to use Image.Get_Element, but the recognition process often killed browser (?!).


Kind regards,
Andy

Hi Allan,

I tried to recognise search “blue box” as an Image but with no success this time.
Element is not recognised at all and “highlight” pointed to… the activity itself.

obraz

One more question…
Are the “selector” and “recorder” results “positional” (depending on the position of the object on the screen)? If I change the position of the “open file” window, workflow will fail.
How to avoid this effect?


Kind regards,
Andy

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.