NodeRED agent on Docker Desktop OON installation for a low/no coder - please can you make a guide?

[bpin]

OpenRPA version: 1.4.54
OpenFlow version: 1.5.0
Using app.openiap.io or self hosted openflow: self hosted Docker Desktop
Error message: On screen - error message is
(HTTP code 404) no such container - No such image: openiap/noderedagent:latest
Screenshot or video:

Attach a simple workflow from OpenRPA or NodeRED that reproduces the error/issue:

This is a follow on at Allan’s request to Nodered Agent not working, which I had posted on as this issue for me was (I thought) purely about NodeRED.

I am a citizen developer - apart from doing the no-code bit, (I can point and click and use parameters to a degree), I can use the cmd prompt a little bit, some baby steps in Python, but little to zero knowledge in networking, and useless at scripting and coding.

Here is what I have done sofar to try to get OpenRPA/OpenFlow/NodeRED installed on the same desktop, without using any cloud services, just with github and git.

1. Downloaded OpenRPA. Installed and played around with it. Cleared out Docker Desktop and ran it as admin.
2. Read the url - GitHub - open-rpa/docker . Installed git, cloned that latest repository at the url quoted above. cd to the docker subfolder. Ran normal-up.bat.
3. Disconnected my internet access to make sure I was not getting anything from your server, just what I had on my computer.
4. tried localhost - got a 404. tried localhost.openiap.io - got localhost.openiap.io ’s server IP address could not be found. tried localhost.openiap.io/#/main - got localhost.openiap.io ’s server IP address could not be found.
5. Installed nginx on Docker Desktop to make sure the server functioned, to test the Docker installation. It worked fine.
6. Watched the video referred to on the web page referenced above.
7. The web page did not give me the guidance I was seeking about how to integrate NodeRED into the installation.
8. The video focussed on topics other than how to install on a single machine.

I have read the instructions on the url noted above. These have no information regarding NodeRED. I have watched the video referred to in the url. That is for people in a different situation. My objective is to use an OON setup in Docker where the installation is done just from files on github (no assistance from your servers), and have that installation coexist/play nicely with other server software that I also host on Docker.

It does seem as though it really is quite simple for people who know what they are doing.

My request is that you or some other kind person can give us citizen developers a monkey see monkey do guide/set of instructions in a list so if someone follows it they will produce the result. Or perhaps even bundle up everything in a Docker image that can be downloaded from your website.

My hunch is that the help needed is in 2 main areas - one is the network and ports info so that the DD installation is viewable, and secondly showing us exactly how to integrate NodeRED in its new position as an agent - the url to get the code from and instructions.

I hope that providing a guide will help many more people than just myself. Maybe Anders could help ? In the meantime I attach a few screenshots of where my installation is at.

Regards

[Allan_Zimmermann]

I’m moving this to OpenFlow , this is not a bug.

Running locally, does not mean running without internet.
you need access to DNS to resolve domain names
you need access to docker hub, to get and validate images
you need access to npmjs to download and chat for images
just to name a few of the more important once’s

you can run openflow completely offline, without access to the internet, but it’s not simple and definitely not something you should try if you don’t know basic network stuff or how the different component talk to each other. ( not a lot of peple do, so nothing wrong with that )
There is a demonstration here, https://www.youtube.com/watch?v=r_aEHZMSICE

[bpin]

Following along with video (https://www.youtube.com/watch?v=r_aEHZMSIC) After having cloned the github repository into my folder ProjectName\Docker

1. Delete all containers in Docker Desktop
2. Delete all images in Docker Desktop
3. run .\offline-down.bat in >docker
4. run .\offline-pull.bat in >docker
5. Images image
   
   
6. Containers image
   
   
7. No contents in the docker\offline folder -
8. No docker-compose-traefik- files in the sidebar

Following along with video (https://www.youtube.com/watch?v=r_aEHZMSIC) After having cloned the github repository into my folder ProjectName\Docker

1. Delete all containers in Docker Desktop
2. Delete all images in Docker Desktop
3. run .\offline-down.bat in >docker
4. run .\offline-pull.bat in >docker
5. Images image
   
   
6. Containers image
   
   
7. No contents in the docker\offline folder -
8. No docker-compose-traefik- files in the sidebar

The video does not seem to produce the result. This may be because the software has moved on since you made the video. Or it may be because there are other prerequisites that are unstated, but of which I have no knowledge.

Each time I get a suggestion, I spend hours trying to apply the direction given, and still with no result. It would be great if you can give the level of detailed instructions that Microsoft for example, put on their instruction pages.

I would like to end up with an installation of the whole system on my PC that is set up in such a way as it will play nicely with other apps that are installed on the machine. I am prepared to work at it a bit, but need very precise guidance. Will you or someone else on the forum help ? Thanks Allan. Regards

[Allan_Zimmermann]

Why are you not using “normal-up” ?

You cannot run offline, if you don’t know how to setup prerequisites, most importantly dns and configuring docker.
But if you know what your doing, and have dns properly setup and know what the consequence is of re-configuring the docker daemon, then I have update docker-compose-offline.yml and offline-pull to match openflow 1.5 ( added the new docker images and added agent_images
There is no offline folder, as you can see in the docker compose file, it’s using a volume

[bpin]

@Allan_Zimmermann

Why are you not using “normal-up” ?

Because in your last reply you referenced your video “Running OpenFlow offline and some MQTT stuff”, at https://www.youtube.com/watch?v=r_aEHZMSICE . The opening words of that video are “Hello. Today I would like to show how we can run OpenFlow completely offline.” Using “normal-up” is not shown in the first few minutes of that video. So from your comment, there must be prerequisites. And so before using the commands in the video (offline-down, offline-pull, and offline-up), should I first use

“git clone GitHub - open-rpa/docker”
“cd docker”

and if so should I do so when an empty Docker Desktop is open, or should I do so when it is closed, and if it should be done when DD is closed, at what point should I open it ?
Thanks

[Allan_Zimmermann]

I also made it very clear, that offline is most likely not what you need, and then i linked the video to show what it’s all about.

as the page clearly states.
Most people need to use docker-compose.yml / the normal- files
This will make openflow work on the local machine only
For deployments on servers, you need to follow “custom domain names” and then use either normal- files or letsencrypt- files depending on if you have a public ip and need certificates after updating the corresponding docker-compose file.

Offline is only meant for very specific cases, where openflow need to run detached from the internet , like cruise ship’s or OT networks with PLC’s

[bpin]

I am struggling a bit with the terminology. For the sake of clarity, I want to run an OON installation on my PC - a laptop. No servers are involved. The internet will be there most of the time, but not all the time. I just want to

1. download some installation files or get them from github, and then
2. run the OON installation locally -without having to use any cloud services and without necessarily being connected to the internet to keep it running (offline). How do I do that ? (I am using git, and Docker Desktop.)

The installation has also to coexist with everything else on the PC - as I will want to use OON in conjunction with other software on the machine, whether that is other software is more Docker images/containers, or just normal desktop offline software.
Thanks

[Allan_Zimmermann]

So the short answer is:
If your requirement is to run without internet/network, then no, that is not possible.

The long answer, yes, it is possible, but it is complex and not something I will support on the forum.

• OpenRPA => a windows application that can automate the UI ( unlike most other rpa vendors, openrpa does ONLY rpa, if you want to do api integration you should be using an agent or nodered via openflow )
  OpenRPA can run “normally” then it needs to be able to see an openflow, or it can run in an offline mode, where it does not require openflow. In the latter you can disable you network card if you want to.
  In the first, it does not not need internet per say, but it needs to be able to reach openflow doing startup
• OpenFlow => is basically a database and message queue with Access Control Lists, and the ability to orchestrate multiple agents via docker or kubernetes.
• Docker (not openflow) requires internet to start anything, unless you set it up in a special way, including running a local docker image repository. I cannot and I will not offer support on that on the forum.
• Once an instance of mongodb/rabbit and openflow has been started inside docker, non of those require internet access.
• Once you want to start an agent ( like nodered ) dns needs to be properly configured for things to work. If you disconnect/disable your network card, that will breaks. Again, i cannot and will not support that over the forum.
• Once an agent has been started, if the agent is based on nodejs ( like NodeRED is ) or python, a working internet needs to be accessible, so it can install all the needed packages. If you have the technical expertise, it’s is possible to run those repositories locally and configure various services to make them use those.
  Again, i cannot and will not support that over the forum.

[Allan_Zimmermann]

I just re-read everything a few times.
I can see why we “misunderstood” each other.
I was not clear in my first message.
You started normal, and then disconnected the internet.
I answered why disconnecting the internet will not work, and pointed you to a video showing why that is a complex thing to do. I can see why you might have read that as, this is what you need to do.
I’m sorry, that was not the intention.

Just to summarize.

• OpenRPA cannot startup without access to openflow, but once it has started, it should handle short disconnects just fine. But if you plan to do that for a long time, it will memory leak, since it will try and save the data that needs to save to openflow. So if that is an requirement, you should run it in “offline” mode ( without openflow )
• OpenFlow does not need internet access, but docker does. NodeRed does (doing startup ) and agents based on nodejs/python does. Traefik and agent’s depend on working dns. both from the host machine and inside docker.
  So requires a lot of work to make work “off the grid” but is definitely doable, I have a handful of customers that use that. But is not something i want to support over the forum. If you have the technical expertise and run in to an error somewhere I’ll be happy to pinch in with feedback, but I will not create a complete guide and then offer support on that. I already gave 90% of the solution in the offline docker compose files, it is then up to the user, to get the last parts working ( docker/dns )

[bpin]

Thank you for clarifying. I guess it’s a question of focus. I would now like to explore how OpenRPA, used offline can work, and what it will be able to do. I took a look at your website to get an overview (Pricing - OPENIAP ), and I saw these first lines relate just to OpenRPA.

    OpenRPA - RPA, no extras, no whistles and bells, just pure scalable, battle hardened and very secure RPA robots
    OpenRPA - Automate Windows, Java, SAP, IE, Chrome/Edge/Firefox, Mainframe, MS Office and using image recognition.
    OpenRPA - No language lock in, like Python. Recorder and low code Workflow, and embed code written in multiple languages like VB/C#/Python/PowerShell/JavaScript/Perl/Go etc.
    OpenRPA - Deploy scalable robot farms, easily scale up and down per server, and across as many servers you want.
    OpenRPA - Support for Child Session ( some times called Picture in Picture )
    OpenRPA - Can run stand alone or be connected to OpenIAP Flow for enhanced features and security.
    OpenRPA - Supports high density (HD) robots, for scalable and unattended robotics.
    OpenRPA - Fully controllable using PowerShell or NodeRED or Agents or OpenIAP Flow <<

Questions - does the above mean that by using OpenRPA in offline mode (in a non OON installation and without Docker) I can create and deploy multiple unattended robots on a windows machine ? I am reading the docs, and it looks like the answer is yes, which means that it should achieve my objectives in a far simpler way than using Docker ?
Would I need to run it with Powershell or is even that not necessary ?

The only question is in about the line “OpenRPA - Can run stand alone or be connected to OpenIAP Flow for enhanced features and security.”, what enhanced features would I be missing if I ran offline - ?

Thanks for your help

[Allan_Zimmermann]

@bpin

deploy multiple robots unattended robots

Depends on what you consider unattended. If you are running them in the console of a windows machine and handling scheduling workflows your self, sure you can run as many as you want.
If you mean High-Density Robots ( robots running inside Remote Desktop instances ) then no, that requires openflow.

@bpin

enhanced features

Access to API’s, Access to a secure database, access easy backup of all data across all agents/robots, access to secured credentials across robots/agents … just to name a few.

[bpin]

To answer your questions -
From my understanding, possibly neither of the examples you mentioned. I have also seen HD Robots slightly defined differently between vendors. So I will be as specific as I can for the sake of clarity, but using desktop analogies as I understand that terminology a bit better than client server. I have also re-read the offline page, and seen some of the postings on the forum including a reference to LiteDB.

In this context, I consider an unattended robot, either

• a robot that can be started automatically either at user logon or with the launching of a specific software package on my machine (where certain robot actions will be triggered by some kind of simple user level human intervention after the robot has started and is “listening”) - case 1, or
• where the launch of the robot will be controlled directly by a keyboard shortcut or some other kind of simple user level human intervention - case 2

Use cases
Single user

• if the attended robots are constructed as running all the time and listening - and I have had advice it is easier to program an individual robot if it is to run in case 1 mode rather than case 2 mode (and some robots may be very problematic to construct if they would have to run in case 2 mode), then there will definitely be a need to have more than one different robots running and listening at the same time.
• if a robot is designed for case 2 mode, the only time I can see it being necessary to have more than one robot is if a particular process does something like rebuild an index which takes sufficient time that the users will simply let that happen in the background and will start doing something else on the machine which might need the intervention of a robot. So in case 2, there might be a need to have more than one different robots running at the same time.

Multiple users

• in the case of multiple users logged on to the same workstation, one on the main screen and one or more via other devices connecting to the workstation each on a remote desktop instance, there is definitely a need to be able to have multiple different robots running concurrently when considering the machine as a whole. Each user might then need multiple robots themselves given the case 1 and case 2 examples above.

Regarding the enhanced features, this speaks to my question about necessary functionality. I can reword my questions slightly so that I better define the functionality I need.

The objectives I referred to in my last post are (to specify these) to

• have a set of unattended robots as I defined them above, running in the single user scenarios as I defined them above, in an accessible database on my machine which I can back up when required when in admin mode, and so that the process of starting and operating these is otherwise invisible to me when logged in as a user except when using keyboard shortcuts or other user type interventions, and
• so that I (using typical simple admin mode interventions), can administer the robots when I am logged in as an administrative user (when I will not be in “design mode”).

My questions

1. Does OpenRPA have the capabilities on its own or with LiteDB when used in the offline mode you linked to above, to achieve these objectives for both case 1 and case 2 operation with a single user ? (the multi-user scenario is ruled out as you said it needs OpenFlow)
2. If the answer is no, then please let me know which of these objectives can be achieved (with OpenRPA on its own or with LiteDB), and if none of them, what additional software would I need ? Could I use Powershell scripts for example ?

I would appreciate it if you can put this in very simple terms - I would be happy to discuss this offline and chat it through with you if it is easier.

Many thanks

[Allan_Zimmermann]

You are combining and mixing up words that most people working with RPA software would consider opposites. It’s really hard to answer if you don’t know the terminology but are using the words that has special meaning for people working with rpa software.

• unattended means a robot that works without human intervention. In small setups that would be on the console of a windows pc somewhere, in bigger installations that would run in some type of HD robot using RDP or VDI. In that cases workflows are triggered by a schedule ( can be coded in OpenRPA or triggered external using many different means ) or they get triggered by events ( could be a detector in OpenRPA, like the file watcher ) or externally using a webhook, a database trigger, a workitem queue, a message queue, an MQTT message etc.
• attended means something triggered by and/or assisting a user. These robots are either started directly inside OpenRPA by the user, or triggered by an action the user does ( copying a file, pressing a hot key, opening a specific window, using web forms in OpenFlow etc )
• Hybrids, seems to always go wrong, but that will not let people stop trying. Normally the main reason for doing that is either you want “the human” to keep an eye on the automation, but still be able to keep working ( that is when Child Sessions ( also called Picture in Picture ) can be handy), or to save on license cost. The latter should not matter when working with open source software.

But even RPA vendors can mess up, like UiPath that insist on calling a workflow a robot, but also have robot software running. Or other vendors that call a Process ( that can span multiple workflows and other technonolgies and/or humans ) a robot.
So to clarify,

• to us, a robot is an instance of OpenRPA running somewhere
• to us, a workflow is a single workflow running inside OpenRPA
• to us, an agent is a software package running either inside docker, remotly as a daemon or inside the desktop assistant. And Agent could be your own code, or common packages like NodeRED
• for us, a process is the people’s definition of a task you are trying to automate, that can span one or more workflows in OpenRPA, one or more workflows in NodeRED, one or more packages in agents, one or more human forms and one or more humans interaction with the process. When spanning more than one, we use workitem queues to control the flow and keep track where in the process we are.

If all you need is to automate something on a single pc, you don’t need central backup, you don’t need api access, and you don’t need access to a secure database, you don’t need access to an event based workflows or scheduling of workflows and so on. Then sure you can install OpenRPA in offline mode.

LiteDB is the database OpenRPA uses to cache things when connected to OpenFlow and/or is where it stores data when running in offline mode. It’s suppose to be thread safe, but there has been a few reports now and then, where the database has been corrupted, so if your running in offline mode, make sure to take backup often. If using openflow, you don’t need to care, in that case you can just delete the .db file and then robot will recreate it from the data in openflow. And all data will be handled when you backup mongodb.

you can run OpenRPA on premise, or in the cloud.
You can run OpenFlow on premise or in the cloud.
You have already been giving the guides on how to do that.

You can run OpenRPA without access to the internet.
Either by using it “offline” or by connecting it to an openflow that is also off the internet.
If running OpenRA offline, then some things will not work, like workitems and HD robots.
If you want to run OpenFlow without internet access, hire some people who has the expertise, I cannot and will not guide you on that over the forum. But if that is a requirement you have, yes that can be done.
If neither fulfill your need, you a blessed with at least 70 other rpa vendors, so there is plenty of options out there.

[bpin]

My overall objective atm is to find out whether OpenIAP offerings will be a suitable tool to achieve what I want to achieve, and if so in what way must they be used. I believe they are suitable, but I am not certain. I have been seeking clarification.

@Allan_Zimmermann

It’s really hard to answer if you don’t know the terminology but are using the words that has special meaning for people working with rpa software

It is also really hard to ask questions when there is in fact disagreement in the market and in the RPA discussion groups, about the meaning of words that, as you rightly point out, should have the same special meaning for people working with rpa software. It seems they don’t, as you have also confirmed. It makes the discussion quite difficult.

However, I am (and have always been) sufficiently clear about some aspects of the terminology to assist in narrowing this down, for example the essential difference between attended and unattended; for my purposes in this dialogue, your definition of attended, using the word something is very close both to UIPath’s definitions on their forum covering the same subject, and to my understanding that attended automation / (“something(s)” triggered by and/or assisting a user), is what I want. (despite my previously having muddied the water by using the word robot in a different way than the way you have now defined it)

For the sake of this discussion, this allows us to leave out what you call unattended, and also hybrid, and concentrate only on attended. We can also leave out the multiuser scenario as this is not envisaged at this stage. (My automations as envisaged are single user only.)

I really do not mind whose terminology I use, and for a discussion on this forum it would be best to use your terminology. It was very helpful your setting it out. There do remain a few questions to clarify these definitions, and it is probably best to hit these off before my asking the other questions, which I will be better placed to redraft.

@Allan_Zimmermann

attended means something triggered by and/or assisting a user. These robots are either started directly inside OpenRPA by the user, or triggered by an action the user does ( copying a file, pressing a hot key, opening a specific window, using web forms in OpenFlow etc )

Definition/Terminology questions -

• You define a robot as an instance of OpenRPA, how can a robot be started directly inside OpenRPA (second sentence of above definition of “attended”, word robots) ? I don’t understand this. Can you clarify ?

• If one is running OpenRPA in offline mode as per your link, can one instance (which in this context I take to mean that OpenRPA has only been opened once) run multiple different workflows at the same time, or does additional each workflow that is started mean OpenRPA is opened again for each additional workflow, thus also presumably building up the RAM requirement ? How does this work ?

• Using OpenRPA terminology, do I take it that a project inside OpenRPA is simply a collection of workflows ?

Once I have these points clear (sorry they are so basic), I will be able to restate my questions using your terminology. (FWIW - I believe the OpenIAP terminology means that where I used the word robot (in my post #12) I should replace it by the word workflow, and also what I called a case 1 should be called a long running workflow. Is this correct ?)

@Allan_Zimmermann

Long running workflow
You can create a workflow that never ends.
A State Machine or even a simple DoWhile with a sleep, is a perfectly acceptable way to , “wake up” once in a while and check if it’s time to run the workflow using Invoke OpenRPA activity

Thanks ! I would really appreciate it if you can answer these questions one at a time so that all the terminology is defined in terms that are not open to interpretation - that would be very helpful.

[bpin]

Hello - any thoughts ? I have seen some further info regarding the possibility of using Powershell, and this looks quite promising …