Add files to workitem in NodeRED based on form

[Peter_Madsen]

Hi guys,
I was not able to find a guide on this so i will just create a new topic

I have a form in OpenFlow which looks like this:

which is connected to som nodes in NodeRED:

Now when i activate the workflow and submit a file in the form i can see the api property file which is an array. I assume this array can be attatched directly to the Workitem if assigned to the msg.files property.
function node Prepare WI:

msg.files = msg.payload.container.file

delete msg.payload

return msg

In my case the file component is within a container as seen here:

but when the workitem is created the file isnt attatched to it:

what is the right way to achieve this?

i also thought about query the openflow database files collection but i dont see an _id property of the file in the debug message

[Peter_Madsen]

I have also tried to just use a get request using the http request node but i get this error:

i havent found a way to make reuqests on NodeRED that allows to ignore ssl errors

I dont want to change the NodeRED env variables like this:
NODE_TLS_REJECT_UNAUTHORIZED=0

[Peter_Madsen]

Okay, so the http request node has a property that you can set to get around this:

but now i am confused how the request returns status 404 when i just provide the url of the link to the download.

if i paste the url directly into the browser i download the file without issues. but i get status 404 if i run the request from nodered or postman

[Peter_Madsen]

After inspecting the URL response, I noticed multiple headers that are presumably designed to ensure browser-like interaction for the request. Please correct me if I’m wrong, but this suggests that the URL is not intended to be used for retrieving the file content programmatically.

[Rutger_van_Schoonhov]

@Peter_Madsen

msg

Hi,
I have created an example for adding files to workitems in NodeRED.
Main thing: msg.files should contain a list of files.
The file itself should be a bufffer.

[
    {
        "id": "f7b0b5448c53a937",
        "type": "function",
        "z": "f2c1449a89fa18cc",
        "name": "create wi with file",
        "func": "// We want someStruct as a json file attached to a workitem\nconst someStruct = {\n    \"name\": \"example\",\n    \"greeting\": \"Hello world!\"\n};\n\n// I grabbed it from an example where I wanted something downloaded by an agent. \nvar downloadFile = {\n    \"filename\": \"test.json\",\n    file: Buffer.from(JSON.stringify(someStruct))\n}\nmsg.files = [downloadFile];\n\nmsg.newWorkitem = {\n    \"name\": \"test\",\n    \"payload\": {\n        \"key1\":\"Put your payload here\" \n    }\n}\n\nreturn msg;",
        "outputs": 1,
        "timeout": 0,
        "noerr": 0,
        "initialize": "",
        "finalize": "",
        "libs": [],
        "x": 1750,
        "y": 360,
        "wires": [
            [
                "096a5029d1f9c0c8"
            ]
        ]
    },
    {
        "id": "096a5029d1f9c0c8",
        "type": "workitem addworkitem",
        "z": "f2c1449a89fa18cc",
        "name": "",
        "config": "3ada51595225d3e1",
        "payload": "newWorkitem.payload",
        "payloadtype": "msg",
        "workitem": "workitem",
        "workitemtype": "msg",
        "topic": "newWorkitem.name",
        "topictype": "msg",
        "nextrun": "nextrun",
        "nextruntype": "msg",
        "priority": "2",
        "prioritytype": "num",
        "files": "files",
        "filestype": "msg",
        "success_wiq": "success_wiq",
        "success_wiqtype": "msg",
        "failed_wiq": "failed_wiq",
        "failed_wiqtype": "msg",
        "x": 1930,
        "y": 360,
        "wires": [
            [
                "391caa8edfa16ee7"
            ]
        ]
    },
    {
        "id": "c827806d3f8dd9b0",
        "type": "inject",
        "z": "f2c1449a89fa18cc",
        "name": "",
        "props": [
            {
                "p": "payload"
            },
            {
                "p": "topic",
                "vt": "str"
            }
        ],
        "repeat": "",
        "crontab": "",
        "once": false,
        "onceDelay": 0.1,
        "topic": "",
        "payload": "",
        "payloadType": "date",
        "x": 1580,
        "y": 360,
        "wires": [
            [
                "f7b0b5448c53a937"
            ]
        ]
    },
    {
        "id": "391caa8edfa16ee7",
        "type": "debug",
        "z": "f2c1449a89fa18cc",
        "name": "test file",
        "active": true,
        "tosidebar": true,
        "console": false,
        "tostatus": false,
        "complete": "true",
        "targetType": "full",
        "statusVal": "",
        "statusType": "auto",
        "x": 2080,
        "y": 360,
        "wires": []
    },
    {
        "id": "3ada51595225d3e1",
        "type": "iworkitemqueue-config",
        "wiq": "testq",
        "wiqid": "",
        "name": "testq"
    }
]

Hope it helps.
Gr,
Rutger
PS: You can select and copy the code, then press CTRL - I (import) in NodeRED, paste, click Import and deploy.

[Peter_Madsen]

Thank you, for sharing your solution

Maybe i am missing something, but i dont see how i can use this to add the files directly chosen in the openflow form. Will this work with that too?

[Rutger_van_Schoonhov]

I am sorry, I was too quick. I thought you had the data already in NodeRED.
I need to dive into that with the form. (it has been years since I last used forms, but I should have an example somewhere).
I’ll take a look, but it may take me some time as deadlines at work are approaching rappidly…

[AndrewK]

I was curious about this one as well, and tried a small workflow, and I think I’m also missing something.

What I got working, although it’s clearly a workaround, was:
Upload the file through the form.
Get the name property out of the payload (the one with the guid in it, not the original name), query the files collection for the id of the object with that uniqueName, and add that to the workitem. After that, in OpenRPA/agent, fetch it using the id and GetFile/download.

What I tried that didn’t work:

• Use download node in NodeRed to get the file as a buffer to pass it to workitem → access denied (why? I don’t know - it seems illogical, as all the ACL seem fine)
• Use GetFile in OpenRPA with UniqueName as the name → file not found (it can only download by the non-unique filename, and that is not what’s needed in most cases)

I’m pretty sure I’m missing something, but that feels like a very roundabout way of doing it, and I’m pretty sure there should be a shorter path to this that I don’t know.

[Peter_Madsen]

No worries i appreciate the thought. GL at work!

[Peter_Madsen]

Exactly my thoughts as well. It seems impossible to use the URL directly from the file component as there are so many headers required on the download link.

That was also the only workaround i could get to work, but it does seem a bit of a detour as it feels like the informaiton should be right there, easy to access and pass, at least having the file _id property be available.

I’m currently working on a solution to make this proces a bit more stream-lined

[Peter_Madsen]

I came up with a subflow to make things easier.
If anyone has a different solutions im still very curious to see that.

If you want to check out the subflow you can find it here:
Get files from OpenFlow - NodRED - Guides - OpenIAP

[AndrewK]

Thanks Peter, the subflow looks nicely done (also learned a thing or two about designing more readable subflows thanks to this ).

Question though - what was the reason for the static delay at the start?

One thing to note, is that using this method to attach the file to wi seems to be duplicating the file - the form uploads it, then the wi submission uploads it again from the buffer, so that needs to be potentially cleaned out in the calling workflow (f.e. deleting the original file after adding the wi, to avoid clutter).

That said, all of this would be much easier if the upload form would be adding the file _id to its payload (which could be used in download file directly as a unique identifier).

[Peter_Madsen]

you are welcome, im glad to contribute to this awesome forum.

Im glad you mention the bit about adding the file to the workflow it took me a while to figure out where this extra file came from i didnt decide on removing the extra file but yes it is not convinent to have a dupe.

but in my opinion i think it would make a lot more sense to have a build in property on the add workitem node like (attach files from openflow using uniqueue names) so that the dupes could bed avoided

also the delay is just to quickly reflect the nodes status when activated. hope it makes sense

indeed totally agree with the _id that was what i attempted to begin with

[Allan_Zimmermann]

In OpenFlow, i check if the _id and filename exists in the array,
but since grpc changes that to id i’m unsure if that logic works right now, with NEW workitems …

I need a little time to setup a reproduction of this, but then i can make sure that you CAN add an existing uploaded file to a work item, so you don’t get duplicates.

[Peter_Madsen]

That would be very convientient and a great feature. But we also need to make sure that the unique identifier is accessible through the compenent file’s api in the openflow form

[Allan_Zimmermann]

Ah, yeah, forgot to address that.
So when you upload a file on a form, it needs to be unique, but since the form does not have full context, AND can be reloaded multiple times, I use forms.io’s id in the database. That is why you need to find the file by searching the file(s) by searching for uniquename in the file(s) metadata … field