We’ve been experimenting a bit with the (old) dotnetapi, and have a couple of questions related to it:
Is there any point in opening issues/submitting PR’s, since the new one is in the works?
How can we authenticate from outside the agents, for local debugging/testing? The signin call looks simple, but not sure what the apiurl should be when running from local. I guess we could use openrpa to “fetch” a debugging jwt from there?
The examples do a RegisterQueue on signin, but I’m not sure what’s the purpose of it (since it makes the example much more complicated as well).
I’ve checked the videos about agents, but didn’t find an answer to the above.
My progress on the Rust code has been drastically down. I’m getting bombarded with requests and meetings, so it might be “ok” to also maintain the .NET package for a while. (Just today, I’ve been in non-stop meetings for the last 5 hours.)
Is this for “agent code,” or are you calling OpenFlow from some other place that is not intended to run inside an agent?
The thinking was, you use launch.json with different settings set by calling “initialize project” inside VS Code, and this will set the apiurl and JWT environment variables. That means when you deploy it, it will use whatever gets supplied by the agent when the code is called inside the agent. But I suspect you are asking since you are using Visual Studio? Since my VS Code extension does not work in Visual Studio, you will need to handle that in some other way. I’m really not sure what would work best for you. You can set it on the project or set them before launching Visual Studio. If you don’t want to manage this using environment variables, you can always call connect and sign in yourself. (Connect supports you giving it the URL, and sign in has two overloads, one for username/password and one for using a JWT token.) If you need to generate a token, you can just log in to OpenFlow as the user you need to run as and then open the /jwtlong URL.
3. All my examples use the “OnSignedin” event. I do this to show how to make code that can work even if you lose connection to OpenFlow.
For code that you will run “linea,” you can just await a call to Connect() and then signin(), do the work, and then release the client.
The client is “missing” a prober “close” function, that will allow you to disconnect without reconnecting, let me know if you need this
Ok, I’ll see if we can have some things there PR’d (the download file f.e. requires a JObject casted as object for a query instead of the usual string, which it then converts from object to JObject and then to string).
I’m using VSCode, but the init project doesn’t detect anything but js (even though I definitely have C# ), so no proper launch.json for me. Hence I cloned the sample project (the workitemagent). But that one doesn’t have it either.
Currently stuck at “unknown client type” when trying to signin, which is I think the null response handler for protowrap.SendMessage, so I guess there’s something off with the apiurl maybe? How should it look like when connecting from outside, when the openflow is under https://openflow.mydomain.com?
Ok, noted.
PS. Thinking a bit, when you asked this:
I guess the same methods as used for testing, could be used for connecting from any arbitrary application, right? So for example if we have a different .net project (let’s say a cli somewhere), we could add the dotnetapi to it, and (with the no-injections method) connect to OpenFlow from there?
I had that too, once i remembered to await at protowrap.Connect(client) it went await, but now ( and i would swear it would a few hours ago ) i cannot make it login ! without adding a delay … everything really, fails ( and runs as guest )
I must be missing an await in my own client code, because if i await Task.Delay(2000) after Connect it works …
Maybe this will help you, this is my launch file without the token
{
"version": "0.2.0",
"configurations": [
{
// Use IntelliSense to find out which attributes exist for C# debugging
// Use hover for the description of the existing attributes
// For further information visit https://github.com/OmniSharp/omnisharp-vscode/blob/master/debugger-launchjson.md
"name": ".NET Core Launch (console)",
"type": "coreclr",
"request": "launch",
"preLaunchTask": "build",
// If you have changed target frameworks, make sure to update the program path.
"program": "${workspaceFolder}/bin/Debug/net6.0/dotnetworkitemagent.dll",
"args": [],
"cwd": "${workspaceFolder}",
// For more information about the 'console' field, see https://aka.ms/VSCode-CS-LaunchJson-Console
"console": "integratedTerminal",
"stopAtEntry": false,
"env": {
"apiurl": "grpc://grpc.app.openiap.io:443",
"jwt": "eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVwwedwedwe",
"wiq": "dotnetagent"
}
},
{
"name": ".NET Core Attach",
"type": "coreclr",
"request": "attach"
}
]
}
Aaaand it works
It looks like it was the apiurl.
Interestingly, it works for me without the delay, but not always (same on the agent in openflow, it sometimes needs a second try on startup, and afterwards works just fine).
, and it exits out of the reconnecting call early, making subsequent calls fail with InvalidOperationException → can’t write, because the previous write is still in progress
At first I thought it might be this line:
But after looking a bit more into it, I think it might actually/also be the Task.Run calls in the “fire and forget” style for logging in. Or both. Theoretically it’s an AsyncDuplexCall, so it should implement the .GetAwaiter() for an await call, so an await there might be feasible as well.
Looking through the output of my little test, I can see that the issues start spiraling when it Setting up server stream grpc.XXX:443 happens and then before the OnSignedIn fires a PopWorkItem (or any other call) happens.
So I think it might be the signin calls that are causing it, by either:
a. They’re still running when a new call is scheduled, therefore the .MoveNext issue with concurrent writes
b. The client is unauthorized (disconnected) when the call happens, and it doesn’t fully await.
Disclaimer: I have a very shallow understanding of this, so it might be completely off, aside of the output I see in the test.
So that reinforces the conclusion that the fire and forget _ = Task.Run calls are the cause, although the stream setup might be as well.
For whatever reason my VS doesn’t like that project, so I can only open it as single files, so can’t build it properly, so this is the extent I can help with this, I think.
So my proposed solution would be to change all the Task.Run background calls within there to standard awaits (effectively not offloading it), since otherwise we’re introducing race conditions and threading issues. At least that’s my current understanding.
I pushed a new version 0.0.9 that includes a static method for connecting and awaiting sign-in. So create your client like this, then you can call a function without delay right away.
Great, thank you
Since this is a static initialization, what about reconnecting?
(the grpc disconnects after 1 minute)
Right now it’s handled with some retrying helpers on operations, and an OnDisconnect event, but since this creates a new client (if I read it correctly), not sure if this will help for the reconnecting part.
To elaborate:
I’m using the following .OnDisconnected:
Which is the same as the QuickConnect uses underneath:
So while this might be solving the initial connection, the reconnection I think still works the same?
I’m running the tests right now.
Of course there might be a reason on my end why it disconnects every minute, but I’m not sure how to handle that otherwise.
If I don’t have the reconnect callback, it just awaits forever.
Just to be clear → we have a working workaround for this.
It’s not pretty, but it works, so please don’t treat this as anything urgent or anything like that.
Also, I’ll be away next 2 weeks, so won’t be able to test too much.
Not sure how helpful this will be, but from time to time it also errors on something like this:
�Retry attempt 1 after delay of 250ms after exception: System.Collections.Generic.KeyNotFoundException: The given key '4736c748' was not present in the dictionary.
at XXXagent.RetryHelper.RetryAsync[T](Func`1 func, Int32 retryCount, Int32 initialDelayMs) in /app/package/package/RetryHelper.cs:line 63
< at openiap.PopWorkitem(String wiq, Boolean includefiles)
7 at protowrap.RPC(openiap client, Envelope envelope)
A at System.Collections.Generic.Dictionary`2.get_Item(TKey key)
And after running it for ~24h, it just got pretty much stuck on these types of errors.
The agent currently checks some queues every 5s, I guess it could be better to add a proper watch there, but I’m not sure how to (and didn’t have time to figure it out). It is highly possible that we’re just doing something wrong due to a gap in understanding the underlying systems.
@RadoslawNowakowski will be continuing this project during my absence, so maybe he’ll have some findings next week that could help out in narrowing out the weird disconnects.
Weird. I don’t use the dotnap API a lot, so I have limited experience, but I have had agents running for days and one of my customers has had an agent running for months without issues.
Is this on a Docker agent or your local machine? You mentioned a lot of disconnects.
The idea is:
The agent registers a message queue and waits for messages. When it gets a message, it pops an item from a work item queue and processes it.
So, you create a work item queue and set AMQP to the queue the agent registered. Then OpenFlow will send a message to the queue when there are items ready to be popped.
This way, you don’t need to “pop” to check for new items at an interval.
For low-intensity jobs or people “in a hurry,” there is an alternative.
On a work item queue, you can also select an agent and a package. The package must be a non-daemon package (run, process item, quit).
In that case, OpenFlow sends a message to the agent service:
Service pops item from work item queue.
Stores payload in a file and saves the file path in the payloadfile environment variable.
Stores all work item files in the package folder
Runs the package.
Once the process exits, reads the content of payloadfile and updates the payload.
Adds any new files in the package folder to the work item.
If the package exit code is not 0, retry the work item; otherwise, mark it as completed.
Perfect for simple scripts that don’t need to “talk” to OpenFlow.
(and didn’t have time to figure it out). It is highly possible that we’re just doing something wrong due to a gap in understanding the underlying systems.