OCR-based Interaction with Legacy Application – Double Click on Detected Value

Hi Allan,

I’m working on a robot that interacts with an old project management software, which unfortunately only supports OCR-based automation due to its outdated technology. Here’s a short description of my current challenge:

Use Case:

The robot receives a numeric value (in Hebrew interface) from a list and is supposed to:

  1. Search for this number on the application window using OCR.
  2. Identify and remember the exact screen position of the number.
  3. Move the mouse to that location.
  4. Perform a double-click on it.

Issues I’m facing:

I’m struggling with the accurate detection of the value and the subsequent mouse action. Specifically:

  • I’m not sure how to store the OCR-detected position and pass it to a mouse action.
  • Double-clicking often fails or occurs in the wrong place.
  • I’m working in a Hebrew-language interface, which includes right-to-left layout and numeric values.

Notes:

  • I’m using only OCR activities (no selectors).
  • The robot types the number correctly, but I need help figuring out the best practice to locate and interact with the value via OCR.
  • Double-click action should happen precisely where the number is shown on the screen.

Any guidance or best practice examples (especially with Hebrew/RTL support) would be greatly appreciated!

Thanks a lot,
Amit
Weaversoft .io