What makes an intelligent process automation tool actually intelligent

The definition everyone agrees on, and where it stops being helpful

Pull up the popular explainers and the listicles and they converge. Intelligent process automation is RPA with a layer of machine learning, natural language processing, or generative AI on top so it can deal with messy inputs. Then the article becomes a list of vendor names. None of them tell you the one thing that predicts whether a given tool will run on a Tuesday in your environment six months from now.

Here is the uncomfortable part. The word "intelligent" usually refers to the bolt-on AI: document classification, an LLM that drafts an email, a model that routes a case. But that intelligence sits downstream of a much older problem. Before any model can reason about a step, the tool has to perceive the screen and act on the right control. If perception is brittle, the smartest model in the world is reasoning over a guess.

What the tree actually looks like inside Mediar

This is the part you cannot get from a category overview. When Mediar records a workflow, it does not save a video. It captures the accessibility tree before and after every action, then represents each element in a simplified format. The desktop recorder spells the format out in a single line of context it hands to the model (in recording_processor.rs):

UI_TREE_STRUCTURE: The UI tree is a simplified
representation of the accessibility tree. Each line
has the format:
'LineNumber. RomanNumeralIndentation. [Role] 'Name' {Attributes}'.

Two details matter. First, the capture is paired: a before tree and an after tree surround each action. The diff between them is how the tool learns what a step actually did, which means a step is recorded as intent ("the Customer Code field gained focus and then held a value") rather than as a raw click at a pixel. Second, the representation is a role-and-name hierarchy, not an image. A model reasoning over [Edit] 'Customer Code' has a real description of the screen to work with. That is what the word "intelligent" should point at: the agent reasons over structure, not over a screenshot.

The same recorder and the same accessibility primitives are open source under the Terminator SDK, so teams that want to verify or extend how the tree is read can do it directly rather than taking a marketing page on faith.

Where this actually pays off

The perception argument is not abstract. It maps directly to the systems that have defeated traditional RPA programs: SAP GUI and SAP Business One, Oracle EBS, mainframe terminals, banking core systems like Jack Henry, Fiserv, and FIS, and EHRs like Epic and Cerner. These share a trait. They expose no clean API and no stable web DOM, and their layouts and field identifiers drift between versions. A tool that reads structure rather than pixels has a fighting chance there.

To be honest about the boundary: if your work lives entirely in modern web apps with clean APIs, a browser-based agent is often the simpler choice and you may not need any of this. The accessibility-tree approach earns its keep specifically on the no-API desktop layer, the place browser agents cannot reach. Picking an intelligent process automation tool is really a question of which of those two worlds your hardest workflows live in.

How to evaluate an intelligent process automation tool

When a vendor demos, the demo always works. The questions that separate tools are the ones that probe perception and failure:

• How do you identify an element? If the answer is coordinates, image matching, or auto-generated selectors, expect maintenance every time a screen changes. Ask whether it reads the accessibility tree.
• What happens when the UI updates? Push for the mechanism, not the adjective. A real answer describes an ordered fallback, like ID, then bounds, then text. "Our AI adapts" is not a mechanism.
• Can it run on my legacy desktop apps at all? Name your worst system out loud, SAP GUI or a green-screen terminal, and ask for a recorded run against it, not a slide.
• Can I audit what it captured? A workflow you can read and version-control is reviewable. An opaque recording is not. Mediar emits an inspectable workflow file rather than a black box.

You can record your own first workflow in the no-code web app at app.mediar.ai/web and watch the before/after tree capture happen on a screen you already know.