UiNode Methods - spectrum968/UiPath GitHub Wiki

Table of Contents

UiNode class Methods Child Click Clone DragAndDrop ExpandClippingRegion ExtractData FindAll FindFirst FindImage FindString FromDesktop FromRelativePoint FromScreenPoint FromScreenRegion FromSelector FromUIElem FromWindow Get GetCaretPos GetBrowser GetFocus GetSelector GetTopLevelWindow HasAttribute HideCaret Hover InjectAndRunJS IsEqual IsCoveredByAnotherWindow IsValid IsVisible MatchSelector Next Parent Previous Reset Scrape Screenshot SelectInteractive Set SetFocus ShowCaret StartHighlight StopHighlight TopParent WaitAttribute WaitDisappear WaitForReady WaitImage WriteText UiNode class

Methods

Child

UiNode Child(string selector) Finds a UI object, child of the current node according to selector search criteria.

Parameters selector - XML string specifying the conditions the child UI object should meet Return a new UiNode that meets the selector. Remarks

Child method is a shortcut for FindFirst with scope set to UI_FIND_DESCENDANTS.

Click

UiNode Click(int dx, int dy, UiClickType clickType, UiMouseButton mouseBtn, UiInputMethod inputMethod) Simulates a click on the current UI object.

Parameters dx
horizontal displacement of the click position

dy
vertical displacement of the click position

clickType
single or double click (see: UiClickType)

mouseBtn
left, middle or right click (see: UiMouseButton)

inputMethod hardware events, Win32 messages or node-dependent (see: UiInputMethod)

Return the current node making method chaining possible. 'Errors

E_UINODE_INVALID_CLICK_BTN and E_UINODE_INVALID_CLICK_MODE are raised for invalid clickType and mouseBtn input parameters.

E_UINODE_CLICK_ACTION_ERR is raised if the combination of clickType + mouseBtn is not supported by the current UI object.

E_UINODE_CLICK_GENERIC_ERR is raised when Click fails due to unknown errors in the underlying automation technology (like Active Accessibility, Java Accessibility Bridge, HTML events).

Remarks

dx and dy are relative to clippingRegion or to top-left corner of the UI object if no clippingRegion is set. If the current node represents the desktop or is uninitialized then dx and dy are in screen coordinates.

Clone

UiNode Clone() Creates new object that is a copy of the current instance.

DragAndDrop

UiNode DragAndDrop(int x, int y, UiNode targetNode, int targetX, int targetY) Simulates drag and drop action from current UI object to targetNode UI object. This method is not yet implemented!

Parameters (x,y) - starting point of the drag & drop action targetNode - the drop target UI object (targetX, targetY) - ending point of the drag & drop action Remarks

x and y are relative to top-left corner of the current UI object or to clippingRegion if one is specified. targetX and targetY are relative to top-left corner of targetNode or to targetNode. clippingRegion if one is specified.

ExpandClippingRegion

UiNode ExpandClippingRegion(UiDirection dir) Expands the current clipping region of a UiNode object.

Learn more about UiDirection.

Parameters dir - the direction to expand the clipping region Return the current UiNode, making method chaining possible. Remarks the clipping region is expanded to the left, right, top or bottom of the UiNode according to a given UiDirection. ExtractData

string ExtractData(string dataSelector) Extracts structured / tabular data from web pages.

Parameters dataSelector - XML text describing the data to be scraped. Usually dataSelector is automatically generated by UiWorkflow extract data wizard. Here is a sample describing Google search results:

object[] FindAll(UiFindScope scope, string selector) Retrieves a collection of UiNode objects according to selector search criteria.

Parameters scope
direct children, all descendants or top levels (see: UiFindScope)

selector XML string specifying the conditions the UI objects in the collection should meet

Return an array of UiNode objects that meets the scope and selector. Errors

E_UINODE_SYNTAX If the selector is invalid this error is raised. E_UINODE_INVALID_FINDALL_CALL is raised when the selector and the type of the current node does not match (see Remarks).

Remarks The parameter selector is a non-empty XML string that must contain only one XML element: <type_collection attr_1="val_1" attr_n="val_n"></type_collection> where type_collection is one of: wnd, ctrl, java, webctrl. If there are no XML attributes then all elements of type_collection type are considered. For some XML attributes, * wildcard is accepted as detailed in the table below:

type_collection Valid attributes wnd aaname, app, title, cls, ctrlName, ctrlId (* wildcard supported by all attributes) ctrl name, automationId, labeledBy (* wildcard supported), role java name, virtualName (* wildcard supported), role webctrl tag + any HTML attribute (* wildcard supported) Note: Learn more about how to generate these node Selectors here. Calling constraints:

// Assume that uiNode is valid object of type_node type. uiNode.FindAll(scope, "<type_collection attr_1=val_1 … />"); Not all type_node - type_collection combinations are valid as explained in the table below:

type_node type_collection wnd wnd wnd ctrl wnd java (if uiNode.hwnd is a Java window) ctrl ctrl java java html webctrl webctrl webctrl webctrl ctrl (if the HTML element is a window-less or ) Here is a JScript / WSH sample.

var uiElem = WScript.CreateObject("UiNode");

// The user manually selects a window on screen. uiElem.SelectInteractive();

// Find the collection of all descendant windows. var collection = uiElem.FindAll(1, ""); var text = "";

// Browse the collection and extract texts. for (var i = 0; i < collection.length; ++i) { var crntElem = collection.item(i);

 text += crntElem.GetValue();
 text += "\n";

}

WScript.Echo(text); FindFirst

UiNode FindFirst(UiFindScope scope, string selector) Retrieves a UI object, child of the current node or top level, according to scope and selector search criteria.

Parameters scope
direct children, all descendants or top levels (see: UiFindScope)

selector XML string specifying the conditions the UI object should meet

Return a new UiNode that meets the scope and selector. Errors

E_UINODE_INVALID_IDENTIFIER If no UI object is found then this error is raised.

E_UINODE_SYNTAX If the selector is invalid then this error is raised.

Remarks

For UI_FIND_TOP_LEVELS the selector must contain only one XML element. The current UiNode can be uninitialized only for UI_FIND_TOP_LEVELS otherwise E_UINODE_UNINITIALIZED_ELEMENT error is raised if the element is not initialized.

FindImage

UiRegion FindImage(UiImage pImage, double dAccuracy) Searches the current UI object on screen for the position that matches the given image with a specific accuracy.

Parameters image - the image that is to be found inside the current UI object accuracy - percent unit (0, 1] of the minimum similarity between the images Return

a UiRegion which represents the rectangle of the located image. This rectangle is relative to the UI object.

Errors

E_UINODE_IMAGE_NOT_FOUND error is raised if the image is not found with the given accuracy.

Remarks If the UiNode object has a clippingRegion set then this method searches only in that region. If you want to get a collection of all matches see the FindAll method

FindString

UiRegion FindString(string text, int occurence, UiScrapeOptions options) Searches for text in a given UI object and returns the bounding rectangle of the text found. Learn more about UiScrapeOptions.

Parameters text
the text you are looking for.

occurence
specifies how many times the text has to be found in order to have a final match.

options
UiScrapeOptions object that specifies how to extract the text.

Return a UiRegion which represents the rectangle of the located text. This rectangle is relative to the UI object. Learn more about UiRegion

Errors E_INVALIDARG error is raised for invalid input parameters. Remarks

If the UiNode object has a clippingRegion set then this method searches only in that region. When the text is not found it returns a null UiRegion. The text parameter is case sensitive. Occurence is 0 based, which means that you must specify 0 for the first occurence.

FromDesktop

UiNode FromDesktop() Connects the current node to the top level desktop UI object.

Return the current node making method chaining possible. Remarks

If the current UiNode is already initialized, its internal data are released before connecting to desktop UI object. If the current node represents the desktop then Click and Hover use screen coordinates to simulate mouse actions. Also WriteText will simulates keystrokes on the window that has the focus.

FromRelativePoint

UiNode FromRelativePoint(int x, int y, UiRelativeTo relTo) Finds an UI object based on (x,y) offset relative to the current UI node and a specified anchor point.

Parameters x horizontal offset relative to current node and its anchor point

y vertical offset relative to to current node and its anchor point

relTo specifies the anchor point: top-left, top-right, bottom-left, bottom-right or center; see UiRelativeTo

Return a new Uinode based on offset coordinates. FromScreenPoint

UiNode FromScreenPoint(int x, int y) Initializes the internal data of the current UiNode with the UI object that contains the screen point (x, y).

Parameters x
horizontal coordinate of point

y
vertical coordinate of point

Return the current node making method chaining possible. Errors E_UINODE_UNINITIALIZED_ELEMENT error is raised if the element is not initialized. Learn more about E_UINODE_UNINITIALIZED_ELEMENT.

Remarks

If the current UiNode is already initialized, its internal data are released before initializing from screen point.

FromScreenRegion

UiNode FromScreenRegion(UiRegion pVal) Initializes the internal data of the current UiNode with the UI object that contains the UiRegion pVal.

Parameters pVal
a UiRegion in screen coordinates.

Return the current node making method chaining possible. Errors E_INVALIDARG error is raised for invalid input parameters. Remarks

The UiNode returned is the smallest UI object that encloses the UiRegion passed as parameter. If there isn't any UI object that completely encloses the UiRegion, than the UiNode returned is the top level window that contains the UI object located at the center of the region. The clippingRegion of the current UiNode represents the UiRegion relative to UiNode's screen position. The clippingRegion of the current UiNode can have negative coordinates.

FromSelector

UiNode FromSelector(string selector) Initializes the internal data of the current UiNode according to Selector parameter.

Parameters selector XML string specifying the conditions that the user interface object should meet

Return the current node making method chaining possible. Errors

E_UINODE_INVALID_IDENTIFIER If no UI object is found then this error is raised.

E_UINODE_SYNTAX If the selector is invalid then this error is raised.

E_UINODE_CANCELED_ERR is raised if the current operation is canceled (see Cancel event).

Remarks

The method searches for a UI object that matches the selector in the amount of time specified by timeout property.

FromUIElem

UiNode FromUIElem(object uiElem) Converts an existing UIElement to UiNode.

Parameters uiElem the UIElement object that will be used to initialize the current UiNode

Return the current node making method chaining possible. Remarks

UIElement library is deprecated. New projects should use UiNode instead. If the current UiNode is already initialized, its internal data are released before conversion. For Java UI objects the conversion is actually a transfer of ownership; the uiElem will become uninitialized after conversion.

See also: UIElem.InitializeFromUiNode and How to mix old and new API.

FromWindow

UiNode FromWindow(UiWindow window) Converts a UiWindow library object to UiNode.

Parameters window the UiWindow library object that will be used to initialize the current UiNode

Return the current node making method chaining possible. Remarks If the current UiNode is already initialized, its internal data are released before connecting to window UI object. Get

object Get(string attr) Retrieves the specified attribute value.

Parameters attr - the name of the attribute Return Depending on the attr name, a string, bool or object is returned. Errors E_UINODE_INVALID_ATTRIBUTE is raised if attr is not supported.

Learn more about E_UINODE_INVALID_ATTRIBUTE

Remarks

For attributes type and name please check the Attributes section.

GetCaretPos

UiRegion GetCaretPos() Retrieves the client coordinates of the caret's current position.

Return the caret's position as a UiRegion object. Remarks the result is actually a point, the UiRegion object having zero widht and height.

GetBrowser

UiBrowserGetBrowser() Creates a UiBrowser object out of a UiNode. Learn more about UiBrowser

Return a new browser object. Errors E_INVALIDARG is raised if the current UiNode is not a or node.

Remarks

UiBrowser provides more browser-specific features for nodes that are web pages or web controls.

GetFocus

UiNode GetFocus() Retrieves the UI object that has the focus. This method is not yet implemented!

Return the node that has the focus or null. Remarks It starts with the current node and searches for a descendant that has the focus. The method returns the current node itself if the current UI object has the focus. If no object has the focus then null is returned. GetSelector

string GetSelector(bool refresh) Computes the selector of the current UiNode.

Parameters refresh if true, the function will recompute the selector. If the function is called for the first time, the selector is calculated anyway.

Return a selector in XML format. Errors

E_UINODE_UNINITIALIZED_ELEMENT is raised if the node is not initialized.

E_UINODE_INVALID_ELEMENT is raised if the user interface object is no longer valid.

Remarks

UiNode objects initialized by FromSelector already have a selector.

GetTopLevelWindow

UiWindow GetTopLevelWindow() Creates a UiWindow object out of a UiNode.

Return a new UiWindow associated with the top level parent window of the current UI object. Remarks

UiWindow provides more window-specific features.

HasAttribute

bool HasAttribute(string attr) Determines whether the current node has the specified attribute. This method is not yet implemented!

Parameters attr the name of the attribute

Return true if the current node supports attr, false otherwise. Remarks

For more information check Attributes section.

HideCaret

void HideCaret() Removes the caret from the screen. This method is not yet implemented!

Hover

UiNode Hover(int dx, int dy, UiInputMethod inputMethod, int time) Simulates mouse hover event over the current UI object. Learn more about UiInputMethod.

Parameters dx
horizontal displacement of the mouse cursor position

dy
vertical displacement of the mouse cursor position

inputMethod hardware events, Win32 messages or node-dependent (see: UiInputMethod)

time
the amount of time to keep the mouse cursor over the UI object

Return the current node making method chaining possible. Errors

E_UINODE_CLICK_ACTION_ERR is raised if the current UI object does not support inputMethod for simulating hover.

E_UINODE_CLICK_GENERIC_ERR is raised when Click fails due to unknown failures in the underlying automation technology (like Active Accessibility, Java Accessibility Bridge, HTML events).

Remarks

dx and dy are relative to clippingRegion or to top-left corner of the UI object if no clippingRegion is set. If the current node represents the desktop or is uninitialized then dx and dy are in screen coordinates.

InjectAndRunJS

string InjectAndRunJS(string code, string input) Injects and executes JavaScript code into HTML document containing the current UI node.

Parameters code Java Script code to be injected and executed or a file path of a text file containing the code to be executed.

input string input parameter for the JavaScript code.

Return a string object returned from JavaScript code. Remarks The JavaScript contains a single anonymous function like bellow: function (element, input) { return "result"; } where element is the HTML element corresponding to the calling UI node and input is the input string provided by the caller. Any JavaScript exception will be propagated to InjectAndRunJS method.

IsEqual

bool IsEqual(UiNode anotherNode) Compares two UiNode objects.

Return true if the two UiNodes point to the same user interface objects, false otherwise. Remarks You can have two different UiNodes pointing to the same user interface object on the screen. The method does not compare UiNodes objects themself but the UI objects they are pointing to. IsCoveredByAnotherWindow

bool IsCoveredByAnotherWindow() Checks if the current UiNode object is totally or partially covered by another window.

Return false if the UI object is not covered, true otherwise. IsValid

bool IsValid() Checks if a UiNode object is in a valid state or not.

Return true if the UI object is still valid, false otherwise. Remarks Even if you have a valid UiNode object, the corresponding user interface object can be destroyed (e.g. a window was closed). In this case IsValid returns false. IsVisible

bool IsVisible() Gets a value indicating whether the current UiNode is visible in the user interface (UI).

Return true if the UI object is visible, false otherwise. MatchSelector

bool MatchSelector(string selector, bool matchChildren) This method checks if the current UiNode object can be uniquely identified by the specified selector.

Parameters selector
XML string which will be matched against the current UiNode object.

matchChildren specifies if the children of the UI nodes matching the selector should be considered as "matching".

Return This function returns true if there is a unique correspondence between the current UiNode object and the given selector. Otherwise, it returns false. Errors

E_INVALIDARG will be raised for an invalid selector parameter.

E_UINODE_SYNTAX will be raised if the selector parameter does not have a proper XML syntax.

Remarks

The "classic" method of checking if a UiNode matches a selector is to initialize a second UiNode object from that selector using the FromSelector method and then comparing the internal data of the objects. This approach is very inefficient, because, most of the time, you will have a mismatch coming from the very first child node in the selector.

MatchSelector is a fast method of checking whether the selector can uniquely describe the current UiNode. Unlike the FromSelector method, MatchSelector does not iterate the whole hierarchy specified by the selector. This method is able to stop at the first attribute mismatch, which will probably occur very early, if the selector does not match.

Moreover, the XML nodes in the selector which describe intermediary UI nodes in a unique manner can be dismissed faster by the matching algorithm. The XML nodes that have the index attribute set are considered to be not unique among the parent sub-tree. If the index attribute is not set, then the XML node indicates a unique UI sub-node. Our methods and tools which generate selectors verify this uniqueness and set the index attribute accordingly. If you write the selector yourself, or modify an existing one, make sure that the intermediary UI node is unique among the parent sub-tree before removing its index attribute.

Next

UiNode Next() Retrieves the next node in the UI objects hierarchy. This method is not yet implemented!

Return next node relative to current node or null. Remarks The next logical object is returned, generally a sibling to the current object or null if there is no next node. Parent

UiNode Parent() Retrieves the parent node of the current UiNode.

Return parent UiNode or null. Remarks null if the current node has no parent (it is a top level node). Previous

UiNode Previous() Retrieves the previous node in the UI objects hierarchy. This method is not yet implemented!

Return previous node relative to current node or null. Remarks The previous logical object is returned, generally a sibling to the current object or null if there is no previous node. Reset

UiNode Reset() Releases any resources used internally by a UiNode and brings the node back to uninitialized state.

Return the current node making method chaining possible. Remarks

For an uninitialized node, Click and Hover use screen coordinates to simulate mouse actions. Also WriteText will simulate keystrokes on the window that has the focus.

Scrape

UiScrapeResult Scrape(UiScrapeOptions options) Extracts text from the current UiNode. Learn more about UiScrapeResult.

Parameters options UiScrapeOptions object that specifies how to extract the text.

Return a UiScrapeResult object that contains the scraped text and some additional information. Errors E_INVALIDARG error is raised for invalid input parameters. Remarks

If the UiNode object has a clippingRegion set then only that region is scraped. The method returns null if the UI object text cannot be extracted with the specified scrape options. If UI_AUTOMATIC is used then, after execution, the options parameter will contain the scraping method choosen to extract the text.

Screenshot

UiImage Screenshot() Retrieves a screenshot of the current UI object.

Return UiImage object representing the screenshot of the current UI object. Remarks The screenshot takes into account clippingRegion property and it's cropped to that region. SelectInteractive

UiNode SelectInteractive(UiSelectionType selectionType, bool hidePreview = false) Initializes the internal data of the current UiNode with the UI object or region the user selects on screen. This method requires interaction with the user. Learn more about UiSelectionType.

Parameters selectionType a UiSelectionType that specifies the default section mode

hidePreview option to not show selection informative zoomed panel (default false). This parameter is new in v6

Return the current node making method chaining possible. Errors E_UINODE_SELECTION_INTERNAL_ERR error is raised if selection is canceled.

Remarks

For rectangular region selections, the clippingRegion of the current UiNode represents the UiRegion relative to UiNode's screen position, similar to FromScreenRegion

Set

UiNode Set(string attr, object value) Sets the value of the specified attribute.

Parameters attr
the name of the attribute

value the value to be set

Errors

E_UINODE_INVALID_ATTRIBUTE is raised if attr is not supported or is read-only.

Remarks

Setting the value of an attribute changes the state of the underlying UI object on the screen (e.g. selectedItem really selects the item inside the combo or list control). For attributes type and name please check the Attributes section.

SetFocus

UiNode SetFocus() Sets the keyboard focus to the current UI object

Return the current node making method chaining possible. ShowCaret

void ShowCaret() Makes the caret visible on the screen at the caret's current position. This method is not yet implemented!

Remarks When the caret becomes visible, it begins flashing automatically. StartHighlight

UiNode StartHighlight() Shows a visual highlighting rectangle around the UiNode.

Return the current node making method chaining possible. Remarks

This method is useful for debugging purposes. The highlight rectangle takes into account clippingRegion property.

StopHighlight

UiNode StopHighlight() Removes the visual highlighting rectangle around the UiNode.

Return the current node making method chaining possible. Remarks

This method is useful for debugging purposes. It removes the visual highlighting rectangle previously displayed by StartHighlight.

TopParent

UiNode TopParent() Retrieves the top parent node of the current UiNode.

Return top parent node. Remarks The top parent is a top level window. WaitAttribute

UiNode WaitAttribute(string attr, object value) Waits for the attr attribute of the current UI object to meet a specified value.

Parameters attr
the attribute name

value the required attribute value the method is waiting for

Return the current node making method chaining possible. Errors

E_UINODE_WFR_TIMEOUT_ERR is raised when the attribute does not meet the required value in timeout.

E_UINODE_CANCELED_ERR is raised when the wait is canceled (see Cancel event).

Remarks

The method waits the amount of time specified by timeout property.

WaitDisappear

void WaitDisappear() Waits for the current UI object to disappear from screen.

Remarks The method waits the amount of time specified by timeout property.

WaitForReady

UiNode WaitForReady(UiWaitForReadyLevels readyLevel) Waits for the UI object to meet a given ready level.

Learn more about UiWaitForReadyLevels.

Parameters readyLevel - the level to be reached. Return the current node making method chaining possible. Errors

E_UINODE_WFR_TIMEOUT_ERR is raised when the UI object does not meet the required readyLevel in timeout.

E_UINODE_CANCELED_ERR is raised when the wait is canceled (see Cancel event).

Remarks

The method waits for readyLevel the amount of time specified by timeout property.

WaitImage

UiNode WaitImage(UiImage Image, double Accuracy, bool Appear) Waits for a UiImage to appear or dissapear in the amount of time specified by timeout property

Parameters Image
represents the UiImage object you are waiting for

Accuracy
the precentage of similarity used to find the UiImage

Appear
specifies the event you are waiting for

Return the current node making method chaining possible. Errors

E_UINODE_TIMEOUT_ERR If time elapsed and the UiImage didn't appear/disappear then this error is raised

E_INVALIDARG error is raised for invalid input parameters. Remarks

Accuracy parameter is a double between 0 and 100. If Appear is 0 then the method waits for a UiImage to disappear, else it waits for the UiImage to appear.

WriteText

UiNode WriteText(string text, UiInputMethod inputMethod) Puts text in the UI object identified by the current UiNode.

Parameters text
the text to write into the control

inputMethod hardware events, Win32 messages or node-dependent (see: UiInputMethod)

Return the current node making method chaining possible. Errors

E_UINODE_KEYS_SYNTAX is raised when the special syntax in text parameter is incorrect (see Remarks).

E_UINODE_CANNOT_USE_CTRL_API_METHOD is raised when UI_CONTROL_API method failed.

E_UINODE_KEYS_OTHER_ERR is raised when WriteText fails due to unknown errors in the underlying automation technology (like Active Accessibility, Java Accessibility Bridge, HTML events).

Remarks

When UI_HARDWARE_EVENTS method is specified, special characters like Alt, Shift, Ctrl can be simulated. In order to specify such a character the following convention is used:

A special characters sequence is enclosed in []. A sequence of special characters consists of a sequence of tokens that describe the key that is pressed or released in order to synthesize that character. A token has the form TOKEN:=OPERATION(SPECIAL_CHARS) OPERATION:=k|d|u SPECIAL_CHARS:=alt|lalt|ralt|shift|lshift|rshift|ctrl|lctrl|rctrl|lwin|rwin|ins|del|home|end|pgup|pgdn|enter| left|right|up|down|tab|esc|back|pause|f1|f2|f3|f4|f5|f6|f7|f8|f9|f10|f11|f12|caps|num|add| sub|mul|div|decimal|break|num0|num1|num2|num3|num4|num5|num6|num7|num8|num9|numenter|scroll|sleep k:=synthesize a key down followed by a key up event for the specified key. d:=synthesize a key down event for the specified SPECIAL_CHARS. u:=synthesize a key up event for the specified SPECIAL_CHARS. Example: text = "AbC[d(ctrl)]a[u(ctrl)k(alt)]" This sample sequence outputs "AbC", then holds down (d) the Ctrl key, presses "a" and then releases Ctrl (u) and hits Alt (k = press and release).