Python API¶

Windows¶

class RPA.Windows.Windows¶

Bases: robotlibcore.DynamicCore

The Windows is a library that can be used for Windows desktop automation.

This library is at this moment in “BETA” stage as an alternative library for RPA.Desktop.Windows. Main difference being that this library is using uiautomation dependency instead of pywinauto.

Installation

This library, RPA.Windows is available via rpaframework-windows package. This package is first available as a separate package installation from rpaframework package. In the next stage this is integrated into rpaframework package. Ultimately this library will replace current library RPA.Desktop.Windows if seen as approriate.

channels:
  - conda-forge
dependencies:
  - python=3.7.5
  - pip=20.1
  - pip:
    - rpaframework-windows==1.1.1

About terminology

ControlType is a value referred to by locator keys type: or control. Represents type of application object, which can be e.g. Window, Button or ListItem.

Element is an entity of an application structure (e.g. certain button in a window), which can be identified by a locator.

WindowsElement is an library container object for the Element. All keywords returning element, return element as WindowsElement and all keywords taking in locator or root_element arguments accept WindowsElement as an argument value.

Structure of the WindowsElement

class WindowsElement:
    item: Control        # ``item`` contains object instance of the element
    locator: str         # ``locator`` that found this element
    name: str            # ``Name`` attribute of the element
    automation_id: str   # ``AutomationId`` attribute of the element
    control_type: str    # ``ControlTypeName`` attribute of the element
    class_name: str      # ``ClassName`` attribute of the element

Example of the WindowsElement usage

${rows}=    Get Elements    class:DataGridRow
# ${rows} is a list of ``WindowsElement``s
FOR    ${row}    IN    @{rows}
    Log To Console   ${row.name}                # access ``WindowsElement``
    Log To Console   ${row.item.AutomationId}   # access ``WindowsElement.item`` directly
    Log To Console   ${row.item.Name}           # same as ``${row.name}``
END

Locators

Locators are based on different strategies that can used identify Control object.

Available strategies that can be used for the locators:

Key	Maps to search property
name	Name
class	ClassName
type	ControlType
control	ControlType
id	AutomationId
automationid	AutomationId
regex	RegexName
subname	SubName
index	foundIndex (int)
offset	offset coordinates (x (int), y (int)) from control center
executable	target window by its executable name
handle	target window handle (int)
desktop	SPECIAL target desktop, no value for the key e.g. desktop and name:Calculator
process	NOT YET SUPPORTED target window by its executable’s process id
depth	searchDepth (int) for finding Control (default 8)

About root element on locators

Locators work on currently active root element. At the start root element is the whole desktop. There are different ways on changing this root element.

Keyword Control Window is the most common method of setting certain system window as a root element for further actions using locators.

Locators themselves support cascading syntax (denoted by character > in the locator string), which can denote root element in “parent (root) & child” terms.

For example.

Click  id:controls > id:activate

On the above example the left side of the > character, id:controls, represents the root element (can be called as “parent element” in this case). Right side of the locator string, id:activate, represents “child” element and it will be searched under the “parent element”.

This way element search is more efficient, because search are restricted to certain section of element tree which can be quite huge especially on the desktop level and in certain applications.

Keyword examples:

Control Window    name:Calculator
Control Window    Calculator  # will execute search by 'name:Calculator'
Control Window    executable:Spotify.exe

some example locators, and can be omitted ie. space ` ` between locator keys means the same thing as and:

id:clearButton
type:Group and name:'Number pad' > type:Button and index:4
type:Group and name:'Number pad' > control:Button index:5
id:Units1 > name:${unit}
class:Button offset:370,0

About locator restrictions

Visual locators are not supported in this library and they can’t be used in the same chain with these Windows locators. Visual locators are supported by the RPA.Desktop library. Locator chaining (image and Windows locators) support will be added in the future.

Locator syntax does not yet support OR operation (only AND operations).

About search depth

The library does element search depth by default to the level of 8. This means that locator will look into 8 levels of elements under element tree of the root element. This can lead into situation where element can’t be found. To fix this it is recommended to set root element which can be found within 8 levels OR defining depth in the locator string to a bigger value, e.g. id:deeplyNestedButton depth:16. Useful keywords for setting root element are Control Window, Set Anchor and Get Element.

Keyboard and mouse

Keys for the keyboard actions are given using uiautomation specification.

Special keys which are given within {} syntax.

Key	Maps to action
LBUTTON	Left mouse button
RBUTTON	Right mouse button
CANCEL	Control-break processing
MBUTTON	Middle mouse button (three-button mouse)
XBUTTON1	X1 mouse button
XBUTTON2	X2 mouse button
BACK	BACKSPACE key
TAB	TAB key
CLEAR	CLEAR key
RETURN	ENTER key
ENTER	ENTER key
SHIFT	SHIFT key
CTRL	CTRL key
CONTROL	CTRL key
ALT	ALT key
PAUSE	PAUSE key
CAPITAL	CAPS LOCK key
KANA	IME Kana mode
HANGUEL	IME Hanguel mode (maintained for compatibility; use VK_HANGUL)
HANGUL	IME Hangul mode
JUNJA	IME Junja mode
FINAL	IME final mode
HANJA	IME Hanja mode
KANJI	IME Kanji mode
ESC	ESC key
ESCAPE	ESC key
CONVERT	IME convert
NONCONVERT	IME nonconvert
ACCEPT	IME accept
MODECHANGE	IME mode change request
SPACE	SPACEBAR
PRIOR	PAGE UP key
PAGEUP	PAGE UP key
NEXT	PAGE DOWN key
PAGEDOWN	PAGE DOWN key
END	END key
HOME	HOME key
LEFT	LEFT ARROW key
UP	UP ARROW key
RIGHT	RIGHT ARROW key
DOWN	DOWN ARROW key
SELECT	SELECT key
PRINT	PRINT key
EXECUTE	EXECUTE key
SNAPSHOT	PRINT SCREEN key
PRINTSCREEN	PRINT SCREEN key
INSERT	INS key
INS	INS key
DELETE	DEL key
DEL	DEL key
HELP	HELP key
WIN	Left Windows key (Natural keyboard)
LWIN	Left Windows key (Natural keyboard)
RWIN	Right Windows key (Natural keyboard)
APPS	Applications key (Natural keyboard)
SLEEP	Computer Sleep key
NUMPAD0	Numeric keypad 0 key
NUMPAD1	Numeric keypad 1 key
NUMPAD2	Numeric keypad 2 key
NUMPAD3	Numeric keypad 3 key
NUMPAD4	Numeric keypad 4 key
NUMPAD5	Numeric keypad 5 key
NUMPAD6	Numeric keypad 6 key
NUMPAD7	Numeric keypad 7 key
NUMPAD8	Numeric keypad 8 key
NUMPAD9	Numeric keypad 9 key
MULTIPLY	Multiply key
ADD	Add key
SEPARATOR	Separator key
SUBTRACT	Subtract key
DECIMAL	Decimal key
DIVIDE	Divide key
F1	F1 key
F2	F2 key
F3	F3 key
F4	F4 key
F5	F5 key
F6	F6 key
F7	F7 key
F8	F8 key
F9	F9 key
F10	F10 key
F11	F11 key
F12	F12 key
F13	F13 key
F14	F14 key
F15	F15 key
F16	F16 key
F17	F17 key
F18	F18 key
F19	F19 key
F20	F20 key
F21	F21 key
F22	F22 key
F23	F23 key
F24	F24 key
NUMLOCK	NUM LOCK key
SCROLL	SCROLL LOCK key
LSHIFT	Left SHIFT key
RSHIFT	Right SHIFT key
LCONTROL	Left CONTROL key
LCTRL	Left CONTROL key
RCONTROL	Right CONTROL key
RCTRL	Right CONTROL key
LALT	Left MENU key
RALT	Right MENU key
BROWSER_BACK	Browser Back key
BROWSER_FORWARD	Browser Forward key
BROWSER_REFRESH	Browser Refresh key
BROWSER_STOP	Browser Stop key
BROWSER_SEARCH	Browser Search key
BROWSER_FAVORITES	Browser Favorites key
BROWSER_HOME	Browser Start and Home key
VOLUME_MUTE	Volume Mute key
VOLUME_DOWN	Volume Down key
VOLUME_UP	Volume Up key
MEDIA_NEXT_TRACK	Next Track key
MEDIA_PREV_TRACK	Previous Track key
MEDIA_STOP	Stop Media key
MEDIA_PLAY_PAUSE	Play/Pause Media key
LAUNCH_MAIL	Start Mail key
LAUNCH_MEDIA_SELECT	Select Media key
LAUNCH_APP1	Start Application 1 key
LAUNCH_APP2	Start Application 2 key
OEM_1	Used for miscellaneous characters; it can vary by keyboard.For the US standard keyboard, the ‘;:’ key
OEM_PLUS	For any country/region, the ‘+’ key
OEM_COMMA	For any country/region, the ‘,’ key
OEM_MINUS	For any country/region, the ‘-‘ key
OEM_PERIOD	For any country/region, the ‘.’ key
OEM_2	Used for miscellaneous characters; it can vary by keyboard.
OEM_3	Used for miscellaneous characters; it can vary by keyboard.
OEM_4	Used for miscellaneous characters; it can vary by keyboard.
OEM_5	Used for miscellaneous characters; it can vary by keyboard.
OEM_6	Used for miscellaneous characters; it can vary by keyboard.
OEM_7	Used for miscellaneous characters; it can vary by keyboard.
OEM_8	Used for miscellaneous characters; it can vary by keyboard.
OEM_102	Either the angle bracket key or the backslash key on the RT 102-key keyboard
PROCESSKEY	IME PROCESS key
PACKET	Used to pass Unicode characters as if they were keystrokes. The VK_PACKET key is the low word of a 32-bit Virtual Key value used for non-keyboard input methods. For more information, see Remark in KEYBDINPUT, SendInput, WM_KEYDOWN, and WM_KeyUp
ATTN	Attn key
CRSEL	CrSel key
EXSEL	ExSel key
EREOF	Erase EOF key
PLAY	Play key
ZOOM	Zoom key
NONAME	Reserved
PA1	PA1 key
OEM_CLEAR	Clear key

Examples.

lib = Windows()
# {Ctrl}, {Delete} ... are special keys' name in SpecialKeyNames.
lib.send_keys('{Ctrl}a{Delete}{Ctrl}v{Ctrl}s{Ctrl}{Shift}s{Win}e{PageDown}') #press Ctrl+a, Delete, Ctrl+v, Ctrl+s, Ctrl+Shift+s, Win+e, PageDown
lib.send_keys('{Ctrl}(AB)({Shift}(123))') #press Ctrl+A+B, type '(', press Shift+1+2+3, type ')', if '()' follows a hold key, hold key won't release util ')'
lib.send_keys('{Ctrl}{a 3}') #press Ctrl+a at the same time, release Ctrl+a, then type 'a' 2 times
lib.send_keys('{a 3}{B 5}') #type 'a' 3 times, type 'B' 5 times
lib.send_keys('{{}Hello{}}abc {a}{b}{c} test{} 3}{!}{a} (){(}{)}') #type: '{Hello}abc abc test}}}!a ()()'
lib.send_keys('0123456789{Enter}')
lib.send_keys('ABCDEFGHIJKLMNOPQRSTUVWXYZ{Enter}')
lib.send_keys('abcdefghijklmnopqrstuvwxyz{Enter}')
lib.send_keys('`~!@#$%^&*()-_=+{Enter}')
lib.send_keys('[]{{}{}}\|;:'",<.>/?{Enter}')

Mouse clicks can be executed with keywords specific for a type of a click, e.g. Click (normal click), Double Click and Right Click.

How to inspect

Most common, and recommended by Microsoft, inspector tool for Windows is Accessibility Insights that can be installed separately. Other options are tools Inspect Object and UI Automation Verify, which can be accessed by installing Windows SDK.

Recording

The package provides some rudimentary inspecting and recording via script windows-record, which can be started in the command line (in a environment containing rpaframework-windows installation).

Recording inspects elements on mouse click and can be stopped with keyboard ESC. Expected console output.

C:\Users\User\robots\>windows-record
keyboard and mouse listeners started

--------------------------------------------------------------------------------
COPY & PASTE BELOW CODE INTO *** Tasks *** or *** Keywords ***
--------------------------------------------------------------------------------

Control Window    Taskbar  # Handle: 131380
Click   name:'Type here to search'
Control Window    Calculator  # Handle: 3411840
Click   name:'Five'
Click   name:'Eight'
Click   name:'Five'

--------------------------------------------------------------------------------

Examples

Both Robot Framework and Python examples follow.

The library must be imported first.

*** Settings ***
Library    RPA.Windows

Windows Calculator task

*** Tasks ***
Do some calculations
    [Setup]  Windows Run   calc.exe
    Control Window    name:Calculator
    Click    id:clearButton
    Send Keys   keys=96+4=
    ${result}=    Get Attribute    id:CalculatorResults    Name
    Log To Console    ${result}
    ${buttons}=  Get Elements  type:Group and name:'Number pad' > type:Button
    FOR  ${button}  IN  @{buttons}
        Log To Console   ${button}
    END
    [Teardown]   Close Current Window

Python example

from RPA.Windows import Windows

library = Windows()

def test_do_some_calculations():
    try:
        library.windows_run("calc.exe")
        library.control_window("name:Calculator")
        library.click("id:clearButton")
        library.send_keys(keys="96+4=")
        result = library.get_attribute("id:CalculatorResults", "Name")
        print(result)
        buttons = library.get_elements("type:Group and name:'Number pad' > type:Button")
        for button in buttons:
            print(button)
    finally:
        library.close_current_window()

ROBOT_LIBRARY_DOC_FORMAT = 'REST'¶

ROBOT_LIBRARY_SCOPE = 'GLOBAL'¶

add_library_components(library_components)¶

get_keyword_arguments(name)¶

get_keyword_documentation(name)¶

get_keyword_names()¶

get_keyword_source(keyword_name)¶

get_keyword_tags(name)¶

get_keyword_types(name)¶

run_keyword(name, args, kwargs=None)¶