Python control mouse keyboard

Welcome to PyAutoGUI’s documentation!¶

PyAutoGUI lets your Python scripts control the mouse and keyboard to automate interactions with other applications. The API is designed to be simple. PyAutoGUI works on Windows, macOS, and Linux, and runs on Python 2 and 3.

To install with pip, run pip install pyautogui . See the Installation page for more details.

PyAutoGUI has several features:

• Moving the mouse and clicking in the windows of other applications.
• Sending keystrokes to applications (for example, to fill out forms).
• Take screenshots, and given an image (for example, of a button or checkbox), and find it on the screen.
• Locate an application’s window, and move, resize, maximize, minimize, or close it (Windows-only, currently).
• Display alert and message boxes.

Here’s a YouTube video of a bot automatically playing the game Sushi Go Round. The bot watches the game’s application window and searches for images of sushi orders. When it finds one, it clicks the ingredient buttons to make the sushi. It also clicks the phone in the game to order more ingredients as needed. The bot is completely autonomous and can finish all seven days of the game. This is the kind of automation that PyAutoGUI is capable of.

Examples¶

>>> import pyautogui >>> screenWidth, screenHeight = pyautogui.size() # Get the size of the primary monitor. >>> screenWidth, screenHeight (2560, 1440) >>> currentMouseX, currentMouseY = pyautogui.position() # Get the XY position of the mouse. >>> currentMouseX, currentMouseY (1314, 345) >>> pyautogui.moveTo(100, 150) # Move the mouse to XY coordinates. >>> pyautogui.click() # Click the mouse. >>> pyautogui.click(100, 200) # Move the mouse to XY coordinates and click it. >>> pyautogui.click('button.png') # Find where button.png appears on the screen and click it. >>> pyautogui.move(400, 0) # Move the mouse 400 pixels to the right of its current position. >>> pyautogui.doubleClick() # Double click the mouse. >>> pyautogui.moveTo(500, 500, duration=2, tween=pyautogui.easeInOutQuad) # Use tweening/easing function to move mouse over 2 seconds. >>> pyautogui.write('Hello world!', interval=0.25) # type with quarter-second pause in between each key >>> pyautogui.press('esc') # Press the Esc key. All key names are in pyautogui.KEY_NAMES >>> with pyautogui.hold('shift'): # Press the Shift key down and hold it. pyautogui.press(['left', 'left', 'left', 'left']) # Press the left arrow key 4 times. >>> # Shift key is released automatically. >>> pyautogui.hotkey('ctrl', 'c') # Press the Ctrl-C hotkey combination. >>> pyautogui.alert('This is the message to display.') # Make an alert box appear and pause the program until OK is clicked. 

This example drags the mouse in a square spiral shape in MS Paint (or any graphics drawing program):

>>> distance = 200 >>> while distance > 0: pyautogui.drag(distance, 0, duration=0.5) # move right distance -= 5 pyautogui.drag(0, distance, duration=0.5) # move down pyautogui.drag(-distance, 0, duration=0.5) # move left distance -= 5 pyautogui.drag(0, -distance, duration=0.5) # move up 

The benefit of using PyAutoGUI, as opposed to a script that directly generates the image file, is that you can use the brush tools that MS Paint provides.

FAQ: Frequently Asked Questions¶

Q: Can PyAutoGUI work on Android, iOS, or tablet/smartphone apps.

A: Unfortunately no. PyAutoGUI only runs on Windows, macOS, and Linux.

Q: Does PyAutoGUI work on multi-monitor setups.

A: No, right now PyAutoGUI only handles the primary monitor.

Q: Does PyAutoGUI do OCR?

A: No, but this is a feature that’s on the roadmap.

Q: Can PyAutoGUI do keylogging, or detect if a key is currently pressed down?

A: No, PyAutoGUI cannot do this currently.

Fail-Safes¶

Like the enchanted brooms from the Sorcerer’s Apprentice programmed to keep filling (and then overfilling) the bath with water, a bug in your program could make it go out of control. It’s hard to use the mouse to close a program if the mouse cursor is moving around on its own.

As a safety feature, a fail-safe feature is enabled by default. When a PyAutoGUI function is called, if the mouse is in any of the four corners of the primary monitor, they will raise a pyautogui.FailSafeException . There is a one-tenth second delay after calling every PyAutoGUI functions to give the user time to slam the mouse into a corner to trigger the fail safe.

You can disable this failsafe by setting pyautogui.FAILSAFE = False . I HIGHLY RECOMMEND YOU DO NOT DISABLE THE FAILSAFE.

The tenth-second delay is set by the pyautogui.PAUSE setting, which is 0.1 by default. You can change this value. There is also a pyautogui.DARWIN_CATCH_UP_TIME setting which adds an additional delay on macOS after keyboard and mouse events, since the operating system appears to need a delay after PyAutoGUI issues these events. It is set to 0.01 by default, adding an additional hundredth-second delay.

• Installation
  • Windows
  • macOS
  • Linux
  • General Functions
  • Fail-Safes
  • Mouse Functions
  • Keyboard Functions
  • Message Box Functions
  • Screenshot Functions
  • The Screen and Mouse Position
  • Mouse Movement
  • Mouse Drags
  • Tween / Easing Functions
  • Mouse Clicks
  • The mouseDown() and mouseUp() Functions
  • Mouse Scrolling
  • The write() Function
  • The press(), keyDown(), and keyUp() Functions
  • The hold() Context Manager
  • The hotkey() Function
  • KEYBOARD_KEYS
  • The alert() Function
  • The confirm() Function
  • The prompt() Function
  • The password() Function
  • The screenshot() Function
  • The Locate Functions
  • Platforms Tested
  • pyautogui package

  This documentation is still a work in progress.

  Источник

  PyAutoGUI 0.9.54

  PyAutoGUI lets Python control the mouse and keyboard, and other GUI automation tasks. For Windows, macOS, and Linux, on Python 3 and 2.

  Навигация

  Ссылки проекта

  Статистика

  Метаданные

  Лицензия: BSD License (BSD)

  Метки gui, automation, test, testing, keyboard, mouse, cursor, click, press, keystroke, control

  Сопровождающие

  Классификаторы

  Описание проекта

  PyAutoGUI

  PyAutoGUI is a cross-platform GUI automation Python module for human beings. Used to programmatically control the mouse & keyboard.

  If you need help installing Python, visit https://installpython3.com/

  Dependencies

  PyAutoGUI supports Python 2 and 3. If you are installing PyAutoGUI from PyPI using pip:

  Windows has no dependencies. The Win32 extensions do not need to be installed.

  macOS needs the pyobjc-core and pyobjc module installed (in that order).

  Linux needs the python3-xlib (or python-xlib for Python 2) module installed.

  Pillow needs to be installed, and on Linux you may need to install additional libraries to make sure Pillow’s PNG/JPEG works correctly. See:

  https://stackoverflow.com/questions/7648200/pip-install-pil-e-tickets-1-no-jpeg-png-support http://ubuntuforums.org/showthread.php?t=1751455 

  If you want to do development and contribute to PyAutoGUI, you will need to install these modules from PyPI:

  Example Usage

  Keyboard and Mouse Control

  The x, y coordinates used by PyAutoGUI has the 0, 0 origin coordinates in the top left corner of the screen. The x coordinates increase going to the right (just as in mathematics) but the y coordinates increase going down (the opposite of mathematics). On a screen that is 1920 x 1080 pixels in size, coordinates 0, 0 are for the top left while 1919, 1079 is for the bottom right.

  Currently, PyAutoGUI only works on the primary monitor. PyAutoGUI isn’t reliable for the screen of a second monitor (the mouse functions may or may not work on multi-monitor setups depending on your operating system and version).

  All keyboard presses done by PyAutoGUI are sent to the window that currently has focus, as if you had pressed the physical keyboard key.

  (PyAutoGUI uses Pillow for image-related features.)

  You can also locate where an image is on the screen:

  The locateCenterOnScreen() function returns the center of this match region:

  How Does PyAutoGUI Work?

  The three major operating systems (Windows, macOS, and Linux) each have different ways to programmatically control the mouse and keyboard. This can often involve confusing, obscure, and deeply technical details. The job of PyAutoGUI is to hide all of this complexity behind a simple API.

  • On Windows, PyAutoGUI accesses the Windows API (also called the WinAPI or win32 API) through the built-in ctypes module. The nicewin module at https://github.com/asweigart/nicewin provides a demonstration for how Windows API calls can be made through Python.
  • On macOS, PyAutoGUI uses the rubicon-objc module to access the Cocoa API.
  • On Linux, PyAutoGUI uses the Xlib module to access the X11 or X Window System.

  Support

  If you find this project helpful and would like to support its development, consider donating to its creator on Patreon.

  Источник

  How to control your mouse and keyboard using the pynput library in Python

  The pynput library allows you to control and monitor/listen to your input devices such as they keyboard and mouse.

  The pynput.mouse allows you control and monitor the mouse, while the pynput.keyboard allows you to control and monitor the keyboard.

  In this article, we will be moving the cursor to a specific position, automate clicks, and simulate keystrokes from the keyboard.

  Without further ado, let’s get started.

  Getting Started

  Since the pynput module does not come packaged with Python, you will have to manually download and install it using the pip package manager.

  To do this, launch your terminal and use the command below.

  Once the library has been successfully downloaded and installed, you are all set to import its various module to your Python script.

  Since we will be importing various modules for both the keyboard and mouse, we will discuss about the import statements later on.

  Controlling the Mouse

  In order to control and simulate your mouse using Python, you must import the mouse module from the pynput library. Since we will be simulating clicks and movement as well, we will import them along with the module.

  from pynput.mouse import Button, Controller

  Firstly, we will be using Controller() method to move the mouse around the screen.

  Now, if you want to move your mouse to any position on your screen, you just provide its coordinates.

  If you want to see where your mouse moved, you can print it.

  print('Current mouse position −> '.format(mouse.position))

  If you want to move the mouse relative to its current position, use the move function.

  If you want to simulate button presses,

  mouse.press(Button.left) mouse.release(Button.left) mouse.press(Button.right) mouse.release(Button.right)

  You can even simulate scrolls using pynput,

  This will scroll two steps down, the x coordinate is used to navigate the scroll from left to right and the y coordinate for top to bottom.

  Example

  from pynput.mouse import Button, Controller mouse = Controller() mouse.position = (50,60) print('Current mouse position −> '.format(mouse.position)) mouse.move(30,15) mouse.press(Button.left) mouse.release(Button.left) mouse.press(Button.right) mouse.release(Button.right) mouse.click(Button.left, 2) mouse.scroll(0,2)

  Using above methods, you can simulate drawing shapes on paint using basic coordinate geometry.

  Controlling the Keyboard

  Firstly, we import the required modules and functions. In the keyboard module within the pynput library, we will be using the Key and Controller functions.

  From pynput.keyboard import Key, Controller

  We will be using the Controller method to control the Keyboard and simulate keystrokes.

  Now, to simulate keystrokes, we have the press and release methods

  keyboard.press('a') keyboard.release('a')

  This works for all alphabets, including upper case. For upper case, you just use “A” instead of “a”.

  You can simulate other keypresses such as ctrl, alt, space etc as well.

  keyboard.press(Key.space) keyboard.release(Key.space) keyboard.press(Key.ctrl) keyboard.release(Key.ctrl)

  If you want to simulate typing in sentences or words at once, you can use the type function.

  And that’s basically how you control or simulate a keyboard device on Python.

  Example

  from pynput.keyboard import Key, Controller keyboard = Controller() keyboard.press('a') keyboard.release('a') keyboard.press(Key.space) keyboard.release(Key.space) keyboard.press(Key.ctrl) keyboard.release(Key.ctrl) keyboard.type('Hello World!!')

  Conclusion

  You now know how to simulate keyboard and mouse input devices using Python’s pynput library.

  Using this, you can build automated bots that perform clicking actions in clicker games, build a spam bot that sends out various different forms of messages at once. The applications are quite limitless as you can build any kind of automation tool using this method.

  If you want to read more about pynput and explore its various other functions, you can check out its official documentation at −

  Источник