Environment variables in python script

Environment Variables in Python

Environment variables are key-value pairs that are defined in your shell environment outside of the Python executable. You can access them in Python with the os.environ mapping object.

Environment variables are really useful for passing secret information or other configurations that you don’t want to pass as command line arguments. A good use case for example is a containerized application running in Docker or Kubernetes where you can configure the application using environment variables.

They can also be used to define the operating mode of your software, for example if some code needs to be executed only in some of your production environments. An environment flag could also be used to enable verbose logging and so on.

Defining environment variables

Your operating system defines many environment variables by default. For example, on my Ubuntu server the PATH and HOME variables look like this:

>> [email protected]:~$ echo $PATH /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin >> [email protected]:~$ echo $HOME /home/janne

As you may have noticed they are written in all capital letters. If your variable names consist of multiple parts you can separate them with underscores.

There are two ways to assign them. The first option is to define the values on the same line with the actual command:

>> [email protected]:~$ MY_ENV_VAR=hello env | grep MY_ENV_VAR MY_ENV_VAR=hello

The above command defines MY_ENV_VAR and then uses the env command to print all environment variables. The output is piped to a program called grep which filters the final output to contain only the line with the given text.

The other option is to use the export command:

>> [email protected]:~$ export MY_OTHER_VAR="Hello again!" >> [email protected]:~$ echo $MY_OTHER_VAR Hello again!

The difference between these options is that export makes the variable available to all commands that are executed afterwards. When the variables are defined on the same line than the command itself it only applies to that single call and is not visible to subsequent program calls.

Reading and writing env in Python

The os.environ object works quite like a normal dictionary and with it you can get and set the environment values. The object is initialized during Python startup so any changes that are made after that are not reflected to the os.environ object unless they are made in the code itself.

Let’s assume that you want to read an API key from the environment. This is a typical use case as you don’t want to add sensitive credentials to source control systems. The key should be defined in your production server environment and fetched during runtime.

This is a minimal example that just prints the value on the console:

import os  API_KEY = os.environ.get("API_KEY") print(API_KEY)

I’ve stored the code to a file called env.py , let’s see what happens if we run it.

The program prints out “None” because the variable has not been set yet. Note that the program doesn’t crash even though the value is missing, the get method just returns None. You can define a default value to be assigned when the variable is not found.

API_KEY = os.environ.get("API_KEY", "mydevkey")

It’s quite obvious how this works:

>> [email protected]:~$ python3 env.py mydevkey >> [email protected]:~$ export API_KEY=deadbeef123badfeed >> [email protected]:~$ python3 env.py deadbeef123badfeed

You can write to the environment as if you were changing a dictionary value.

import os  API_KEY = os.environ.get("API_KEY", "mydevkey") print(API_KEY) os.environ["API_KEY"] = "Overridden!" print(os.environ["API_KEY"])

Note that the value can also be read like a dictionary. Be aware that using square brackets will cause a KeyError if the variable has not been set. Running this code gives the expected result:

>> [email protected]:~$ python3 env.py deadbeef123badfeed Overridden!

As you can see the API_KEY value was overridden. If you set a new value in the code it will be available whenever the value is requested, also in child processes spawned by the Python executable. However, if the value has been read during module import it wont be loaded again unless done explicitly.

But how did this affect our shell?

>> [email protected]:~$ echo $API_KEY deadbeef123badfeed

As you can see the original value is still there. Changes made in Python don’t show up after the execution has ended.

The os module also contains a wrapper module around the environ.get method that saves you a few keystrokes. The equivalent call is

Dotenv files

You can also load the values from a file and then access the values normally. For this purpose there is the python-dotenv package that you can install with:

>> python3 -m pip install -U python-dotenv

Now you can create a file called .env with all of your configurations, for example:

DB_USER=postgres DB_PASSWORD=supersecret

Then, if you create your Python app in the same directory with the environment file you can use this code to read the values.

import os  from dotenv import load_dotenv load_dotenv()  DB_USER = os.environ.get("DB_USER") DB_PASSWORD = os.getenv("DB_PASSWORD")  print(f"User: DB_USER], password: DB_PASSWORD>")

I have stored the code as myapp.py and running it gives the following results:

>> [email protected]:~$ python3 myapp.py User: postgres, password: supersecret

As you can see the values were successfully fetched from the environment file and then printed to the console.

You can also specify the file path manually, override values from the system, or read values from file-like objects. For these and other more advanced use cases you should take a look at the python-dotenv GitHub repo.

Conclusion

Environment files are really powerful and easy! Now you know how to get started using them. You know how to read them, change their values, and how to use dotenv files.

Hopefully you’ve learned something new and found this post helpful. How and where do you use environment files?

Read next in the Python bites series.

Источник

Use cases for Python environment variables

Today’s post focuses on environment variables in Python. They are one of several possible mechanisms for setting various configuration parameters. We can:

• read environment variables (through os.environ or dotenv ) [the current post]
• have the script accept command-line arguments (use argparse )
• load configuration settings from a file, such as:
  • a JSON file (use json )
  • a YAML file (use pyyaml )
  • a XML file (use lxml , ElementTree or minidom )
  • an INI file (use configparser ) [check out this post]
  • your DIY file format (for which you will be rolling your own parser)

  What is the best solution?

  There is no one-size-fits-all solution. It depends on what you’re trying to achieve and on how the current software architecture looks like. If you’re working on a command-line tool that must accommodate a plethora of options, chances are you’ll be using argparse . For other types of projects (such as a server or a client), a configuration file might be more practical. Yet in other situations you may also want to consider using environment variables.

  We will be looking in more detail at three such use cases in this post, where we will see how environment variables can be a good choice.

  But first, let’s get the next point out of the way:

  But environment variables are evil, right?

  Indeed, using environment variables for non-sensitive information that you could just as well transmit via command-line arguments or via a configuration file is not ideal. Why? Because being environment variables, they actually live outside of the code base. Sure, you can access them based on their key (their name) and attach some meaning to them, but this is neither the most Pythonic, nor the most effective way, to do things (if this can be avoided).

  Nevertheless, there are also legit cases where environment variables are preferable:

  • when setting execution mode (e.g. debug or development mode vs production mode)
  • when they improve security practices
  • when they are the only way to get some values into a “black box” (more on that later)

  Before diving into the use cases, let us first briefly see how to access environment variables in Python.

  Accessing environment variables in Python

  Environment variables are read through os.environ . Although they can also be modified or cleared, such changes are only effective in the current Python session (and for subprocesses started with os.system() , popen() , fork() and execv() ). In other words, if you change an environment variable in a Python script, the change will not be reflected in the environment once that script exits.

  os.environ

  In the most simple form, you can export an environment variable through the shell:

  Источник