AWS Lambda Environment Variables Best Practices
According to the official AWS Lambda documentation, environment variables are a tool that allows you to adjust your function’s behavior without updating the lambda’s code.
“An environment variable is a pair of strings that is stored in a function’s version-specific configuration.
Their values are set during the Lambda deployment time and can only be changed by redeploying the Lambda with the updated values.
Environment variables are seen as basic utility and pretty much everybody is using them for simple configuration values that change values between AWS accounts — dev/test/production values for instance.
However, despite being such an integral and basic part of any Lambda’s deployment, many engineers use them in an unsafe manner which can cause nasty bugs or even crashes in production.
In this blog, I will show you how to properly parse, validate and use your environment variables in your Python AWS Lambda. And while the code examples are written in Python, the presented principles are relevant to all programming languages.
The code in this post has been moved to an open source project you can use: The AWS environment variables modeler:
https://github.com/ran-isenberg/aws-lambda-env-modeler
Python environment variables are stored in a dictionary in the ‘os’ module— os.environ. These variables, sets of string keys and values, can be accessed by calling os.getenv(‘my_var_name’).
If ‘my_var_name’ is not defined as an environment variable, this function will return a None object instead of a string.
Why is this important? Why should you care?
Many people access os.getenv sporadically and all over the AWS Lambda files. However, they usually do not check that the values are not None or that they have valid values (valid ARN, valid Https rest endpoint etc.) and just assume it’s ok since their tests did not fail.
If your test coverage is truly amazing and you cover every single os.getenv, you might be out of hazard’s reach. However, if that’s not the case, you might have a horrible crash/bug lurking around in your code, just waiting for a misconfiguration error in your AWS Lambda deployment code or configuration.
Ok, let’s head over to the proposed solution.
The best practice for handling environment variables is to validate & parse them as soon as the AWS Lambda is triggered. If there is a misconfiguration, a validation exception will be raised with all the relevant details. In addition, the parsed variables must be globally accessible from anywhere in the code.
This can be accomplished by using Pydantic, a performance-oriented parsing and validation library. Read more about Pydantic in my first blog.
We will define the environment variables as a Pydantic schema class and supply an os.enviorn dictionary as an input to the Pydantic parser. Pydantic will raise a very detailed exception in case one or more parameters fails the validation. You can read more about the validation capabilities here.
In addition, we would like to be able to access this configuration in any Lambda files or inner functions in the same agility that calling os.getenv gives us, BUT in a secure manner. This will be achieved by calling a getter function that will return a global instance of the parsed configuration.
Let’s start by defining our environment variables, their names, types and value constraints. In the example below we define a Pydantic Data class MyEnvSchema, which holds 3 environment variables: LOG_LEVEL, ROLE_ARN and REST_ENDPOINT.
MyEnvSchema extends the default Pydantic BaseModel class as defined here.
This schema will make sure that:
- LOG_LEVEL is one of the strings in [‘INFO’, ‘DEBUG’, ‘ERROR’] list.
- ROLE_ARN exists and is between 20 and 2048 characters long as defined here.
- REST_ENDPOINT is a valid HTTP URL.
How it works
We will use the imported lambda_handler_decorator from the AWS Lambda Powertools repository. You can read more about it here.
This will turn init_environment_variables function into a decorator that will be used on the AWS Lambda handler. As defined in lines 7 and 14, the decorator receives a generic model argument which can be any Pydantic BaseModel type of class name (not instance).
At line 10 in the example below, we define a global instance, ENV_CONF for the parsed configuration. init_environment_variables will initialize this instance.
At line 18 we pass the os.environ dictionary as kwargs to Pydantic’s constructor of the model class. Pydantic will raise a ValidationError or TypeError in case of a misconfiguration (i.e missing variables or if the value constraints we defined are not matched — REST_ENDPOINT is not a valid HTTP URL etc.).
Once the code gets to line 22, the global instance ENV_CONF is parsed and validated and the Lambda handler can be triggered safely.
The second function, get_environment_variables will return the ENV_CONF global variable once it is initialized. It can be called from anywhere in the AWS Lambda handler code including inner functions.
Putting it all together
At line 8 the init_environment_variables decorator is set on the handler with model=MyEnvSchema which we defined earlier. This will validate, parse and initialize a global instance of the Pydantic class.
At line 10, we are inside the lambda handler and can now safely call the get_environment_variables getter and use the environment variables data class.
That’s it!
While this solution is specific to Python, the described methods can be written in any other language.
Want to learn more about Serverless?
Check out my website https://www.ranthebuilder.cloud
About Me
Hi, I’m Ran Isenberg, an AWS Community Builder (Serverless focus), a Cloud System Architect and a public speaker based in Israel.
I see myself as a passionate Serverless advocate with a love for innovation, AWS and smart home applications.
