envsubst – Substitute Variables for Environment Variables

Hi 👋,

Introduction

In this short article I want to showcase a nice and useful Linux command.

The comand 🥁 envsubst 🥁. In short it Substitutes the values of environment variables.

It’s great for populating configuration files with values from environment variables, a common operation for developers containerizing their applications.

Example Usecase

Let’s say we are containerizing an application and we have the following file configuration.yaml, and we want to modify the values of the environment field and the log_level field without adding the additional complexity of mounting the configuration.yaml file into the container/pod.

configuration:
  server: "the-app"
  environment: "production"
  log_level": "info"

To change the values of the environment and loglevel fields we create a configuration-template.yaml like so:

configuration:
    server: "the-app"
    environment: "$ENV_ENVIRONMENT"
    log_level": "$ENV_LOGLEVEL"

Then we run eventsubt to substitute the configuration values:

export ENV_ENVIRONMENT=stagging
export ENV_LOGLEVEL=trace

envsubst < configuration-template.yaml > configuration.yaml

The resulting configuration.yaml file will contain:

configuration:
    server: "the-app"
    environment: "stagging"
    log_level": "trace"%  

Conclusion

The envsubst command enables us to write configuration files with placeholders that will be replaced with actual values from the environment variables. Before being aware if this command I always thought that I have to somehow parse the files, substitute variables dynamically, ensure that the substitution didn’t change the file format or break something. Now I just envsubst .

Thanks for reading and happy dev-ing! 🖥️🧑‍💻

envsubst help

Usage: envsubst [OPTION] [SHELL-FORMAT]

Substitutes the values of environment variables.

Operation mode:
-v, –variables output the variables occurring in SHELL-FORMAT

Informative output:
-h, –help display this help and exit
-V, –version output version information and exit

In normal operation mode, standard input is copied to standard output,
with references to environment variables of the form $VARIABLE or ${VARIABLE}
being replaced with the corresponding values. If a SHELL-FORMAT is given,
only those environment variables that are referenced in SHELL-FORMAT are
substituted; otherwise all environment variables references occurring in
standard input are substituted.

When –variables is used, standard input is ignored, and the output consists
of the environment variables that are referenced in SHELL-FORMAT, one per line.

Report bugs in the bug tracker at https://savannah.gnu.org/projects/gettext
or by email to bug-gettext@gnu.org.

fzf – A command-line fuzzy finder 🌸

Hi 👋,

fzf is a command line tool that can be used to fuzzy search files, history, and command outputs. Check out the Github repository.

This is a tool that I wish I’ve knew about earlier in my career, and for this reason I decided to blog about it.

You can also set key bindings; some popular ones are:

CTRL-T – Paste the selected files and directories onto the command-line

CTRL-R – Paste the selected command from history onto the command-line

ALT-C – cd into the selected directory

If you’re on Linux and want to try it, then installing it using Git is straight forward and it also sets up your shell and key bindings.

On Windows, to enable the key bindings you can install the PSFzf module by running Install-Module PSFzf in an admin Powershell.

Edit your $PROFILE file and add the following items:

Remove-PSReadlineKeyHandler 'Ctrl+r'
Remove-PSReadlineKeyHandler 'Ctrl+t'
Import-Module PSFzf

Thanks for reading! 🍻

Introduction to Pyenv for Linux Users

Hello,

In this article I will introduce you to pyenv, a tool for managing python environments.

Installing pyenv is pretty straight forward, you’ll need to clone the repo and add the binaries to the path.

For a typical Debian based distro using the Zsh shell the instructions would be:

git clone https://github.com/pyenv/pyenv.git ~/.pyenv

echo 'export PYENV_ROOT="$HOME/.pyenv"' >> ~/.zshrc 
echo 'export PATH="$PYENV_ROOT/bin:$PATH"' >> ~/.zshrc

Then, in order for this to take effect, you need to reload the shell with: source ~/.zshrc, or just restart your terminal. 😀

For the full installation instructions and shell auto completion refer to: Installation

Basics

While that’s enough to use Pyenv, I also like to install the pyenv-virtualenv plugin, to manage my Python virtual environments. Please refer to the Installation Instructions.

Note: If you use Zsh you can add pyenv to your plugins in .zshrc in order to have command autocomplete, it also loads pyenv-virtualenv.

To list all the available python versions we can run:

➜ ~ pyenv install -l
Available versions:
...
3.8.0
3.8-dev
3.8.1
3.8.2
3.8.3
3.9.0a6
3.9-dev
3.10-dev

Before installing one, we need to download all the python dependencies for our system, if we don’t do that, the Python compilation will most likely fail.

For a Debian based Linux distro the following command should work:

sudo apt-get update; sudo apt-get install --no-install-recommends make build-essential libssl-dev zlib1g-dev libbz2-dev libreadline-dev libsqlite3-dev wget curl llvm libncurses5-dev xz-utils tk-dev libxml2-dev libxmlsec1-dev libffi-dev liblzma-dev

For other distros and macOS please refer to: Suggested build environment

When all the dependencies are met, we can install the desired Python version with: pyenv install 3.9-dev or any other Python version.

We can list our installed python versions with: pyenv versions.

I have previously installed python-3.8.3 and created a virtual environment from it. I’ll explain this later.

To use our newly installed Python version we have two options:

  • Use it globally: pyenv global 3.9-dev
  • Use it locally: pyenv local 3.9-dev

Global usage means that pyenv will use bash magic. When you type python, the configured Python version will be used, if and only if there’s no local version configured.

Local versions are configured per directory basis, which means that if you are in a project’s root folder and you set the version to 3.9-dev, when typing python in the project’s root or nested directories, the local version (3.9-dev) will be used instead of the global one.

For example:

To query the global/local version use pyenv global (or local) without any arguments. To unset the local version use pyenv local –unset.

What we’ve learned so far is that we can carelessly use any python version side by side, without breaking our system, and we can switch between them using a simple command.

Pyenv is also extremely generous, as we can chose between 425 python versions, all of which can be installed with as single command.

➜ pyenv pyenv install -l | wc -l
425

You can also chose to run two global python versions at the same time, this is useful when you need tools written in Python that are distributed as pip packages. One tool that I can think of is docker-compose.

This offers you great flexibility.

Virtual Environments

A problem that arises when working as a Python developer is that often you are working on multiple projects at once.

We can use python 3.8.3 for two projects, but if a project requires and older MongoDB package and another one requires the newer package to work we have a problem, as we can’t have two versions installed at the same time.

This is where virtual environments and pyenv-virtual plugin comes into play.

We can easily create a new virtual environment with the following command, the first argument being the ‘parent’ and the second argument being the virtual environment name.

pyenv virtualenv 3.9-dev experimental-3.9-dev

We can then use the virtual environment locally with:

pyenv local experimental-3.9-dev

If we no longer need a version we can uninstall it with one command:

pyenv uninstall experimental-3.9-dev

Before ending this article I want to show you how to use the a pyenv Python version in Pycharm.

When creating a new Project -> Pure Python -> Virtual Environment -> Click on the dots -> Virtualenv Environment -> Click on the dots -> Find your pyenv version usually in: /home/user/.pyenv/versions -> Select /bin/python and press Ok 🙂

On an existing project, press SHIFT twice and type Python Interpreter.

That’s it! I hope you enjoyed this article.

Brute-forcing passwords with Hydra

Hello,

In this article you will find the solutions for the Hydra room on Try Hack Me.

Link: https://tryhackme.com/room/hydra

Almost every bruteforcing or cracking task from HackTheBox/TryHackMe uses the rockyou.txt wordlist. You can google it and download it, if you can’t find it in your Kali distro under the path /usr/share/wordlists/rockyou.txt.gz

If you’re using Kali then you have Hydra already installed, all you have to do is run two commands:

Flag 1

Run the following command, the password should be cracked in less than one minute.

root@kali:~/Desktop# hydra -l molly -P rockyou.txt 10.10.86.187 http-post-form "/login:username=^USER^&password=^PASS^:F=incorrect" -V

Flag 2

Start bruteforcing the ssh password then logon via ssh as molly to grab the flag.

root@kali:~/Desktop# hydra -l molly -P rockyou.txt 10.10.86.187 -t 4 ssh

Thanks for reading!