fzf – A command-line fuzzy finder πŸŒΈ

Hi πŸ‘‹,

fzf is a command line tool that can be used to fuzzy search files, history, and command outputs. Check out the Github repository.

This is a tool that I wish I’ve knew about earlier in my career, and for this reason I decided to blog about it.

You can also set key bindings; some popular ones are:

CTRL-T – Paste the selected files and directories onto the command-line

CTRL-R – Paste the selected command from history onto the command-line

ALT-C – cd into the selected directory

If you’re on Linux and want to try it, then installing it using Git is straight forward and it also sets up your shell and key bindings.

On Windows, to enable the key bindings you can install the PSFzf module by running Install-Module PSFzf in an admin Powershell.

Edit your $PROFILE file and add the following items:

Remove-PSReadlineKeyHandler 'Ctrl+r'
Remove-PSReadlineKeyHandler 'Ctrl+t'
Import-Module PSFzf

Thanks for reading! 🍻

FastAPI Uvicorn logging in Production

Hello πŸ™‹β€β™‚οΈ,

Running a ⏩FastAPI ⏩ application in production is very easy and fast, but along the way some Uvicorn logs are lost.

In this article I will discuss how to write a custom UvicornWorker and to centralize your logging configuration into a single file.

To keep things as simple as possible I’ve put all my code in a single Python file.

main.py

import uvicorn as uvicorn
from fastapi import FastAPI, APIRouter

router = APIRouter(prefix="")


def create_app():
    fast_app = FastAPI()
    fast_app.include_router(router)
    return fast_app


@router.get("/")
def read_root():
    return {"Hello": "World"}


if __name__ == '__main__':
    app = create_app()
    uvicorn.run(app=app)

Running the code will return a {"Hello": "World"} json when you visit the root endpoint / at http://127.0.0.1:8000. 😁

When you check the console window, the following log lines are printed:

INFO:     Started server process [10276]
INFO:     Waiting for application startup.
INFO:     Application startup complete.
INFO:     Uvicorn running on http://127.0.0.1:8000 (Press CTRL+C to quit)
INFO:     127.0.0.1:53491 - "GET / HTTP/1.1" 200 OK

Notice the Uvicorn log GET / HTTP/1.1″ 200 OK.

According to Uvicorn’s deployment docs we should run Uvicorn in a production settings with the following command: gunicorn -k uvicorn.workers.UvicornWorker main:create_app.

(venv2) ➜  FastAPILogging gunicorn -k uvicorn.workers.UvicornWorker main:create_app
[2021-05-17 22:10:44 +0300] [6250] [INFO] Starting gunicorn 20.1.0
[2021-05-17 22:10:44 +0300] [6250] [INFO] Listening at: http://127.0.0.1:8000 (6250)
[2021-05-17 22:10:44 +0300] [6250] [INFO] Using worker: uvicorn.workers.UvicornWorker
[2021-05-17 22:10:44 +0300] [6252] [INFO] Booting worker with pid: 6252
[2021-05-17 22:10:45 +0300] [6252] [WARNING] ASGI app factory detected. Using it, but please consider setting the --factory flag explicitly.
[2021-05-17 22:10:45 +0300] [6252] [INFO] Started server process [6252]
[2021-05-17 22:10:45 +0300] [6252] [INFO] Waiting for application startup.
[2021-05-17 22:10:45 +0300] [6252] [INFO] Application startup complete.

Now, if we visit the root endpoint, the console won’t print “GET / HTTP/1.1” 200 OK anymore/ πŸ€¦β€β™‚οΈ.

To fix it we need a custom UvicornWorker βš™ and a logging configuration file πŸ—ƒ.

Create a new file and name it logging.yaml, then paste the following contents in it:

version: 1
disable_existing_loggers: false

formatters:
standard:
format: "%(asctime)s - %(levelname)s - %(message)s"

handlers:
console:
class: logging.StreamHandler
formatter: standard
stream: ext://sys.stdout

loggers:
uvicorn:
error:
propagate: true

root:
level: INFO
handlers: [console]
propagate: no

This file will configure our root logger and our Uvicorn logger. To read more on the topic please see Python logging configuration.

Next, we will create a custom UvicornWorker class that will set log_config to the path of our logging.yaml file, to pass the logging configuration that we’ve just made to Uvicorn. πŸ¦„

I added the following code in main.py:

class MyUvicornWorker(UvicornWorker):
    CONFIG_KWARGS = {
        "log_config": "/mnt/c/Users/denis/PycharmProjects/FastAPILogging/logging.yaml",
    }

β–Ά If we run the application with:

gunicorn -k main.MyUvicornWorker main:create_app

We should see the Uvicorn access logs printed in the console πŸ¦„

(venv2) ➜  FastAPILogging gunicorn -k main.MyUvicornWorker main:create_app
[2021-05-17 22:31:28 +0300] [6278] [INFO] Starting gunicorn 20.1.0
[2021-05-17 22:31:28 +0300] [6278] [INFO] Listening at: http://127.0.0.1:8000 (6278)
[2021-05-17 22:31:28 +0300] [6278] [INFO] Using worker: main.MyUvicornWorker
[2021-05-17 22:31:28 +0300] [6280] [INFO] Booting worker with pid: 6280
2021-05-17 22:31:28,185 - WARNING - ASGI app factory detected. Using it, but please consider setting the --factory flag explicitly.
2021-05-17 22:31:28,185 - INFO - Started server process [6280]
2021-05-17 22:31:28,185 - INFO - Waiting for application startup.
2021-05-17 22:31:28,185 - INFO - Application startup complete.
2021-05-17 22:31:30,129 - INFO - 127.0.0.1:54004 - "GET / HTTP/1.1" 200

Thanks for reading! πŸ“š

requirements.txt

click==7.1.2
fastapi==0.65.1
gunicorn==20.1.0
h11==0.12.0
httptools==0.2.0
pydantic==1.8.2
PyYAML==5.4.1
starlette==0.14.2
typing-extensions==3.10.0.0
uvicorn==0.13.4
uvloop==0.15.2

Introduction to Pyenv for Linux Users

Hello,

In this article I will introduce you to pyenv, a tool for managing python environments.

Installing pyenv is pretty straight forward, you’ll need to clone the repo and add the binaries to the path.

For a typical Debian based distro using the Zsh shell the instructions would be:

git clone https://github.com/pyenv/pyenv.git ~/.pyenv

echo 'export PYENV_ROOT="$HOME/.pyenv"' >> ~/.zshrc 
echo 'export PATH="$PYENV_ROOT/bin:$PATH"' >> ~/.zshrc

Then, in order for this to take effect, you need to reload the shell with: source ~/.zshrc, or just restart your terminal. πŸ˜€

For the full installation instructions and shell auto completion refer to: Installation

Basics

While that’s enough to use Pyenv, I also like to install the pyenv-virtualenv plugin, to manage my Python virtual environments. Please refer to the Installation Instructions.

Note: If you use Zsh you can add pyenv to your plugins in .zshrc in order to have command autocomplete, it also loads pyenv-virtualenv.

To list all the available python versions we can run:

➜ ~ pyenv install -l
Available versions:
...
3.8.0
3.8-dev
3.8.1
3.8.2
3.8.3
3.9.0a6
3.9-dev
3.10-dev

Before installing one, we need to download all the python dependencies for our system, if we don’t do that, the Python compilation will most likely fail.

For a Debian based Linux distro the following command should work:

sudo apt-get update; sudo apt-get install --no-install-recommends make build-essential libssl-dev zlib1g-dev libbz2-dev libreadline-dev libsqlite3-dev wget curl llvm libncurses5-dev xz-utils tk-dev libxml2-dev libxmlsec1-dev libffi-dev liblzma-dev

For other distros and macOS please refer to: Suggested build environment

When all the dependencies are met, we can install the desired Python version with: pyenv install 3.9-dev or any other Python version.

We can list our installed python versions with: pyenv versions.

I have previously installed python-3.8.3 and created a virtual environment from it. I’ll explain this later.

To use our newly installed Python version we have two options:

  • Use it globally: pyenv global 3.9-dev
  • Use it locally: pyenv local 3.9-dev

Global usage means that pyenv will use bash magic. When you type python, the configured Python version will be used, if and only if there’s no local version configured.

Local versions are configured per directory basis, which means that if you are in a project’s root folder and you set the version to 3.9-dev, when typing python in the project’s root or nested directories, the local version (3.9-dev) will be used instead of the global one.

For example:

To query the global/local version use pyenv global (or local) without any arguments. To unset the local version use pyenv local –unset.

What we’ve learned so far is that we can carelessly use any python version side by side, without breaking our system, and we can switch between them using a simple command.

Pyenv is also extremely generous, as we can chose between 425 python versions, all of which can be installed with as single command.

➜ pyenv pyenv install -l | wc -l
425

You can also chose to run two global python versions at the same time, this is useful when you need tools written in Python that are distributed as pip packages. One tool that I can think of is docker-compose.

This offers you great flexibility.

Virtual Environments

A problem that arises when working as a Python developer is that often you are working on multiple projects at once.

We can use python 3.8.3 for two projects, but if a project requires and older MongoDB package and another one requires the newer package to work we have a problem, as we can’t have two versions installed at the same time.

This is where virtual environments and pyenv-virtual plugin comes into play.

We can easily create a new virtual environment with the following command, the first argument being the ‘parent’ and the second argument being the virtual environment name.

pyenv virtualenv 3.9-dev experimental-3.9-dev

We can then use the virtual environment locally with:

pyenv local experimental-3.9-dev

If we no longer need a version we can uninstall it with one command:

pyenv uninstall experimental-3.9-dev

Before ending this article I want to show you how to use the a pyenv Python version in Pycharm.

When creating a new Project -> Pure Python -> Virtual Environment -> Click on the dots -> Virtualenv Environment -> Click on the dots -> Find your pyenv version usually in: /home/user/.pyenv/versions -> Select /bin/python and press Ok πŸ™‚

On an existing project, press SHIFT twice and type Python Interpreter.

That’s it! I hope you enjoyed this article.