Problem: Recently I was working on an ETL(Extract Transform and Load) job, the requirement is to read data from an .spc file and convert it to json and upload it into google’s Bigquery. The task is simple but when I am trying to dockerize it, all I found in is how to deploy a python web based Flask apps only. There is also one more tricky part I need to send a command line argument to python script via docker. So let’s know how I solved this.

Solution:

  1. In the docker file CMD you need to add the following piece of code: CMD [“python”, “dataingest/main.py”, “–config”, “/etc/config/config.json”]
  2. Here dataingest/main.py is your main file, whatever is your starting script you need to enter here.
  3. -config and /etc/config/config.json are my command line arguments, I want to send my environment configuration file as a command line argument. /etc/config/config.json is the path of my config file in the system. -config specifies to python that you are sending a configuration file via as a command line argument.
  4. That’s it now you can run your docker.

Understanding the folder structure

  1. The most important thing is to get folder structure right otherwise you will be lost.
  2. Here is the visual representation of the folder structure I used:

Let’s see the Dockerfile now

FROM python:3.8.3
	
WORKDIR /application
	
# Install and upgrade pip
RUN pip install --upgrade pip
	
# Collect pip requirements
COPY requirements.txt .
	
# Install pip requirements
RUN pip install -r requirements.txt
	
COPY src/ .
	
# Switching to a non-root user
RUN useradd appuser && chown -R appuser /application
USER appuser
	
CMD ["python", "dataingest/main.py", "--config", "/etc/config/config.json"]

Here:

FROM python:3.8.3

This specifies the python version to be downloaded and used.

WORKDIR /application

our application code is saved under this location in the docker system COPY src/ . we are copying our entire src folder and placing it under /application So our src folder will reside in the application folder.

That’s it now you can run your docker now.